Dear Kelsey,
First of all, welcome to SAM!
I assume you are using the Flat Plate PV model's "Simple Efficiency Module Model" option to change the module efficiency.
Without seeing your file, it is hard for me to know exactly what might explain this. Keep in mind that it is the system nameplate capacity (kW) that SAM uses to calculate the capacity factor, not the module capacity. It is a combination of the module efficiency and number of modules in the system that determines the system's nameplate capacity, not just the module efficiency (assuming the module area does not change). The system's output (kWh) might also be affected by inverter clipping if both systems have the same inverter capacity, but for some hours of the year, the one with more efficient modules has higher DC output that exceeds the inverter's rated input capacity. Inverter clipping would cause a non-linear relationship between kW capacity and kWh output.
As for a description of the capacity factor calculation, please follow the link in my original post above to see the description in the online copy of SAM's help system.
Best regards,
Paul.