A significant number of existing reinforced concrete (RC) structures are vulnerable to earthquakes due to non-compliance with capacity design principles and inadequate ductile detailing, with substandard columns playing a decisive role in their collapse. This study conducts a comprehensive quantitative assessment of the effectiveness of Carbon Fiber-Reinforced Polymer (CFRP) sheet retrofitting on the seismic performance of non-ductile RC moment frames, from the component level to the system level. Numerical models of six column specimens, both unretrofitted and CFRP-retrofitted, were developed in OpenSees and validated against reliable experimental results. Subsequently, three RC moment-resisting frame buildings with 4, 8, and 12 stories were modeled, their deficient columns were identified, and multiple retrofitting scenarios—no retrofit, one CFRP layer, and two CFRP layers—were applied. Seismic performance was evaluated using Nonlinear Static Pushover Analysis and Incremental Dynamic Analysis (IDA) under 20 far-field ground motions, and collapse fragility curves were generated. The validated column models showed very good agreement with experimental data, generally within 5% in terms of capacity, stiffness, and ductility. At the structural level, CFRP retrofitting improved collapse capacity by approximately 50% in the 8-story building, 45% in the 4-story building, and 20% in the 12-story dual-system structure, with fragility curves showing a clear shift toward higher seismic resistance. The results highlight the diminishing benefit of applying two CFRP layers compared to one, underscoring the importance of economic considerations. Overall, retrofitting substandard columns with a single layer of CFRP sheets is recommended as an efficient and cost-effective strategy for significantly enhancing the seismic performance and reducing the vulnerability of existing RC moment frames, particularly in bare-frame systems.