In this paper, we propose an iterative lowcomplexity carrier frequency offset (CFO) compensation algorithm based on the conjugate gradient (CG) method for uplink orthogonal frequency division multiple access systems. In general, a linear minimum mean square error (MMSE) CFO compensation algorithm is simple and efficient, but it requires an inverse operation to compute the MMSE solution of the CFOinduced inter-carrier interference (ICI) matrix whose size equals the number of subcarriers. Thus, the computational complexity becomes prohibitively high when the number of subcarriers is large. To address this issue, we employ the iterative CG method to solve linear equations whose matrix is Hermitian positivedefinite. First, we derive the eigenvalue distribution of the CFOinduced ICI matrix. Based on the distribution, we present several observations which lead to the iteration number required for the convergence. Especially, the analysis shows that for an interleaved carrier assignment scheme, the maximum iteration number to compute an exact solution is at most the same as the number of mobile units. Simulation results exhibit that the proposed scheme can yield almost the same bit error rate performance of the MMSE compensation algorithm with substantially reduced computational complexity and memory requirements.