In order to harness the strong horsepower of multi-core processors and meet the demand of high parallelism，a new parallel conjugate gradient algorithm is proposed，which focuses on solving the linear equations of large-scale sparse matrices． For the GPU coprocessors，the memory hierarchy of GPU is effectively utilized，optimization methods，such as thread and matrix mappings，data merging and data multiplexing，are adopted，and an effective thread scheduling is conducted to hide the high latency of accessing the global memory of GPU． For Xeon Phi processors，the computation of high parallelism is effectively utilized to optimize data communication and transmission，data dependence reduction，vectorization and asynchronous computation，and effective thread scheduling is also conducted to hide the high latency of accessing global memory of GPU． Finally，the proposed algorithm is proved to be feasible and correct by tests on GPU and Xeon Phi，and its parallel efficiencies in two different ways are compared． It is found that the proposed algorithm on GPU has a better acceleration effect than itself on Xeon Phi．