In order to improve the accuracy of disparity estimation, an algorithm based on color segmentation and graph cut algorithm is proposed. In this algorithm, the current image is segmented into several color areas by em-ploying the mean shift algorithm, and the graph cut is implemented on the pixel set of each color area to allocate disparities for the pixels in the color area. Unlike the traditional global optimization algorithm, the proposed algo-rithm takes the pixel set of a color area rather than the whole image as an entirety to perform the global optimiza-tion, thus improving the disparity accuracy of object boundaries. Experimental results demonstrate that the proposed algorithm is more effective than the traditional one.