Passenger flow forecast is of great significance to the organization and management of urban rail transit. This paper constructed a graph convolution and recurrent neural network (GCGRU model) by combining graph convolutional network with recurrent neural network. The graph convolutional network was used to learn the complex topological structure of an urban rail network and capture spatial correlation characteristics. Then one of the recurrent neural network variants called gated recurrent unit was used to learn the variation of multi-characteristics of traffic trends and to capture the temporal characteristics. An experiment was carried out with the passenger flow data obtained from the entire network with all subway cross-sections in Shanghai in a whole year, and the mean decrease impurity method provided by random forest was used for feature selection. The experimental results show that the GCGRU model can well capture the temporal and spatial correlation in the prediction of large-scale urban rail transit passenger flow, with a prediction accuracy of 89%. The prediction results can provide a basis for managers to manage and organize rail transit passenger flow as well as provide travelers with early warning information, ensuring the safe and efficient operation of the urban rail transit network.