Aiming at the problem that deep learning-based rolling bearing fault diagnosis algorithms need to learn from a large amount of labeled data and face poor diagnosis effect when the number of samples is limited, this paper proposed a small-sample rolling bearing fault diagnosis method based on the Gramian angular difference field (GADF) and generative adversarial networks (GAN). Firstly, a data enhancement method based on GADF transform was proposed, and it converts a few 1D vibration signals into 2D GADF images by GADF transform. GADF subgraphs are obtained by cropping to obtain a large number of image samples. Then, a conditional generative adversarial network (CGAN) was combined with Wasserstein GAN with gradient penalty (WGAN-GP) to construct a novel generative adversarial network, which enhances the model training stability by conditional auxiliary information with gradient penalty and designs dynamic coordinate attention mechanism to enhance the spatial perception of the model, so as to generate high-quality samples. Finally, the generative samples were used to train the classifier, and the diagnosis results were obtained on the validation set. Two sets of bearing fault diagnosis experiments in a small sample environment were conducted using the Southeast University dataset and the Case Western Reserve University dataset, respectively. The results show that, compared with traditional generative adversarial networks as well as advanced small-sample fault diagnosis methods, the proposed method can obtain the best results in five fault diagnosis metrics, including accuracy and precision, and can accurately diagnose the type of bearing faults under small-sample conditions.