Bearing is one of the most widely used rotating parts in industrial equipment. If the bearing runs in fault condition for a long time, it will cause huge economic loss and threaten personal safety, so that the investigation of bearing fault diagnosis is of great significance. Fault diagnosis technology based on deep learning is becoming more and more mature, but there are problems such as over-fitting, unstable effect and low accuracy in the case of small samples. In order to solve these problems, this paper proposes a Transformer variant model MDT (Multi-Head Convolution and Differential Self-Attention Transformer) to realize end-to-end few-shot fault diagnosis. This model combines the new data embedding algorithm of MC (Multi-Head Convolution) and the DSA (Differential Self-Attention) mechanism. The MC algorithm performs multi-path one-dimension convolution on the sample, extends the sample from one dimension to two dimensions by multi-channel output, and extracts rich fault information in each frequency domain in the original sample through multiple convolution kernel sizes. As compared with the original dot product self-attention in Transformer, the DSA mechanism obtains the corresponding attention weight vector for each feature through the difference, so as to extract deeper fault features from the sample. MDT inherits the powerful ability of Transformer to process sequence data, which can extract richer fault information from time-domain signals and avoid the overfitting problem common in small-sample models. Experimental results show that the proposed method can stably obtain more than 99% test accuracy in the bearing fault diagnosis task with only 100 training samples per fault type, and has strong anti-overfitting ability and strong robustness.