For the detection of non-uniform and non-periodic irregular texture surface defects，it is difficult to achieve conventional image reconstruction as the background texture is a non-stationary signal． It is also difficult to obtain sufficient information about the defects such as shape，gray scale and other image features in advance． And the visual features of similar defects can present great dispersion． The existing detection methods based on target features are inapplicable． Therefore，this paper proposed an unsupervised learning surface defect detection method based on image inpainting． Firstly，the method used a small number of normal texture samples as training set to train the network model． Then，the missing area was set artificially in the sample image and the network model was used to predict the content of the missing areas during the testing stage． By combining the structural similarity evaluation and residual error of the reconstructed image and the image to be tested，the defect detection and separation were realized． The experimental results show that the proposed method can not only effectively detect the defects on the regular texture surface but also detect the defects of irregular texture surfaces effectively，showing good practicability and adaptability．