In the field of multi-view clustering, many methods learn the similarity matrix directly from the original data, but this ignores the effect of noise in the original data. In addition, some methods must perform a feature decomposition on the graph Laplacian matrix, which leads to reduced interpretability and requires post-processing such as k-means. To address these issues, this paper proposed a fast multi-view clustering based on a unified label matrix. Firstly, a non-negative constraint was added to the objective function from the unified viewpoint of the normalized cut of the relaxation and the ratio cut. Then, a structured graph reconstruction was performed on the similarity matrix by the indicator matrix to ensure that the obtained graph has strong intra-cluster connections and weak inter-cluster connections. In addition, the number of iterations was reduced by setting a unified label matrix, thus further improving the speed of the method. Finally, the problem was solved optimally based on an alternating direction multiplication strategy. The algorithm aligns the multi-view dataset by randomly selecting the anchor addresses, and aligning the views can significantly improve the accuracy of clustering. The problem of the high computational complexity of traditional spectral clustering algorithms was effectively solved by using singular value decomposition instead of feature decomposition in the iterative process. Labels were obtained directly by indicating the column labels of the largest element of the matrix by row index. Experimental results on four real datasets demonstrate the effectiveness of the algorithm, and show that its clustering performance outperformed the nine existing benchmark algorithms.