Acceleration data acquired by bridge structural health monitoring systems provide crucial information for structural anomaly identification and early damage detection. However, the inherent complexity and significant noise contamination of raw acceleration signals substantially reduce the separability of anomalous features, consequently increasing the risk of misjudgment. To address this, a self-supervised anomaly identification framework that integrates multi-frequency decomposition and multi-scale temporal modeling is proposed, along with a novel anomaly-sensitive metric termed the Gaussian Kernel Weighted Residual Sum of Squares (GKWRSS). The proposed method initially employs Empirical Mode Decomposition (EMD) to extract frequency-sensitive components. Subsequently, a Transformer network is utilized to establish long-range temporal dependencies within the signal and reconstruct the input. Finally, anomaly identification is realized by utilizing a Gaussian Mixture Model (GMM) to cluster the GKWRSS values of the original input signal and its reconstructed counterpart. The proposed EMD-Transformer-GMM framework effectively mitigates cross-frequency interference in the signals, thereby enhancing the separability of anomalous features. Furthermore, owing to the transposed convolutional decoder introduced in the Transformer layer, the architecture enables self-supervised learning. It reconstructs the input signal while preserving fine-grained temporal features and frequency-domain specificity. Additionally, this method classifies anomalies based solely on signal discrepancies, eliminating the need for labeled anomaly samples. Compared to the traditional Residual Sum of Squares (RSS), the GKWRSS significantly improves the separability between normal and anomalous states while maintaining sensitivity to local anomalies. Validation using real-world bridge monitoring data demonstrates that the EMD-Transformer-GMM model achieves an average anomaly identification accuracy of 95.73%, confirming its reliability and practical utility for bridge SHM and anomaly detection.