In pile foundation static load tests at construction sites, counterweight blocks often present fuzzy boundaries and highly irregular shapes, which makes conventional image segmentation algorithms insufficient in accuracy and severely limits the efficiency of safety monitoring and quality management. To tackle this challenge, we propose a fine-grained segmentation algorithm for counterweight blocks based on a multi-scale fusion TransUNet framework. Specifically, we first construct a ViT-based multi-scale skip-connection scheme, in which global semantic features are adaptively upsampled to three different resolution levels and then fused with local detail features through dual skip connections, thereby strengthening the network’s feature representation capability for complex textures and irregular shapes of counterweight blocks. Second, a Triplet Attention mechanism is embedded into the decoder to jointly enhance spatial, channel, and depth dimensions, improving the network’s ability to delineate fuzzy boundaries and detect small counterweight blocks. Finally, to address the pronounced class imbalance between background and foreground, we design a weighted hybrid loss that combines Dice loss and Focal loss, further enhancing the model’s sensitivity to small stone blocks. Experiments on real static load test platform images demonstrate that the optimized network improves IoU by 9.6% over the baseline TransUNet, which verifies the effectiveness and superiority of the proposed method and highlights its strong potential for engineering applications.