Low-altitude inspection technology, noted for its remarkable advantages in cost reduction and efficiency improvement, is increasingly becoming a key enabling technology in modern inspection systems. The efficient operation of unmanned aerial vehicle (UAV) inspections depends on a well-structured top-level planning framework, the core of which comprises three closely interrelated decision-making layers: site selection, task assignment, and path planning. This paper presents a systematic review of current research and modeling methodologies concerning this integrated planning framework. Firstly, task characteristics are categorized and analyzed from multiple dimensions, including inspection targets, operational scenarios, and combination modes. Building on this foundation, the modeling approaches, optimization objectives, and constraint systems of the three core layers site selection, task assignment, and path planning are discussed in detail. By establishing a three-tier analytical framework, the review provides a more systematic and hierarchical perspective for the field. Finally, in response to existing research challenges in system coordination, real-time adaptability, and environmental robustness, this paper highlights several future research directions, including three-level integrated optimization, cloud–edge–device collaborative computing, and robust planning that integrates data-driven and physics-based models. The goal of this study is to offer a comprehensive reference for both theoretical research and engineering practice in the advancement of intelligent planning systems for UAV inspection.