华南理工大学学报(自然科学版) ›› 2020, Vol. 48 ›› Issue (8): 10-21.doi: 10.12141/j.issn.1000-565X.200010

• 电子、通信与自动控制 • 上一篇    下一篇

基于视频监控的手扶电梯乘客异常行为识别

杜启亮1,2 黄理广田联房1,2 黄迪臻靳守杰李淼4   

  1. 1. 华南理工大学 自动化科学与工程学院,广东 广州 510640; 2. 华南理工大学 自主系统与网络控制教育部重点实验室,广东 广州 510640; 3. 广州地铁集团有限公司,广东 广州 510335; 4. 日立电梯 (广州) 自动扶梯有限公司,广东 广州 510660

  • 收稿日期:2020-01-09 修回日期:2020-05-15 出版日期:2020-08-25 发布日期:2020-08-01
  • 通信作者: 杜启亮(1980-),男,博士,副研究员,主要从事机器人与机器视觉研究。 E-mail:qldu@scut.edu.cn
  • 作者简介:杜启亮(1980-),男,博士,副研究员,主要从事机器人与机器视觉研究。
  • 基金资助:
    国家科技部海防公益类项目 (201505002); 广东省科技计划项目 (2016B090912001); 广东省重点领域研发计划 “新一代人工智能” 重大科技专项 (2018B010109001); 广东省重点领域研发计划 “精准农业” 重 点 专 项(2019B020214001); 广州市产业技术重大攻关计划 (2019-01-01-12-1006-0001); 华南理工大学中央高校基本科研业务费专项资金资助项目 (2018KZ05); 华南理工大学研究生教育改革项目 (zysk2018005)

Recognition of Passengers'Abnormal Behavior on Escalator Based on Video Monitoring

DU Qiliang1,2 HUANG LiguangTIAN Lianfang1,2 HUANG Dizhen1 JIN Shoujie3 LI Miao4   

  1. 1. School of Automation Science and Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China;2. Key Laboratory of Autonomous Systems and Network Control of the Ministry of Education,Guangzhou 510640,Guangdong,China;3. Guangzhou Metro Group Co.,Ltd.,Guangzhou 510335,Guangdong,China; 4. Hitachi Elevator (Guangzhou) Escalator Limited Liability Company,Guangzhou 510660,Guangdong,China
  • Received:2020-01-09 Revised:2020-05-15 Online:2020-08-25 Published:2020-08-01
  • Contact: 杜启亮(1980-),男,博士,副研究员,主要从事机器人与机器视觉研究。 E-mail:qldu@scut.edu.cn
  • About author:杜启亮(1980-),男,博士,副研究员,主要从事机器人与机器视觉研究。
  • Supported by:
     Supported by the Coast Defence Public Welfare Project of the Ministry of Science and Technology of China(201505002),the Science and Technology Planning Project of Guangdong Province (2016B090912001),the Key Field R&D Program “New Generation Artificial Intelligence”Major Science and Technology Project of Guangdong Province (2018B010109001) and the Key Field R&D Program “Precision Agriculture”Key Special Project of Guangdong Province (2019B020214001)

摘要: 针对乘客在搭乘扶梯时的危险行为难以被实时准确检测的问题,提出了一种基于视频监控的手扶电梯乘客异常行为识别算法。首先,使用 YOLOv3 对图像中乘客的位置进行检测; 接着,使用 MobileNetv2 作为基网络,结合反卷积层对检测出来的乘客进行人体骨架提取; 然后,使用骨架距离作为跟踪依据,采用匈牙利匹配算法对相邻帧间的人体骨架进行匹配,实现视频中乘客的 ID 号分配; 最后,通过图卷积神经网络对乘客关键点信息进行异常行为识别。在 GTX1080GPU 上的实验结果表明,文中提出的识别算法的处理速度能达到 15f/s,异常行为识别准确率达 94. 3%,能够实时准确地识别手扶电梯上乘客的异常行为。

关键词: 手扶电梯, 深度学习, 卷积神经网络, 行人检测, 人体关键点提取, 匈牙利匹配算法

Abstract: Aiming at the problem that dangerous behavior of passengers on the escalator is difficult to be accurately detected in real time,an algorithm for identifying abnormal behavior of escalator passengers based on video surveil-lance was proposed. Firstly,YOLOv3 was used to detect the position of the passenger in the image. Secondly,MobileNetv2 was used as the base network,which was combined with the deconvolution layer to extract the human skeleton of the detected passenger. Thirdly,the Hungarian assignment algorithm based on skeleton distance was used to realize the allocation of passenger ID numbers in the video. Finally,with keypoints as the input,the graph convolutional neural network was used to recognize the abnormal behavior of passenger. The experimental results on the GTX1080GPU show that the proposed recognition algorithm can achieve a processing speed of 15 f/s and an abnormal behavior recognition accuracy rate of 94. 3%,which can accurately recognize the abnormal behavior of passengers on the escalator in real time.

Key words: escalator, deep learning, convolutional neural networks, pedestrian detection, pose estimation, Hungarian assignment algorithm

中图分类号: