华南理工大学学报(自然科学版)

• 电子、通信与自动控制 • 上一篇    下一篇

基于人体骨架序列的手扶电梯乘客异常行为识别

田联房1,2 吴啟超1 杜启亮1,2† 黄理广1 李淼3  张大明3   

  1. 1.华南理工大学 自动化科学与工程学院,广东 广州 510640; 2. 华南理工大学 自主系统与网络控制教育部重点实验室,广东 广州 510640; 3. 日立电梯(广州)自动扶梯有限公司,广东 广州 510660
  • 收稿日期:2018-04-23 出版日期:2019-04-25 发布日期:2019-03-01
  • 通信作者: 杜启亮(1980-),男,博士,副研究员,主要从事机器人与机器视觉等研究 E-mail:qldu@ scut.edu.cn
  • 作者简介:田联房(1969-),男,博士,教授,主要从事模式识别与人工智能等研究.
  • 基金资助:
    国家科技部海防公益类项目(201505002);广东省前沿与关键技术创新专项资金资助项目(2016B090912001);广 州市产学研项目(201604010114)

Recognition of Passengers'Abnormal Behavior on the Escalator Based on Human Skeleton Sequence

TIAN Lianfang1,2 WU Qichao1 DU Qiliang1,2 HUANG Liguang1 LI Miao3  ZHANG Daming3   

  1. 1. School of Automation Science and Engineering,South China University of Technology,Guangzhou 510640,Guangdong,China; 2. Key Laboratory of Autonomous Systems and Network Control of Ministry of Education,Guangzhou 510640,Guangdong,China; 3. Hitachi Elevator (Guangzhou) Escalator Limited Liability Company,Guangzhou 510660,Guangzhou,China
  • Received:2018-04-23 Online:2019-04-25 Published:2019-03-01
  • Contact: 杜启亮(1980-),男,博士,副研究员,主要从事机器人与机器视觉等研究 E-mail:qldu@ scut.edu.cn
  • About author:田联房(1969-),男,博士,教授,主要从事模式识别与人工智能等研究.
  • Supported by:
    Supported by the National Ministry of Science and Technology Hai Phong Public Welfare Project(201505002) and the Special Funds for Frontier and Key Technology Innovation of Guangdong(2016B090912001)

摘要: 手扶电梯(简称扶梯)乘客异常行为识别研究具有重要意义. 针对传统行为识别 算法易受环境影响、不能实时并准确对多目标进行识别的问题,提出一种基于人体骨架序 列的扶梯乘客异常行为识别算法. 该算法首先通过结合可变形组件模型特征的支持向量 机检测乘客人脸,并用改进的核相关滤波器对其进行跟踪,从而得到乘客在扶梯中的运动 轨迹;接着利用卷积神经网络提取轨迹中乘客的人体骨架序列,并通过模板匹配从乘客人 体骨架序列中检测异常行为骨架序列;最后利用动态时间规整将其与各类异常行为骨架 序列匹配,基于 k 近邻方法识别异常行为. 对 10 段扶梯视频的实验结果表明,文中所提的 异常行为识别算法处理速度达到 10 帧/秒,识别准确率为 93. 2%,能够实时、准确地识别 多种乘客异常行为.

关键词: 手扶电梯, 人体骨架序列, 异常行为识别, 支持向量机, 核相关滤波, 卷积神经网络, 动态时间规整

Abstract: The research on recognition of passengers' abnormal behavior on the escalator is of great significance. The traditional behavior recognition algorithm can not accurately recognize the multi-target in real time,and the re- cognition result is easily affected by the environment change. So an algorithm for recognizing passengers'abnormal behavior on the escalator based on human skeleton sequences was proposed. Firstly,the passenger's face was de- tected by the support vector machine and tracked by the improved kernelized correlation filter to obtain the trajecto- ries of the passengers. Then,the human skeleton sequences of passengers were extracted by the convolutional neu- ral network. After that,the abnormal behavior skeleton sequences were detected from the human skeleton se- quences of passengers through template matching. Finally,the abnormal behavior was recognized by matching its skeleton sequence with all kinds of abnormal behavior skeleton sequences through dynamic time warping. The re- sults of experiment on 10 escalator videos show that the algorithm achieves a processing speed of 10 frames per se- cond and the recognition accuracy rate is 93. 2%,so it can accurately recognize a variety of passenger's abnormal behaviors in real time.

Key words: escalator, human skeleton sequence, abnormal behavior recognition, support vector machines, ker- nelized correlation filter, convolutional neural network, dynamic time warping

中图分类号: