Abstract: The research on recognition of passengers＇ abnormal behavior on the escalator is of great significance． The traditional behavior recognition algorithm can not accurately recognize the multi-target in real time，and the re- cognition result is easily affected by the environment change． So an algorithm for recognizing passengers＇abnormal behavior on the escalator based on human skeleton sequences was proposed． Firstly，the passenger＇s face was de- tected by the support vector machine and tracked by the improved kernelized correlation filter to obtain the trajecto- ries of the passengers． Then，the human skeleton sequences of passengers were extracted by the convolutional neu- ral network． After that，the abnormal behavior skeleton sequences were detected from the human skeleton se- quences of passengers through template matching． Finally，the abnormal behavior was recognized by matching its skeleton sequence with all kinds of abnormal behavior skeleton sequences through dynamic time warping． The re- sults of experiment on 10 escalator videos show that the algorithm achieves a processing speed of 10 frames per se- cond and the recognition accuracy rate is 93. 2%，so it can accurately recognize a variety of passenger＇s abnormal behaviors in real time．

Key words: escalator, human skeleton sequence, abnormal behavior recognition, support vector machines, ker- nelized correlation filter, convolutional neural network, dynamic time warping