As the video trajectory has become a new tool for the automatic analysis of video images,an algorithm of video trajectory description based on the spatial-temporal extension to locally linear embedding ( ST-LLE) is proposed. In this algorithm,first,video clips are divided into continuous subsequences,and non-trivial k-nearest neighbors ( ntKNN) are adopted to capture similar video subsequence mode with spatial-temporal constraint. Then,the weights between each video subsequence and its non-trivial k-nearest neighbors are constructed,according to which the low-dimension embedded vectors of the video subsequence are calculated. Thus,the trajectory of the video subsequence is successfully obtained. Experimental results demonstrate that the proposed algorithm effectively describes the complex spatial-temporal structures of video sequences and helps to obtain more reasonable video trajectory,as compared with the original locally linear embedding algorithm.