A real-time surveillance system is investigated on the basis of hidden Markov models (HMMs) using various features extracted from color images, human skeletons, and depth maps to sense anomalous behavior. Herein, the spatial and temporal features are included to enhance surveillance measurement accuracy by identifying and classifying anomalous activity. Hence, the proposed approach detects suspicious behaviors within a short time and achieves a better performance than traditional approaches. The HMM-based framework captures the underlying patterns or structures in sequential information when a human appears in the predefined monitoring area to detect anomalous behaviors. The highlights of the proposed system are its efficiency and practicality, balancing computational requirements and detection accuracy, making it suitable for real-time applications. For evaluating the proposed approach, a dataset collected by a Kinect camera is further divided into training and test data. Furthermore, the proposed approach significantly outperforms that based on naïve Bayes networks in precision rate according to the experimental results. As a result, evaluating observations demonstrates the potential of HMM-based systems to enhance security monitoring, providing reliable and effective solutions for instant anomalous behavior detection to ensure the security and protection of sensitive information and equipment in monitoring scopes.