Recognizing human action from image sequences is an active area of research in computer vision. In this paper, we present a novel method for human action recognition from image sequences in different viewing angles that uses the Cartesian component of optical flow velocity and human body shape feature vector information. We use principal component analysis to reduce the higher dimensional shape feature space into low dimensional shape feature space. We represent each action using a set of multidimensional discrete hidden Markov model and model each action for any viewing direction. We performed experiments of the proposed method by using KU gesture database. Experimental results based on this database of different actions show that our method is robust.