A scene-adaptive rate control algorithm with an enhanced frame skipping is proposed for low bit rate video coding. Most existing rate control algorithms for low bit rate video focus on bit allocation at the macroblock level under a bit rate constraint. The proposed rate control method looks at bit allocation from the frame level perspective and provides an effective bit allocation to reduce the temporal visual fluctuation of image quality such as flickering and motion jerkiness. It is demonstrated that the new rate control algorithm allocates target bits according to picture coding complexities and alleviates the problem of the frame skipping occurred by abrupt scene changes. Experimental results indicate that the proposed control method provides an effective alternative to existing frame-layer rate control for non-interactive streaming applications such as videocasting, video-monitoring, and video-based entertainment services.