The basic concept of the scalable video coding (SVC) is to reduce the amount of bits for video representation by exploiting the correlation between coding layers. In the SVC, a macroblock (MB) of the enhancement layer (EL) is predicted using the prediction MB which is the upsampled version of the co-located MB of the base layer (BL). In this paper, we propose additional prediction MBs which are displaced from co-located position. In the proposed method, displacement vector (DV) indicating the best matching MB of the BL is encoded when another MB is more similar to the MB of the EL than the co-located one. The bits for DV are adaptively assigned based on the Lagrangian cost function. Experimental results show that the proposed algorithm achieves better performance in the rate-distortion sense.