Image registration, which aligns a pair of fixed and moving images, is often tackled by the large shape and intensity variation between the images. As a remedy, we present a generalized registration framework that is capable to predict the initial deformation field between the fixed and moving images, even though their appearances are very different. For the prediction, we learn the prior knowledge on deformation from pre-observed images. Especially, our method is significantly differentiated from previous methods that are usually confined to a specific fixed image, to be flexible for handling arbitrary fixed and moving images. Specifically, our idea is to encapsulate many pre-observed images into a hierarchical infrastructure, termed as cloud, which is able to efficiently compute the deformation pathways between the pre-observed images. After anchoring the fixed and moving images to their respective port images (similar images in terms of intensity appearance) in the cloud, we predict the initial deformation between the fixed and moving images by the deformation pathway between the two port images. Thus, the remaining small deformation can be efficiently refined via most existing deformable registration methods. With the cloud, we have obtained promising registration results on both adult and infant brain images, demonstrating the advantage of the proposed registration framework in improving the registration performance.