Group-wise registration has been proposed recently for consistent registration of all images in the same dataset. Since all images need to be registered simultaneously with lots of deformation parameters to be optimized, the number of images that the current group-wise registration methods can handle is limited due to the capability of CPU and physical memory in a general computer. To overcome this limitation, we present a hierarchical group-wise registration method for feasible registration of large image dataset. Our basic idea is to decompose the large-scale group-wise registration problem into a series of small-scale registration problems, each of which can be easily solved. In particular, we use a novel affinity propagation method to hierarchically cluster a group of images into a pyramid of classes. Then, images in the same class are group-wisely registered to their own center image. The center images of different classes are further group-wisely registered from the lower level to the upper level of the pyramid. A final atlas for the whole image dataset is thus synthesized when the registration process reaches the top of the pyramid. By applying this hierarchical image clustering and atlas synthesis strategy, we can efficiently and effectively perform group-wise registration to a large image dataset and map each image into the atlas space. More importantly, experimental results on both real and simulated data also confirm that the proposed method can achieve more robust and accurate registration than the conventional group-wise registration algorithms.