Correspondence matching between MR brain images is often challenging due to large inter-subject structural variability. In this paper, we propose a novel landmark detection method for robust establishment of correspondences between subjects. Specifically, we first annotate distinctive landmarks in the training images. Then, we use regression forest to simultaneously learn (1) the optimal set of features to best characterize each landmark and (2) the non-linear mappings from local patch appearances of image points to their displacements towards each landmark. The learned regression forests are used as landmark detectors to predict the locations of these landmarks in new images. Since landmark detection is performed in the entire image domain, our method can cope with large anatomical variations among subjects. We evaluated our method by applying it to MR brain image registration. Experimental results indicate that by combining our method with existing registration method, obvious improvement in registration accuracy can be achieved.