The objective of this study is to develop a 3D breast image registration algorithm for PET-CT and MR system. Proposed algorithm consists of three stages: breast segmentation, surface matching, and image transformation. Contrast-enhanced MR image volume was used as a reference and CT volume was transformed with varying parameters to calculate similarity between two image modalities. At first, the breast regions were cropped separately by the pre-determined regional masks. For the CT image case, region growing based breast segmentation was explored to eliminate the breast dedicated jig which has been developed to hold the shape of breast during PET-CT scan. After the segmentation, each of the points set of breast surface has been extracted. The extracted point sets were used as a feature vector for the surface matching. For the surface matching, we developed modified version of elastic-convolved iterative closest point (ECICP) algorithm to obtain the optimal transformation parameters. Then PET image was transformed with those parameters and overlaid it onto MR image. The results of this study show that the difference between MR and CT image has been considerably reduced and the suspicious lesions on MR and PET images were matched as well.