In order to obtain positron emission tomography (PET) image with diagnostic quality, we seek to estimate a standard-dose PET (S-PET) image from its low-dose counterpart (L-PET), instead of obtaining the S-PET image directly by injecting standard-dose radioactive tracer to the patient. Therefore, the risk of radiation exposure can be significantly reduced. To achieve this goal, one possible way is to first map both S-PET and L-PET data into a common space and then perform a patch-based estimation of S-PET from L-PET patches. However, the approach of using all training data to globally learn the common space may not lead to an optimal estimation of a particular target S-PET patch. In this paper, we introduce a data-driven multi-level Canonical Correlation Analysis (m-CCA) scheme to tackle this problem. Specifically, a subset of training data that are most useful in estimating a target S-PET patch are identified in each level, and using these selected training data in the subsequent level leads to more accurate common space mapping and improved estimation. In addition, we also leverage multi-modal magnetic resonance (MR) images to provide complementary information to the estimation from L-PET. Validation on a real human brain dataset demonstrates the advantage of our method as compared to other techniques.