The dynamic development of brain cognition and motor functions during infancy are highly associated with the rapid changes of the convoluted cortical folding. However, little is known about how the cortical folding, which can be characterized on different scales, develops in the first two postnatal years. In this paper, we propose a curvature-based multi-scale method using spherical wavelets to map the complicated longitudinal changes of cortical folding during infancy. Specifically, we first decompose the cortical curvature map, which encodes the cortical folding information, into multiple spatial-frequency scales, and then measure the scale-specific wavelet power at 6 different scales as quantitative indices of cortical folding degree. We apply this method on 219 longitudinal MR images from 73 healthy infants at 0, 1, and 2 years of age. We reveal that the changing patterns of cortical folding are both scale-specific and region-specific. Particularly, at coarser spatial-frequency levels, the majority of the primary folds flatten out, while at finer spatial-frequency levels, the majority of the minor folds become more convoluted. This study provides valuable insights into the longitudinal changes of infant cortical folding.