Currently, there are still no early biomarkers to detect infants with risk of autism spectrum disorder (ASD), which is mainly diagnosed based on behavioral observations at three or four years of age. Since intervention efforts may miss a critical developmental window after 2 years old, it is clinically significant to identify imaging-based biomarkers at an early stage for better intervention, before behavioral diagnostic signs of ASD typically arising. Previous studies on older children and young adults with ASD demonstrate altered developmental trajectories of the amygdala and hippocampus. However, our knowledge on their developmental trajectories in early postnatal stages remains very limited. In this paper, for the first time, we propose a volume-based analysis of the amygdala and hippocampal subfields of the infant subjects with risk of ASD at 6, 12, and 24 months of age. To address the challenge of low tissue contrast and small structural size of infant amygdala and hippocampal subfields, we propose a novel deep-learning approach, dilated-dense U-Net, to digitally segment the amygdala and hippocampal subfields in a longitudinal dataset, the National Database for Autism Research (NDAR). A volume-based analysis is then performed based on the segmentation results. Our study shows that the overgrowth of amygdala and cornu ammonis sectors (CA) 1–3 May start from 6 months of age, which may be related to the emergence of autistic spectrum disorder.