Understanding human brain functional development in the very early ages is of great importance for charting normative development and detecting early neurodevelopmental disorders, but it is very challenging. We propose a group-constrained, robust community detection method for better understanding of developing brain functional connectome from neonate to two-year-old. For such a multi-subject, multi-age-group network topology study, we build a multi-layer functional network by adding inter-subject edges, and detect modular structure (communities) to explore topological changes of multiple functional systems at different ages and across subjects. This “Multi-Layer Inter-Subject-Constrained Modularity Analysis (MLISMA)” can detect group consistent modules without losing individual information, thus allowing assessment of individual variability in the brain modular topology, a key metric for developmental individualized fingerprinting. We propose a heuristic parameter optimization strategy to wisely determine the necessary parameters that define the modular configuration. Our method is validated to be feasible using longitudinal 0–1–2 year’s old infant brain functional MRI data, and reveals novel developmental trajectories of brain functional connectome.