Accurate and consistent skull stripping of serial brain MR images is of great importance in longitudinal studies that aim to detect subtle brain morphological changes. To avoid inconsistency and the potential bias introduced by independently performing skull-stripping for each time-point image, we propose an effective method that is capable of skull-stripping serial brain MR images simultaneously. Specifically, all serial images of the same subject are first affine aligned in a groupwise manner to a common space to avoid any potential bias introduced by asymmetric transforms. A brain probability map, which encapsulates prior information gathered from a population of real brain MR images, is then warped to the aligned serial images for guiding skull-stripping via a deformable surface method. In particular, the same initial surface meshes representing the initial brain surfaces are first placed on all aligned serial images, and then all these surface meshes are simultaneously evolved to the respective target brain boundaries, driven by the intensity-based force, the force from the probability map, as well as the force from the spatial and temporal smoothness. Especially, imposing the temporal smoothness helps achieve longitudinally consistent results. Evaluations on 20 subjects, each with 4 time points, from the ADNI database indicate that our method gives more accurate and consistent result compared with 3D skull-stripping method. To better show the advantages of our 4D brain extraction method over the 3D method, we compute the Dice ratio in a ring area (±5mm) surrounding the ground-truth brain boundary, and our 4D method achieves around 3% improvement over the 3D method. In addition, our 4D method also gives smaller mean and maximal surfaceto- surface distance measurements, with reduced variances.