The diagnosis of Alzheimer’s disease (AD) from neuroimaging data at the pre-clinical stage has been intensively investigated because of the immense social and economic cost. In the past decade,computational approaches on longitudinal image sequences have been actively investigated with special attention to Mild Cognitive Impairment (MCI),which is an intermediate stage between normal control (NC) and AD. However,current state-of-the-art diagnosis methods have limited power in clinical practice,due to the excessive requirements such as equal and immoderate number of scans in longitudinal imaging data. More critically,very few methods are specifically designed for the early alarm of AD uptake. To address these limitations,we propose a flexible spatial-temporal solution for early detection of AD by recognizing abnormal structure changes from longitudinal MR image sequence. Specifically,our method is leveraged by the non-reversible nature of AD progression. We employ temporally structured SVM to accurately alarm AD at early stage by enforcing the monotony on classification result to avoid unrealistic and inconsistent diagnosis result along time. Furthermore,in order to select best features which can well collaborate with the classifier,we present as joint feature selection and classification framework. The evaluation on more than 150 longitudinal subjects from ADNI dataset shows that our method is able to alarm the conversion of AD 12 months prior to the clinical diagnosis with at least 82.5 % accuracy. It is worth noting that our proposed method works on widely used MR images and does not have restriction on the number of scans in the longitudinal sequence,which is very attractive to real clinical practice.