Recognizing Event-Related Desynchronization or Synchronization (ERD/ERS) patterns generated by motor imagery tasks is an important process in Brain-computer interfaces (BCI). One of the most well-known algorithms to extract the discriminative patterns is Common Spatial Patterns (CSP). It finds an optimal spatial filter considering the spatial distribution of the ERD/ERS patterns. The CSP algorithm, however, does not consider temporal information of the Electroencephalogram (EEG) signals even though EEG signals are naturally non-stationary. In order to circumvent the limitation, in this paper, we propose a novel method, Time-Dependent Common Spatial Patterns (TDCSP) to classify multi-class motor imagery tasks. We optimize CSP filters in multiple local time ranges of EEG signals individually based on statistical analysis to effectively reflect changes of discriminative spatial distributions over time. We evaluated the proposed method by experiments on BCI Competition IV dataset 2-a, which resulted in high performance outperforming the previous methods in the literature.