Moving target classification is a key ingredient to avoid accident in autonomous driving systems. Recently, fast chirp frequency modulated continuous wave (FMCW) radar has been popularly used to recognize moving targets due to its ability to discriminate moving objects and stationary clutter. In order to protect vulnerable road users such as pedestrians and cyclists, it is essential to identify road users in a very short period of time. In this paper, we propose a deep neural network that consists of convolutional recurrent units for target classification in automotive radar system. In our experiment, using the real data measured by the fast chirp FMCW-based high range resolution radar, we show that the proposed network is capable of learning the dynamics in time-series image data and outperforms the conventional classification schemes.