In Intelligent Transportation Systems (ITS), it is widely used to extract a fixed-size feature vector from raw traffic data for high-level traffic analysis. In several existing works, the statistical approach has been used for extracting feature vectors, which directly extracts features by averaging speed or travel time of each vehicle. However, we can achieve a better representation by taking advantage of state-of-the-art machine learning algorithms instead of the statistical approach. In this paper, we propose a two-phase framework named embed-and-aggregate framework for extracting features from raw traffic data, and a feature extraction algorithm (Traffic2Vec) based on our framework exploiting state-of-the-art machine learning algorithms such as deep learning. We also implement a traffic flow prediction system based on Traffic2Vec as a proof-of-concept. We conducted experiments to evaluate the applicability of the proposed algorithm, and show its superior performance in comparison with the prediction system based on the statistical feature extraction method.