Successive point-of-interest (POI) recommendation based on user check-in histories plays an important role in mobile-based social media platforms. Although a large amount of check-in data including textual content is generated from such platforms, most successive POI recommendation models do not leverage textual contents that provide useful information for understanding user interests. To address this problem, we propose a new content-aware successive POI recommendation (CAPRE) model in this paper. Based on a multi-head attention mechanism and a character-level convolutional neural network, CAPRE encodes user-generated textual contents into content embedding to capture user interests. Based on long short-term memories (LSTMs), CAPRE capture content-aware user behavior patterns from encoded content embedding. Evaluation results on real-world datasets show that CAPRE achieves state-of-the-art recommendation performance.