Traditional approaches for automatic CT prostate segmentation often guide feature representation learning directly based on manual delineation to deal with this challenging task (due to unclear boundaries and large shape variations), which does not fully exploit the prior information and leads to insufficient discriminability. In this paper, we propose a novel hierarchical representation learning method to segment the prostate in CT images. Specifically, one multi-task model under the supervision of a series of morphological masks transformed from the manual delineation aims to generate hierarchical feature representations for the prostate. Then, leveraging both these generated rich representations and intensity images, one fully convolutional network (FCN) carries out the accurate segmentation of the prostate. To evaluate the performance, a large and challenging CT dataset is adopted, and the experimental results show our method achieves significant improvement compared with conventional FCNs.