In the network slicing environment, if insufficient resource is allocated to a resource-hungry slice (e.g., ultrareliable low latency communication (URLLC)), its quality of service (QoS) can be significantly degraded. In this paper, we propose a priority-based dynamic resource allocation scheme (PDRAS) where the resource management agent maintains some information such as priorities and demand profiles of slices. Based on these information, the agent dynamically allocates resources to slices. To maximize QoS of slices while maintaining the total amount of allocated resources below a certain level, a constrained Markov decision process (CMDP) problem is formulated and the optimal allocation policy is achieved by linear programming (LP). Extensive evaluation results are given to demonstrate the effectiveness of PDRAS with the optimal policy compared to that of other schemes in terms of QoS and the efficiency of the resource usage.