In this paper, we investigate a deep learning (DL) approach to solve a generic constrained optimization problem in wireless networks, where the objective and constraint functions can be nonconvex. To this target, the computation process of the solution is replaced by deep neural networks (DNNs). The original problem is transformed to a training task of the DNNs subject to nonconvex constraints. Since existing DL libraries are originally intended for unconstrained training, they cannot be directly applied to our constrained training problem. We propose a constrained training strategy based on the primal-dual method from optimization techniques. The proposed DL approach is deployed to solve transmit power allocation problems in various network configurations. The simulation results shed light on the feasibility of the DL method as an alternative to existing optimization algorithms.