Due to the spread of various information and communication technologies, a huge amount of images are produced and shared for diverse purposes. Several de-identification techniques for photos, such as pixelation, blur, and mask, are routinely used in light of recent worries about the growing number of privacy leakages. However, due to the low image quality and loss of many facial features, these de-identified images are not suitable for use in applications such as training models that require a lot of high-quality data. Therefore, in this paper, we propose a new face de-identification method focusing only on facial regions essential for personal identification. By generating facial landmarks differently from the original person using masking and generative adversarial networks-based inpainting, our method can perform de-identification efficiently. To demonstrate the performance of our proposed scheme, we conducted quantitative and qualitative evaluations using an open dataset. We show that our proposed scheme outperforms other de-identification methods.