Various evolutionary algorithms are being developed to search the optimal solution of various problems in the real world. Evolutionary algorithms search solutions showing the optimal fitness to given problem using their own operators. Engineering benchmark problems can be used for performance measurement of evolutionary algorithms, and the water distribution network design problem is one of the widely used benchmark problems. In this study, the water distribution network design problems are generated by modifications of five problem characteristic factors. Generated benchmark problems are applied to quantitatively evaluate the performance among evolutionary algorithms. Each algorithm shows its own strength and weakness. Optimization results show that the engineering benchmark generation method suggested in this study can be served as a reliable framework for comparison of performances on various water distribution network design problems.