Meta-heuristic algorithms have been developed to solve various mathematical and engineering optimization problems. However, meta-heuristic algorithms show different performances depending on the characteristics of each problem. Therefore, there have been many kinds of research to decrease the performance gap for the different optimization problems by developing new algorithms, improving the search operators, and considering self-adaptive parameters setting on their algorithms. However, the previous studies only focused on improving the performance of each problem category (e.g., mathematical problem, engineering problem) without the quantitative evaluation for the operator performance. Therefore, this study proposes a framework for the quantitative evaluation to solve the no free lunch problem using the operators of the representative meta-heuristic algorithms (such as genetic algorithm and harmony search algorithm). Moreover, based on the quantitative analysis results for each operator, there are several types of hybrid optimization algorithms, which combined the operator of harmony search algorithm (HSA), genetic algorithm (GA), and particle swarm optimization (PSO). The optimization process to find the optimal solution is divided into five sections based on the number of function evaluations to see the performance of the search operator according to the section. Representative mathematical problems were applied to quantify the performance and operators. None of the five evaluated applied to mathematical benchmark problems were the best algorithms. Hybrid HSAs showed advanced performance for problems where traditional HSA did not show good performance. However, it still has not escaped the No Free Lunch theorem.