Research on human-robot interaction has recently been getting increasing attention. In the research field of human-robot interaction, speech signal processing in particular is the source of much interest. In this paper, we report on a speaker localization system with six microphones for a humanoid robot called MAHRU of KIST and propose a time delay of arrival (TDOA)-based feature matrix with its algorithm based on the minimum sum of absolute errors (MSAE) for sound source localization. The TDOA-based feature matrix is defined as a simple database matrix calculated from pairs of microphones installed on a humanoid robot. To verify the solid performance of our speaker localization system for a humanoid robot, we present the various experimental results for the speech sources at all directions within S m distance and the height divided into three parts.