For navigation of a service robot, mapping and localization are very important and essential abilities of the robot. Mapping is the task of modeling a robot's environment and localization is the process of determining the position and orientation of a robot with respect to the global map. Exploration is the fundamental task of guiding a robot autonomously during mapping such that it covers the entire environment. In this paper, an efficient exploration scheme based on the position probability of the end nodes of a topological map is proposed. In this scheme, a grid map is built using the range sensor data and then a topological map is constructed in real time using the thinning-based approach. From the position probability, a robot can determine whether or not it needs to visit the specific end node to examine the environment. And the robot can choose which end node will be the next target. The robot can also estimate how much of the environment is explored by observing their position probabilities. The edges of the topological map can also be used for navigation path without additional path planning. This method can generate obstacle free paths even in the dynamic environment. Various experiments show that the proposed scheme can perform exploration more efficiently than other schemes in that, in most cases, exploration for the entire environment can be completed without directly visiting everywhere. It is shown that the exploration can work safely and stably in the dynamic environment.