Diffuse Optical Tomography (DOT) involves a nonlinear optimization problem to find the tissue optical properties by measuring near-infrared light noninvasively. Many researchers used linearization methods to obtain the optical image in real time. However, the linearization procedure may neglect small but sometimes important regions such as small tumors at an early stage. Therefore, nonlinear optimization methods such as gradient- or Newton- type methods are exploited, resulting in better resolution image than that of linearization methods. But the disadvantage of nonlinear methods is that they need much computation time. To solve this trade-off dilemma between image resolution and computing time, we suggest second order inverse Born expansion algorithm in this paper. It is known that a small perturbation of photon density is represented by Born expansion with respect to the perturbation of optical coefficients, which is an infinite series of integral operators having Robin function kernel. Whereas, inverse Born expansion is an implicit representation of a small perturbation of optical coefficients by an infinite series of the integral operators with respect to the photon density and its perturbation, which is appropriate series expansion for inverse DOT problem. Solving the inverse Born expansion itself and the first order approximation correspond to nonlinear and linear method, respectively. We formulated a second order approximation of the inverse Born expansion explicitly to make numerical implementation possible and showed the convergence order of the proposed method is higher than the linear method.