We propose a speed up method for the Histograms of Oriented Gradients - Local Binary Pattern (HOG-LBP) based pedestrian detector. Our method is based on the two-stage cascade structure. In the first stage evaluation, instead of extracting the features from all the region inside the detection window like in the conventional method, we extract the features from the regions which best characterize the pedestrian only. By reducing the features to be evaluated, each candidate is evaluated faster. To determine which regions are best for characterizing the pedestrian, we train the AdaBoost classifier to select the blocks whose Support Vector Machine responses of the pedestrian samples are most different from the non-pedestrians. In the second stage, we simply use the conventional HOG-LBP classifier to reevaluate the candidates which pass the first stage evaluation. Experimental results show that the detection algorithm is about three times faster than the conventional HOG-LBP SVM algorithm.