The joint photographic experts group (JPEG) is the most popular image on the internet nowadays. Therefore, reversible data hiding (RDH), which can be used to ensure the widely used JPEG images' originality and integrity, has attracted more and more attention. However, when performing RDH in JPEG images, besides the embedding capacity and visual quality, which have to be considered for uncompressed images, the storage size of the marked JPEG file should also be considered. In this paper, a new JPEG RDH scheme based on multiple histograms modification is proposed. Firstly, according to zero run length (ZRL) of the discrete cosine transform (DCT) coefficients, multiple DCT coefficient histograms are generated. Then, the message bits are embedded via the histogram shifting algorithm. In order to improve the efficiency of the proposed method, a new adaptive embedding strategy based on the ZRL and the number of zero coefficients in each 8 × 8 block is also proposed. Experimental results show that our method can effectively reduce the increase of image storage size caused by information embedding.