This paper presents an efficient JPEG steganography method based on improved matrix encoding. Compared to the original matrix encoding (ME), the proposed improved matrix encoding uses two intersected ME blocks of the DCT coefficients as a single combined block. We propose a way to get a join solution of the two intersected ME, such that the intersected area does not affect the result. Due to intersection the improved matrix encoding may use a matrix encoding scheme with higher embedding rate. In order to survive steganalysis we hides data to DCT coefficients which cause the lowest distortion after modification. We used the original bitmap image for computing the distortion and getting the modified JPEG image. The proposed insert-remove strategy modifies the input stream of the DCT coefficients by inserting or removing coefficients 1 or -1. Any insertion and removing results the different solutions for the improved matrix encoding. Among all possible solutions the proposed method chooses solution with the lowest distortion. Such method significantly increases the number of possible solutions and, as a result, decreases the total distortion after data hiding. The experiments include the steganalysis of the proposed improved matrix encoding with and without using the insert-remove strategy. The experiment results shows that the proposed methods has lower detectability of the steganalysis compared to the existing steganographic methods.