The north finder has been widely used in precision guided weapons, unmanned aerial vehicles (UAVs), and other device. In this paper, a north finder based on a low-cost micro-electro-mechanical system (MEMS) gyro is proposed. In order to reduce the substantial bias effect on the gyro’s north finding angle, a bidirectional carouseling method is applied. To improve the north finding accuracy, the phase difference between the turntable rotation rate and the gyro output is used. After the calculation of the phase difference, a five-order Butterworth filter is chosen to eliminate the gyro high-frequency noise. Experimental results demonstrate that the proposed method based on the phase difference can effectively improve the north finding accuracy by 50% compared with the traditional carouseling method.