Image dehazing is a crucial technique in the study of computer vision. The most widely used image dehazing approach is the dark channel prior (DCP) method proposed by He et al. [IEEE Trans. Pattern Anal. Mach. Intell. 33 (2011) 2341]. Because a DCP-based method generates halo artifacts under certain conditions, this study aims to solve this problem and propose a DCP-based method that uses a mask with an adaptive size. The proposed method is based on the inverse ratio of the gradient of a hazy image and calculates the corresponding mask size. A small mask size is used for regions with a large gradient to solve the halo problem and a large mask size is used for regions with a small gradient to achieve the dehazing effect. Subsequently, the gradient was smoothened and the γ function was corrected using a Gaussian filter to obtain a more favorable nonlinear relationship. Finally, the ant colony optimization (ACO) algorithm was employed to determine the optimal parameters for the Gaussian filter and γ function. A new dehazing performance index (DPI) was also proposed in this study as the cost function for the ACO algorithm. The experimental results of this study verified that the proposed method can effectively minimize the effect of halo artifacts without compromising the dehazing performance and color distortion.