In the process of remote sensing data processing, cloud data will have a considerable negative impact on data processing. Therefore, a matrix optimization cloud detection algorithm based on multiple receptive fields is proposed in this paper. First, the algorithm adopts a block reshaping model to improve the efficiency of the algorithm, while reducing the need for hardware configuration. Second, the cloud region adaptive sparse attention matrix is used to improve the characteristics of cloud regions with different concentrations. Finally, the multi-receptive field scaling module is used to improve the ability to segment cloud regions at different scales. The experimental results show that the accuracy of the proposed algorithm is 85.5%, and the recall rate is 86.4%. The proposed algorithm can basically accurately screen the location of a cloud region and provide technical support for the subsequent image application processing.