In object detection in remote sensing images, owing to the complex background environment, there are problems of poor robustness to interference and low detection accuracy for small objects. The algorithm proposed in this paper combines the attention mechanism with the spatial pyramid structure to improve the You-only-look-once algorithm version 3 (YOLOv3), and it also includes the pyramid attention module to improve the performance of the detection model. The feature pyramid attention module is introduced into deep features, and the feature pyramid attention structure is combined with global context information to better learn object features. The global attention upsampling module is introduced into low-level features, and the global information provided by global pooling is used as a guide to select low-level features. The object detection model can more fully acquire the features of important information and selectively suppress irrelevant features, thereby improving the detection accuracy of the algorithm. To verify the performance of the proposed algorithm, it is used to detect airplanes, storage tanks, ships, baseball diamonds, and running tracks in remote sensing images, and its performance is compared with that of other algorithms. Experiments prove that the proposed algorithm has better detection performance and can improve the detection accuracy of each object in remote sensing images.