In response to the energy crisis, regions worldwide are continuously expanding their photovoltaic (PV) installation capacities. However, as the scale grows, the rapid identification of faults in PV panels has become a significant challenge. Although unmanned aerial vehicles equipped with IR thermal imaging sensors can quickly, cost-effectively, and noninvasively detect faults in PV panels, traditional detection methods still face issues with accuracy. This is because fault regions in IR images are often small, exhibit significant scale variations, and have unclear features. To address these issues, in this paper, we propose an IR thermal imaging-based PV panel fault detection model, PV–You Only Look Once (YOLO). The model first predicts a set of weights through input features and performs a weighted fusion of all expert convolution kernels to generate dynamic convolution kernels for specific inputs, thereby enhancing the detection ability for multiscale objects. Additionally, a multibranch dilated convolution structure is used to build convolution kernels with a large receptive field, which are then fused with the original spatial pyramidal pooling-fast module to further improve the detection accuracy of multiscale fault points. Finally, a multi-attention fusion mechanism is introduced, which enhances fault detection accuracy by parallelly fusing local structural attention, channel attention, and spatial attention. Experimental results show that PV–YOLO improves by 5.2 percentage points over the YOLOv8n model in Mean Average Precision at Intersection over Union 0.5, reaching 85.3%, while the recall rate increases by 4.2 percentage points to 80.5%. Compared with other mainstream algorithms, PV–YOLO achieves a better balance between detection accuracy and model complexity.