Owing to the increasingly serious greenhouse effect and rising global temperatures, pest reproduction and metabolism will accelerate, which will lead to significant reductions in crop yields. To date, many studies have applied deep learning models to pest identification tasks. However, there are many pest types with similar shapes, so in this study, we propose a parallel deep learning model with an attention mechanism module to improve the classification of tomato pest species. We used a public dataset and selected Bemisia tabaci, Helicoverpa armigera, Myzus persicae, Spodoptera exigua, Spodoptera litura, Thrips palmi, Tetranychus urticae, and Zeugodacus cucurbitae. These eight common tomato pests were selected with a total of 412 original images. The original images were enhanced to 1,655 images through horizontal flipping and angle rotation. The proposed EXM-Net extracted image features on the basis of Xception and MobileNetV2, added an ECA attention mechanism before the global average pooling layer, and then used the convolution operation to fuse the two model outputs to enhance model performance. The accuracy, precision, recall, F1-score, and PR-AUC score after data augmentation were 98.72, 98.44, 98.86, 99.41, and 99.76%, respectively. After experiments and testing on different datasets, it was confirmed that EXM-Net performs better than a single model and has a high degree of generalization ability. The proposed EXM-Net uses two deep learning models to extract and fuse different features to make up for important features missed by a single model, and combines the attention mechanism module to improve model efficiency and generalization capabilities.