In this study, we developed a thermal error prediction (TEP) model and employed the error compensation in real time in a computer numerical control (CNC) milling machine with three axes in actual cutting operations. Thirty-three PT-100 sensors were used in each critical part of the machine to collect temperature data during cutting. K-means was adopted to select eight crucial temperature sensors from the 33 temperature sensors, and PCA+K-means was used to determine seven critical temperature sensors from the 33 sensors to apply a model for TEP. In this study, we made the prediction model from a back propagation neural network (BPNN). The number of sensors chosen as the critical temperature sensors constitutes the input layer of the BPNN. In contrast, the three neurons in the output layer represent the deformation of X, Y, and Z. After training the model to predict errors, it is brought into the control system for real-time TEP. We conducted a 6 h actual cutting experiment to verify the effect of error compensation, and the average three-axis thermal error was decreased from 50 to 14 μm by the K-means selection method. The PCA+K-means selection method reduced the average thermal three-axis error from 50 to 11 μm as compared with the previous measurements. The results show that these two methods can effectively improve the machining accuracy of the workpiece by combining the BPNN model with a compensated real-time TEP model.