Tool wear substantially affects efficiency, accuracy, and cost in the manufacturing industry. The objective of this research was to propose an innovative method to detect tool wear offline on the basis of a deep learning classification model using audio signals. Tool wear experiments were conducted, and an International Organization for Standardization (ISO) standard was used to categorize tool wear into three levels. Using offline signals and three-level categorization can help non-professionals who operate cutting machines easily determine the tool’s condition. A mechanism was designed to collect the audio signals generated by a tool offline to avoid inconsistencies in manual operation. The collected signals were then converted into the frequency domain using the fast Fourier transform (FFT) to facilitate the observation of frequency variations of the signals, followed by the normalization and extraction of the frequency range. Data augmentation techniques were used to help generate more data to increase the robustness of the classification model, which was built on the basis of convolutional neural networks (CNNs). The results showed that the CNN model achieved an accuracy of 84.44% in classifying the wear of unseen tools, which outperformed the results obtained using the method in one of the previous research studies. In summary, we demonstrated that offline audio signals can be used for tool wear detection, providing a simple solution to detect tool wear without using the complicated online tool wear detection approach.