Fruit quality classification in the consumer market has become a considerable burden following the decrease in the young adult population engaged in agriculture in Taiwan owing to its labor-intensiveness. We propose a system to identify the external quality of fruit, which utilizes a camera as an image sensor and an artificial intelligence algorithm as a classifier. This application is suitable for real operating environments. Fruits are mainly detected by the “you only look once” (YOLO)-V3 algorithm, with the designated fruit continuously tracked using the characteristics of the image, such as size, height, width, etc., and the quality of fruit is detected during the tracking process. Finally, the switching gap of the application distinguishes fruits of different quality. The proposed application detects round fruit such as apples, oranges, and lemons using our newly developed process. We also provide a graphical user interface to control and collect data, evaluate models, and monitor the entire system operation to improve the efficiency of the proposed application. The experimental results show that the proposed application achieves an accuracy rate of up to 88% after testing on 6000 fruit images.