In recent years, with the progress of Industry 4.0, intelligent robotic technologies have been widely used in various fields, such as assembly on a production line and performing minimally invasive surgery. We propose the application of a five-degree-of-freedom robot arm to the automatic testing of smartphones. A camera is set up on a robot arm and the images captured by the camera are used for visual servo control of the robot arm. Image processing is performed by MATLAB, which first converts the red–green–blue (RGB) color space into the hue–saturation–value (HSV) color space, then binarizes the image, and then filters out noises. The control system applies LabVIEW to integrate the image and fuzzy control for the angles of each joint in the movement of the robot arm. By converting the coordinates through the Denavit–Hartenberg (D–H) model, the degrees of joints can be updated by visual servoing without the need for camera calibration. Fuzzy theory is used to obtain the error of the robot arm and then send the command to adjust the movement of the robot arm. Experimental results show that the proposed control system can drive the robot arm to press the required buttons of test smartphones.