Self-driving vehicles have become increasingly popular in recent years. Because of this, the information fusion sensing method using radar and cameras has been widely adopted in vehicles. We use the vehicle camera sensor and robust image segmentation technology to solve its inherent shortcomings. The images used for image segmentation are obtained under adverse weather conditions, or the image object’s color and texture resemble the background. For such images, using the convolutional layer model for image segmentation as a feature extraction method usually leads to error. Any highly robust algorithms for image enhancement for self-driving operation will help alleviate problems related to driving safety. To ensure that the final image segmentation achieves the desired effect and reduces the error rate, we propose a new segmentation-twice method, which correctly classifies the object’s label. The test results of the simulation described in this paper show that this experiment correctly classifies the object’s label. It can provide accurate environmental perception information for autonomous vehicles, improve the segmentation effect of similar fusion background images, and reduce the error rate.