In the application of face recognition, emotion recognition has gradually received attention. The main reason is that human emotions can best reveal human behaviors, feelings, thoughts, and intentions. By analyzing and interpreting the characteristics of human faces, we can learn about a person’s current emotional state. To effectively find out the facial expression feature information and classify expressions, we use image sensors to capture facial expressions and propose a Feature-fusion-based Convolutional Neuro-fuzzy Classifier (FF-CNFC) to implement facial emotion recognition. In the FF-CNFC model, the neuro-fuzzy network classifier replaces the traditional fully connected neural network classifier for reducing the number of adjustable parameters. In addition, different fusion methods, including channel global maximum/average pooling fusion, global maximum/average pooling fusion, and network feature mapping methods, were used for the comparison of expression classification. In our experiment, we used the Multi Pose, Illumination, Expressions (Multi-PIE) face data set. The confusion matrix was used as the evaluation standard, and the accuracy, sensitivity, precision, and F1-score were calculated to evaluate performance of the model and judge it's quality. Experimental results indicated that the accuracy, sensitivity, precision, and F1-score of the proposed FF-CNFC model with global maximum pooling fusion are 99.60, 99.58, 99.58, and 99.58%, respectively, and are higher than those of other similar models. In addition, the proposed FF-CNFC model has a smaller number of parameters than the other models.