Reliable emotion recognition from acoustic sensor signals is a pivotal yet challenging component for achieving natural and empathetic human–robot interaction. We propose a dedicated, deployable audio processing module. The core of this module is a novel integration of the Wavelet Scattering Transform (WST) as a robust preprocessing front-end with a multiscale neural network. Specifically, the WST standardizes raw, sensor-acquired audio signals into translation-invariant and deformation-stable feature maps, effectively mitigating input variability. Building upon this, we construct a Wavelet Scattering Feature Map–Multiscale Network (WSM-MSN) that synergistically combines hierarchical Convolutional Neural Network (CNN) branches for extracting fine-grained local affective features with a High-Resolution Network (HRNet) branch to capture and fuse multiscale contextual dependences, thereby significantly enhancing recognition precision. Extensive evaluations on four datasets of affective acoustic signals (EMODB, RAVDESS, IEMOCAP, eNTERFACE’05) demonstrate the module’s superiority. It achieves consistent unweighted average recall (UAR) improvements of 1.62%, 5.28%, 4.44%, and 3.37%, respectively, over traditional scattering methods and surpasses other comparative algorithms.