The use of hybrid electric vehicles (HEVs) has become more prevalent than before because of their many merits in terms of energy saving and environmental protection. However, as HEVs use an internal combustion engine (ICE) and electric motors, they also have vibration problems that originate from the inherent ICE structure and/or power transfer shifting between the ICE and the electric motors. To find an appropriate control method for reducing the vibration of HEVs, we simulated vibrations from the power transmission line, elastomers, clutch system, engine damper, half-shaft, tires, damper-bypass clutch, and mode shift clutch of HEVs using the driver, ICE, motor/generator, battery, transmission system, mode shift clutch, damper-bypass clutch, and flexible shaft models. These models were simulated separately with different control methods such as pulse cancellation (PC), mode shift clutch speed control (MCC), damper-bypass clutch engagement control (DBCC), half-shaft oscillation feedback control with proportional integral controller (HOFC-PIC), and the combined control method of PC-DBCC-MCC-HOFC-PIC. MATLAB/Simulink was used for the simulation. Although each method was effective in reducing the vibration of certain parts of an HEV, the combined control method was the most effective in mitigating the vibration. In particular, the combined control method reduced the vibration significantly from the resonance of different parts of the HEV’s powertrain, including the flexible shaft, tire, ICE, and mode shift at 5, 11, 16, and 2.5 Hz. The results of this study need to be supplemented with further research but can be used as the basis to set up an effective control strategy for the HEV’s powertrain.