In this study, a type of hysteretic nonlinear model of giant magnetostrictive materials (GMMs) based on the inverse magnetostrictive effect was developed, and the nonlinear dynamic characteristics of a giant magnetostrictive sensor were studied. A Van der Pol nonlinear item was introduced to describe the hysteretic phenomena of GMMs, and the coupling relationship between strain and frequency was determined. The results of a forecast test show that the GMM model can describe well the inverse magnetostrictive effect in different frequencies. On the basis of the GMM model, the magneto-mechanical coupled model of a giant magnetostrictive sensor was developed, and the relationship between output voltage and input excitation force was determined. The nonlinear dynamic characteristics of the giant magnetostrictive sensor were discussed, and the phenomena of accuracy aggravation in high frequency and delay of a giant magnetostrictive sensor were explained. The experimental results show that the model can describe the actual response of the giant magnetostrictive sensor. The new model of the giant magnetostrictive sensor has a simple form and is easy to analyze in theory, which is helpful for application in measure and control fields.