In recent years, smart speakers have been widely used to control related smart home appliances. For people with speech disabilities, smart speakers cannot be used to perform related voice control tasks. To help people with speech difficulties and effectively control related smart facilities, we use the Internet of Things to develop a smart home gesture-based control system. The prototype of the system is established by means of sensors and a control panel system. When a sensor detects a control gesture command, it can send related control signals to a controlled smart device. The controlled devices used in this study include smart home appliances such as doors and windows, air conditioners, curtains, dehumidifiers, and air purifiers. The system functions include the opening/closing of doors, windows, and curtains; the activation/deactivation of heating and cooling systems, dehumidifiers, and air purifiers; temperature and humidity adjustment; and fan operation. With the development of this system, it is possible for people with language disabilities and elderly people to control the operation of various smart devices with gestures. It can also solve the inconvenience of needing to look for remote controls for various facilities. Moreover, it can save precious time and provide a safe, convenient, and smart home living environment for people with speech disabilities.