The commonly used artificial visual interpretation and existing object-oriented computer automatic recognition methods have some disadvantages, such as low efficiency and insufficient accuracy in recognizing earthquake-induced building damage in unmanned aerial vehicle (UAV)-based images. In this paper, we report the latest progress in research on machine learning algorithms in artificial intelligence, then propose a new method of recognizing earthquake-induced building damage. Using the bag of words (BoW) model, scale-invariant feature transformation (SIFT) characteristics were clustered to build an eigenvector tag library with K clustering centers as visual words. After images were expressed by visual words as eigenvectors with unified dimensions, a histogram intersection kernel (HIK) was then employed to construct the histogram intersection kernel support vector machine (HIK-SVM) to classify images and recognize earthquake-induced building damage. Building damage due to the magnitude 6.0 earthquake that occurred in Luxian, Sichuan Province on September 16, 2021 was analyzed as an example. When the proposed method was applied to recognize earthquake damage using UAV-based images, the average recognition accuracy reached 91.7%. The experimental results verified the feasibility and validity of the proposed method.