The behavioral observation of animals in zoos is indispensable for their health management and the improvement of their breeding environment. However, the day-to-day recording of animal behaviors is time-consuming for zookeepers. Hence, we aim to automatically generate animal behavioral observation reports, called “ethograms,” in cooperation with the Sapporo Maruyama Zoo. While studies using contact sensors [e.g., accelerometers, global positioning system (GPS) , and radio frequency identification (RFID)] have had some success in zoos, noncontact sensors (e.g., cameras and microphones) tend to be avoided because of frequent occlusion and the need for nighttime detection. However, noncontact sensors are preferable to contact sensors owing to animal welfare concerns. Here, we propose a method for automatic elephant behavior recognition based on elephant tracking information using video from surveillance cameras. In particular, we focus only on “eating,” which is difficult to detect accurately because it requires relatively long-term observation. Therefore, we solve the problem by using a method based on temporal action localization (TAL), which is a task of predicting when and where a target action is performed over a relatively lengthy period. The TAL method has been applied mainly to humans and less to animals. In our experiments, the average precision of eating behavior detection using TAL was 0.853. The results show that TAL is also effective in animal behavior recognition.