In this study, we propose a bottom-sediment classification system using seabed images. Seabed videos using a digital video (DV) camera were taken for a fishery-resource survey in the scatter scallop fishing grounds in Hokkaido, Japan. Scatter scallop fishing is a method of waiting for naturally growing young shells on the seabed. We acquired about 0.02 km2 of seabed videos in 2015 in Monbetsu. We cannot survey as wide a range using the DV camera as we can using sonar; however, we can obtain high-resolution 75 × 42 cm2 seabed images. We can classify bottom sediment in a narrower range than bottom-sediment classification methods using sonar. Our research aims to classify the following four types of bottom sediment: sand, ballast, gravel, and shell bank. The bottom sediment affects the growth of scallops and the survival rate of young shells. Therefore, understanding the undersea environment is important. In this study, we used a convolutional neural network (CNN) for the bottom-sediment classification from seabed images. Using CNN enables automatic and high-speed classification. This experiment showed average accuracies of about 95% for three types of bottom sediment and 76.5% for the fourth type (gravel). Moreover, we created a fishing-ground map based on the bottom sediment for visualizing the seabed environment.