In this paper, we report a continuation of our research based on our previous works, in which a fishing vessel recognition technique was addressed for the first time in the literature. We also propose a heading direction unification technique that boosts the performance of the fishing vessel identification model. This proposed technique is based on the finding that the recognition rate was improved when the recognition model in the originally proposed method was trained using an image database of fishing vessels with a unified heading direction. Accordingly, a model was developed to recognize the heading directions of fishing vessels as a pretreatment for the originally proposed method. Once a fishing vessel image was recognized as heading against the unified direction, it was flipped horizontally to improve the recognition rate. The model was experimentally validated to have an accuracy of up to 98.82%, and it required only 3.2 MB of memory.