Computer vision technologies have recently been maliciously used to spread misleading information. Because of the low cost of video production, misleading videos have been used for attack ads, criminal fraud, and even political manipulation, which could undermine social progress. Hence, it is important to develop a system for detecting misleading videos that can help a fact-checking center detect misleading videos more efficiently. In this research, we propose a novel video retrieval system based on a deep convolutional neural network that extracts deep visual informatics to retrieve visually alike videos from annotated misleading videos. Moreover, we propose dual-stage confidence filtering that considers both video- and image-level retrieval. This is one of the latest studies on misleading video detection using video-level retrieval, and preliminary experiments demonstrate its superior retrieval performance, enabling it to be applied in real-world applications.