The random sample consensus (RANSAC) algorithm is commonly used to estimate the parameters of the image transformation model based on matching point pairs in the feature-based image matching field. If the dataset of matching point pairs contains outliers, the conventional RANSAC algorithm may take a large number of iterations to obtain the desired model. To reduce mismatching, we propose the bidirectional optimal matching method, aiming to find robust parameters within a short time. The topology-consistency-based sampling method is introduced to divide the dataset into certain consensus sets, and sampling from each of them can reduce randomness. Then, all point pairs from a consensus set are used to estimate a model, and a point pair unsuitable for the model is deleted in each iteration, which is demonstrated to be faster than the conventional RANSAC. The superiority of the proposed method in fingerprint matching based on the scale-invariant feature transform is shown in experiments.