Addressing the limitations of existing reconstruction technologies, which are constrained by the Manhattan model assumption and exhibit insufficient applicability to irregular building shapes, as well as issues related to low model accuracy, dense triangular mesh faces, and weak robustness to chaotic and incomplete point clouds, we propose a 3D reconstruction technology specifically designed for irregular buildings, free from the constraints of the Manhattan model assumption. The proposed method includes a grid outlier removal technique based on eight-connected domains, a one-point Random Sample Consensus (RANSAC) method utilizing single points and their normal vectors for random plane segmentation to extract building planar structural elements, and a technique for 3D reconstruction of irregular buildings based on the selection of 3D polyhedral mesh faces. Validation with various datasets demonstrates that this method offers significant advantages over other reconstruction approaches in terms of model triangulation lightweightness and topological structure correctness.