In the field of small-sample domains, since the introduction of the megatrend diffusion (MTD) method, its effectiveness and practicality have been demonstrated in various studies. Recently, with the popularity of generative deep learning, researchers have integrated Wasserstein generative adversarial networks (WGANs) with the MTD method and proposed a novel framework called WGAN-MTD for virtual data generation. It uses the MTD for producing estimates, which restricts the generative model’s output value range and generates effective synthetic samples. However, the validity of developing virtual samples using real-world data containing outliers remains controversial, and the weight clipping method in WGAN has been shown to affect the stability of model training. In this study, we propose an advanced framework in which the boxplot is integrated with a penalization term to limit the effect of outliers, especially from small samples. The proposed framework considers the convolutional layers to capture local information features and lower the complexity of the model by reducing the number of parameters between the input and output layers. Additionally, we adopt a WGAN with the gradient penalty (GP) method instead of WGAN alone to improve the training stability and precision in the generative model. Experimental results demonstrate that both the boxplot and the penalization term enhance the accuracy of the generative models for small datasets containing outliers.