Aphasia is a language and communication disorder, which is primarily caused by brain damage, leading to impairments in language proficiency and comprehension abilities. Aphasia patients may struggle with auditory comprehension, spoken expressions, reading comprehension, and writing skills at different disorder levels. The golden period for treatment and language rehabilitation (TLR) is within the first six months. If aphasia patients practice daily under the guidance of a speech-language pathologist (SLP), they can gradually achieve their rehabilitation goals. During Mandarin language rehabilitation, aphasic patients undergo treatment through one-on-one communication interactions with the SLP. Then, the SLP evaluates the patient’s mandarin speech-language ability on a case-by-case basis using Mandarin Fluency Assessment Standards (MFAS). Therefore, rehabilitation plans can be personalized in accordance with the evaluation results with the aim of improving the patient’s listening, speaking, and reading abilities. To overcome the concerns of limited manpower and time consumption in treating language disorders, we propose a rehabilitation assistive tool for Mandarin-speaking aphasia patients (MSAPs), enabling them to practice in their daily lives and home environment. By leveraging mixed reality (MR), a virtual SLP (VSLP) is implemented, and then, various interaction scenarios can be simulated to support self-rehabilitation and training. This rehabilitation assistive tool allows aphasia patients to express their daily needs while also automatically evaluating their speech-language abilities. This SLP assistive tool integrates a You Only Look Once, version 9 (YOLOv9)-based object detector and a mel frequency cepstral coefficient (MFCC)-based feature extractor with a one-dimensional (1D) convolutional neural network (CNN) to enable the automated assessment of speech-language proficiency. Hence, through intensive, personalized, and repetitive self-training, MSAPs can effectively enhance rehabilitation outcomes.