Positron annihilation spectroscopy (PAS) is used for the quantification of cavities in a matrix. Although PAS is sometimes considered a nondestructive measurement method, it is worth investigating the interaction of positrons with the matrix during PAS. Here, we have demonstrated that defects are generated in silver-doped phosphate glasses during positron annihilation measurement and that radiophotoluminescence (RPL) is observed after irradiation. There is a linear correlation between the irradiation duration and the observed RPL intensity of the glasses. We observed RPL after a high irradiation dose even after conventional thermal annealing at 360 °C for 1 h. The formation of Ag2+ species detectable by electron spin resonance (ESR) was confirmed. From the concentration of ESR-active Ag2+ and Ag2+ species, it is expected that approximately 0.1% of Ag+ cations were changed after 5 days of positron irradiation.