In this study, poly(vinylpyrrolidone) (PVP)-coated silver nanoparticles were used as a chromogenic substrate for the establishment of a colorimetric enzyme-linked immunosorbent assay (ELISA) based on localized surface plasmon resonance (LSPR) called “plasmonic ELISA”. The PVP-coated silver nanoparticles exhibit a specific color due to the LSPR. In addition, the LSPR absorbance strength of the PVP-coated silver nanoparticles is changed by the aggregation of the silver nanoparticles which depends on the hydrogen peroxide concentration. From these changes in the LSPR absorbance strength in relation to hydrogen peroxide concentration, plasmonic ELISA was established using a glucose oxidase (GOx)-conjugated secondary antibody and glucose for producing hydrogen peroxide by an enzymatic reaction. Furthermore, on the basis of plasmonic ELISA, a cartridge-based single-step plasmonic ELISA device was fabricated for the realization of a wash-free ELISA system. This cartridge-based ELISA system is constructed with a GOxantibody (GOx-conjugated secondary antibody) release pad, a primary antibody immobilized membrane, a glucose release pad, and a PVP-coated silver nanoparticle-containing hydrogel. By using this construction, the antigen concentration can be determined by introducing only a sample solution. As a result, with this cartridge-based ELISA system, the antigen–antibody reaction was successfully detected at different concentrations of antigen (0.1–10 μg/ml).