Multi-agent reinforcement learning (MARL) is gradually becoming an attractive research field of adaptive traffic signal control (ATSC). Nevertheless, in a multi-agent environment, some inherent disadvantages exist, such as the partial observability and non-stationarity caused by the constantly changing decision-making strategies of agents, which have been extensively researched but remain challenging. Herein, NCCLight, which is a fully scalable decentralized MARL model built around an independent advantage actor-critic (IA2C) under the background of ATSC, is rationally designed and validated to offer a feasible approach to realizing communication and coordination between multiple agents. In addition, guided by cognitive consistency theory, the constraint of neighborhood cognitive consistency (NCC) is constructed to achieve communication and coordination between multiple agents. More significantly, cognitive consistency theory is employed in MARL for ATSC for the first time, which is validated by a large number of experiments on both real and synthetic data. We hope that this work can serve as a pioneering reference owing to the better performance of NCCLight than of the most advanced ATSC based on MARL.