Visual perception is the primary modality for robots to perceive environments and plan collision-free motion. However, coupling conventional visual-sensor processing with the deep deterministic policy gradient (DDPG) algorithm in complex unstructured environments causes three deficiencies—inefficient use of visual features, weak perception of unknown regions, and sparse feedback—which limit the environmental adaptability of vision-based robot navigation. To this end, in this paper, we propose an improved DDPG algorithm integrating a self‑attention (SA) mechanism, hindsight experience replay (HER), and an intrinsic curiosity module (ICM), called ICM-DDPG-HER. The SA module in the front‑end visual branch uses scaled dot‑product attention to enhance task‑relevant features while suppressing redundant responses. HER, combined with a progressive learning strategy, relabels failed trajectories with hypothetical goals, converting them into valid samples and alleviating reward sparsity. A dual reward mechanism fusing curiosity-driven intrinsic and environment-driven extrinsic rewards compensates for sparse informative feedback and promotes active exploration. In Gazebo simulations, against the deep Q-network (DQN), DDPG, DDPG-HER, and SA-DDPG-HER, the proposed algorithm achieves the highest episodic reward in static (8.00 vs 5.45/4.10/1.01/−3.14) and dynamic (26.61 vs 24.50/18.57/15.81/7.30) environments and shortens the planned path by 4–40% in dynamic tests. The results indicate faster convergence, greater robustness, and better environmental-perception performance, enabling efficient obstacle avoidance and rapid target reaching in unknown complex environments.