Traditional routing protocols and analytical routing optimization models face limitations in adapting to dynamic and complex environments such as SDN. Deep Reinforcement Learning (DRL) offers promise for addressing these challenges, but its intensive training phase hinders practical implementation. This paper presents a distributed DRL-based routing optimization solution in SDN, enhanced with Generative Adversarial Networks (GAN) to expedite agent training. Our approach, evaluated on a Containernet and OpenAI Gym-based testbed, effectively optimizes network traffic routes for diverse traffic classes, maximizing throughput. Activation of the GAN module significantly reduces training times, enhancing the feasibility of our solution for real-world deployment