Connectivity in remote areas remains an unsolved problem, especially in developing countries. An adequate communications infrastructure can provide important services (e.g., telemedicine, virtual education, etc.) to communities of these regions. However, the management of these networks is complex because, on the one hand, they experience highly variable environmental conditions that require the continuous intervention of a human operator and, on the other hand, most of them are deployed in inaccessible areas. Self-management is proposed as an alternative solution to the management problem for these networks, thus reducing human intervention and, conversely, operational costs. This paper shows a network self-management architecture based on the SDN paradigm and Deep Reinforcement Learning algorithms that can learn the network dynamics and make autonomous decisions to optimize the network performance and adapt to the changing conditions of the environment to meet the QoS demands of the different network services. The proposed architecture has been successfully implemented in a simulated environment and was tested using a case study of QoS-aware routing optimization in a rural scenario.