IoT systems grow quickly and are massively present in urban areas. Their successful deployment requires autonomy that can be built on automated learning technologies such as Deep Learning. The IoT applications require important computational resources, rarely available on devices. Autonomous IoT systems require the computation power available on the edge and cloud servers in order to offload some tasks related to the supported applications and the underlying platforms. Task offloading constitutes a big challenge in autonomous IoT systems due to the huge number of IoT devices for scenar- ios of the family of smart cities. Managing task offloading in such contexts requires adaptive strategies capable of taking into consideration the rapid evolution of available resources and proposing efficient offloading solutions to all received requests. In this paper we use a Deep Reinforcement Learn- ing (DRL) approach capable of handling large state spaces, and resolve the optimization problem in this context, where other techniques can not scale efficiently. Our solution is based on a DRL agent that was developed in the ns3-gym framework and was tested on IoT system scenario imple- mented in the NS3 simulator. The results obtained show that the DRL agent can adapt quickly to resource evolution in the IoT system and can handle big number of demands fulfilling scalabilty requirements of autonomous IoT systems.