The Internet of Things (IoT) is a technological topic with a very important societal impact. IoT application domains are various, such as smart cities, precision farming, smart factories, smart buildings, etc, and the diversity of these application domains is the source of the very high technological heterogeneity in the IoT, leading to interoperabil-ity issues. The semantic web principles and technologies are more and more adopted as a solution to these interoperability issues, leading to the emergence of a new domain, the Semantic Web Of Things (SWoT). Scientific contributions to the SWoT are many, and the diversity of archi-tectures in which they are expressed complicates comparison. To unify the presented state-of-the-art architectures, we propose an architectural pattern, Lower, Middle and Upper Node (LMU-N). LMU-N provides a reading grid used to classify processes to which the SWoT community contributes, and to describe how the semantic web impacts the IoT. A survey, based on this reading grid, capturing the integration of semantics into the IoT is the core of this paper. Then, the evolution of the semantic web to adapt to the IoT constraints is described as well, in order to give a twofold view of the convergence between the IoT and the semantic web toward the SWoT.