This work presents an interacting multiple pedestrian tracking method for monocular systems that incorporates a prior knowledge about the environment and about interactions between targets. Pedestrian motion being ruled by both environment and social aspects, we model these complex behaviors by considering 4 cases of motion: going straight; finding one's way; walking around and standing still. They are combined within an Interacting Multiple Model Particle Filter strategy. We model targets interactions with social forces, included as potential functions in the weighting process of the Particle Filter. We use different social force setups within each motion model to handle high level behaviors (collision avoidance, flocking.. .). We evaluate our algorithm on challenging datasets and show that such semantic information improves the tracker performance compared to the literature.