In this paper we present an optimization-based method for controlling aerial manipulators in physical contact with the environment. The multi-task control problem, which includes hybrid force-motion tasks, energetic tasks, and po-sition/postural tasks, is recast as a quadratic programming problem with equality and inequality constraints, which is solved online. Thanks to this method, the aerial platform can be exploited at its best to perform the multi-objective tasks, with tunable priorities, while hard constraints such as contact maintenance, friction cones, joint limits, maximum and minimum propeller speeds are all respected. An on-board force/torque sensor mounted at the end effector is used in the feedback loop in order to cope with model inaccuracies and reject external disturbances. Real experiments with a multi-rotor platform and a multi-DoF lightweight manipulator demonstrate the applicability and effectiveness of the proposed approach in the real world.