When we perform a collaborative task with another human, we are able to tell, to a certain extent, how things are going and more precisely if things are going well or not. This knowledge allows us to adapt our behavior. Therefore, we think it is desirable to provide robots with means to measure in real-time the Quality of the Interaction with their human partners. To make this possible, we propose a model and a set of metrics targeting the evaluation of the QoI in collaborative tasks through the measure of the human engagement and the online task effectiveness. These model and metrics have been implemented and tested within the high-level controller of an entertainment robot deployed in a mall. The first results show significant differences in the computed QoI when in interaction with a fully compliant human, a confused human and a non-cooperative one.