This paper proposes a new algorithm called States Path Finder (SPF) to solve the general prehensile manipulation problem for multiple robots and multiple objects. Using a constraint graph to represent the problem, SPF finds a list of transitions from the initial state to the goal state, computes a corresponding set of waypoints and connects the waypoints to form a continuous, collision-free solution path. We benchmark SPF on several problem instances with varying characteristics. The results suggest SPF's generality and also its higher efficiency in comparison to Manipulation-RRT, an existing RRT-like algorithm also based on the constraint graph.