Online Learning a Symbolic Abstraction of Actions in Hierarchical RL with Formal Methods
Résumé
Reasoning about actions, planning in open-ended learning requires a hierarchical abstractions of action models and benefits from the use of symbolic methods for goal representation as they structure knowledge for efficient and transferable learning. However, the existing Hierarchical Reinforcement Learning (HRL) approaches relying on symbolic reasoning are often limited as they require a manual goal representation. A symbolic goal representation must preserve information about the environment dynamics. We propose an automatic subgoal discovery via an emergent representation that abstracts (i.e., groups together) sets of environment states that have similar roles in the task, using formal verification methods. We create a HRL algorithm that learns online this representation along with the policies. We show on navigation tasks that the learned representation is interpretable and results in data efficiency.
Origine | Fichiers produits par l'(les) auteur(s) |
---|