Abstract:
Object-centric models represent an image as a set of vectors describing the objects present in the image. This image representation allows for explicit modeling of interactions between objects, which accelerates the convergence of reinforcement learning algorithms. A drawback of using such models in reinforcement learning tasks is their typical use in a frozen state, which can lead to problems caused by distributional shift from the data on which these models were originally trained. This paper presents an algorithm for fine-tuning object-centric models by splitting the training procedure into a model finetuning phase and a policy learning phase. Experiments confirm the effectiveness of this approach, although there exist scenarios where this method underperforms compared to using frozen models.