Original title: RLIF: Interactive Imitation Learning as Reinforcement Learning
Authors: Jianlan Luo, Perry Dong, Yuexiang Zhai, Yi Ma, Sergey Levine
The article compares two learning methods, reinforcement learning, and interactive imitation learning, both valuable for skill acquisition in robotics. While imitation learning, like DAgger, employs experts to refine data, it faces challenges in handling distributional shifts without manually set rewards. This study explores a novel approach—employing reinforcement learning but using user interventions as rewards. Unlike DAgger, it doesn’t rely on near-optimal experts and can improve upon suboptimal human input. The research introduces a unified framework analyzing both methods’ performance, offering insights into their effectiveness. Evaluating these methods in high-dimensional tasks and robotic vision manipulation tasks reveals the superiority of the reinforcement learning approach, especially when dealing with suboptimal experts. This innovative method outperforms DAgger-like techniques across various tasks. For more details, the project website includes code and videos.
Original article: https://arxiv.org/abs/2311.12996