Reward functions are central to reinforcement learning (RL) as they implicitly capture the optimal behavior of the learning agent. Since the behavioral policy of the RL agent is updated based on the provided reward signals, the choice of the reward function can have a very large impact on how fast the reinforcement learning algorithm converges. One of the most popular approaches to speed up the learning process is a method of reward design. Reward design is a technique that replaces original rewards with designed rewards to make the problem easier to learn. In our work, we propose different reward design strategies that guarantee the desired characteristics of the designed reward functions. In particular, we want our designed rewards to satisfy three main characteristics: Invariance i.e., Reward signals should capture desired behavior without reward bugs, Interpretability i.e., Reward signals should be easy to diagnose and verify, and Informativeness i.e., reward signals should lead to effective learning. The theoretical analysis and empirical evaluations across various RL tasks highlight the effectiveness of our proposed methods.
Please contact grad-office@mpi-sws.org for zoom details