色婷婷色综合,亚洲天堂2014,亚洲精品2区,亚洲午夜一区二区

<返回

Reinforcement Learning from Diverse Human Preferences

Wanqi Xue, Bo An, Shuicheng Yan, Zhongwen Xu

IJCAI 2024 Conference

August 2024

Keywords: Reinforcement Learning, Human Preferences, Human Feedback, Rewards

Abstract:

The complexity of designing reward functions has been a major obstacle to the wide application of deep reinforcement learning (RL) techniques. Describing an agent s desired behaviors and properties can be difficult, even for experts. A new paradigm called reinforcement learning from human preferences (or preference-based RL) has emerged as a promising solution, in which reward functions are learned from human preference labels among behavior trajectories. However, existing methods for preference-based RL are limited by the need for accurate oracle preference labels. This paper addresses this limitation by developing a method for crowd-sourcing preference labels and learning from diverse human preferences. The key idea is to stabilize reward learning through regularization and correction in a latent space. To ensure temporal consistency, a strong constraint is imposed on the reward model that forces its latent space to be close to the prior distribution. Additionally, a confidence-based reward model ensembling method is designed to generate more stable and reliable predictions. The proposed method is tested on a variety of tasks in DMcontrol and Meta-world and has shown consistent and significant improvements over existing preference-based RL algorithms when learning from diverse feedback, paving the way for real-world applications of RL methods.

View More PDF>>

主站蜘蛛池模板: 博白县| 奉贤区| 连江县| 赫章县| 定南县| 仪陇县| 左权县| 舒兰市| 呼和浩特市| 贵港市| 遂溪县| 桃江县| 南开区| 神农架林区| 界首市| 吉安县| 襄樊市| 新兴县| 宁化县| 宜丰县| 盐亭县| 濮阳市| 江陵县| 玉龙| 黔西| 万宁市| 辽源市| 化州市| 浏阳市| 乐亭县| 印江| 无锡市| 察雅县| 临清市| 兴业县| 桑日县| 禄丰县| 云和县| 资溪县| 凤山市| 襄汾县|