色婷婷色综合,亚洲天堂2014,亚洲精品2区,亚洲午夜一区二区

<返回

Reinforcement Learning from Diverse Human Preferences

Wanqi Xue, Bo An, Shuicheng Yan, Zhongwen Xu

IJCAI 2024 Conference

August 2024

Keywords: Reinforcement Learning, Human Preferences, Human Feedback, Rewards

Abstract:

The complexity of designing reward functions has been a major obstacle to the wide application of deep reinforcement learning (RL) techniques. Describing an agent s desired behaviors and properties can be difficult, even for experts. A new paradigm called reinforcement learning from human preferences (or preference-based RL) has emerged as a promising solution, in which reward functions are learned from human preference labels among behavior trajectories. However, existing methods for preference-based RL are limited by the need for accurate oracle preference labels. This paper addresses this limitation by developing a method for crowd-sourcing preference labels and learning from diverse human preferences. The key idea is to stabilize reward learning through regularization and correction in a latent space. To ensure temporal consistency, a strong constraint is imposed on the reward model that forces its latent space to be close to the prior distribution. Additionally, a confidence-based reward model ensembling method is designed to generate more stable and reliable predictions. The proposed method is tested on a variety of tasks in DMcontrol and Meta-world and has shown consistent and significant improvements over existing preference-based RL algorithms when learning from diverse feedback, paving the way for real-world applications of RL methods.

View More PDF>>

主站蜘蛛池模板: 连云港市| 全州县| 绍兴市| 永和县| 镇江市| 龙南县| 遵化市| 徐汇区| 怀宁县| 宣威市| 土默特右旗| 正安县| 石泉县| 外汇| 山阴县| 东台市| 漠河县| 景泰县| 开鲁县| 永顺县| 南京市| 玉山县| 万荣县| 胶南市| 库尔勒市| 北流市| 通州市| 黎川县| 扶绥县| 兴海县| 双辽市| 黎川县| 通许县| 彭泽县| 哈尔滨市| 灵台县| 郴州市| 庆城县| 民县| 道真| 金湖县|