Awesome Visual RL

This is a collection of research papers on Visual Reinforcement Learning (Visual RL) and other vision-related reinforcement learning.

If you find some ignored papers, feel free to open issues, or email Qi Wang / GuoZheng Ma / Yuan Pu. Contributions in any form to make this list more comprehensive are welcome. 📣📣📣

If you find this repository useful, please consider giving us a star 🌟.

Feel free to share this list with others! 🥳🥳🥳

Papers

format:
- publisher **[abbreviation of proposed model]** title [paper link] [code link]

🔷 Model-Based   🔶 Model-Free

2024

• 🔶 ICLR 2024 Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stages [Paper] [Torch Code]
• 🔷 ICLR 2024 [TD-MPC2] TD-MPC2: Scalable, Robust World Models for Continuous Control [Paper] [Torch Code]
• 🔶 ICLR 2024 Oral [PTGM] Pre-Training Goal-based Models for Sample-Efficient Reinforcement Learning [Paper] [Torch Code]
• 🔶 ICLR 2024 Spotlight [DrM] DrM: Mastering Visual Reinforcement Learning through Dormant Ratio Minimization [Paper] [Torch Code]
• 🔷 ICLR 2024 [DreamSmooth] DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing [Paper]
• 🔷 ICLR 2024 Oral [R2I] Mastering Memory Tasks with World Models [Paper] [JAX Code]
• 🔶 ICLR 2024 Spotlight [PULSE] Universal Humanoid Motion Representations for Physics-Based Control [Paper] [Torch Code]
• 🔷 ICML 2024 Oral [Dynalang] Learning to Model the World With Language [Paper] [JAX Code]
• 🔷 ICML 2024 [HarmonyDream] HarmonyDream: Task Harmonization Inside World Models [Paper] [JAX Code]
• 🔶 ICML 2024 Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning [Paper]
• 🔶 ICML 2024 [BeigeMaps] BeigeMaps: Behavioral Eigenmaps for Reinforcement Learning from Images [Paper]
• 🔷 NeurIPS 2024 [CoWorld] Making Offline RL Online: Collaborative World Models for Offline Visual Reinforcement Learning [Paper] [Website] [Torch Code]
• 🔷 NeurIPS 2024 [DIAMOND] Diffusion for World Modeling: Visual Details Matter in Atari [Paper] [Torch Code]
• 🔶 RLC 2024 [SADA] A Recipe for Unbounded Data Augmentation in Visual Reinforcement Learning [Paper][Torch Code]
• 🔷 IEEE IOT [CarDreamer] CarDreamer: Open-Source Learning Platform for World Model based Autonomous Driving [Paper] [Code]
• 🔷 arXiv 2024.10 [LS-Imagine] Open-World Reinforcement Learning over Long Short-Term Imagination [Paper] [Torch Code]
• 🔶 arXiv 2024.10 [MENTOR] MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learning [Paper]
• 🔷 arXiv 2024.6 [DLLM] World Models with Hints of Large Language Models for Goal Achieving [Paper]
• 🔷 arXiv 2024.5 [Puppeteer] Hierarchical World Models as Visual Whole-Body Humanoid Controllers [Paper] [Torch Code]

2023

• 🔶 ICLR 2023 [CoIT] On the Data-Efficiency with Contrastive Image Transformation in Reinforcement Learning [Paper] [Torch Code]
• 🔷 ICLR 2023 [MoDem] MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations [Paper] [Torch Code]
• 🔶 ICLR 2023 [TED] Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learning [Paper] [Torch Code]
• 🔶 ICLR 2023 Spotlight [VIP] VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training [Paper] [Torch Code]
• 🔷 ICML 2023 Oral Mastering the Unsupervised Reinforcement Learning Benchmark from Pixels [Paper]
• 🔶 ICML 2023 On Pre-Training for Visuo-Motor Control: Revisiting a Learning-from-Scratch Baseline [Paper] [Torch Code]
• 🔶 ICCV 2023 [CG2A] Improving Generalization in Visual Reinforcement Learning via Conflict-aware Gradient Agreement Augmentation [Paper]
• 🔷 NeurIPS 2023 [STORM] STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2023 [HAVE] Hierarchical Adaptive Value Estimation for Multi-modal Visual Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2023 [PIE-G] Hierarchical Adaptive Value Estimation for Multi-modal Visual Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2023 Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2023 [TACO] TACO: Temporal Latent Action-Driven Contrastive Loss for Visual Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2023 [CMID] Conditional Mutual Information for Disentangled Representations in Reinforcement Learning [Paper][Torch Code]
• 🔷 arXiv 2023.1 [DreamerV3] Mastering Diverse Domains through World Models [Paper][JAX Code][Torch Code]

2022

• 🔶 ICLR 2022 [DrQ-v2] Local Feature Swapping for Generalization in Reinforcement Learning [Paper][Torch Code]
• 🔶 ICLR 2022 [CLOP] Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning [Paper][Torch Code]
• 🔷 ICML 2022 [TD-MPC] Temporal Difference Learning for Model Predictive Control [Paper][Torch Code]
• 🔶 ICML 2022 [DRIBO] DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck [Paper][Torch Code]
• 🔷 ICML 2022 [DreamerPro] DreamerPro: Reconstruction-Free Model-Based Reinforcement Learning with Prototypical Representations [Paper][TF Code]
• 🔶 IJCAI 2022 [CCLF] CCLF: A Contrastive-Curiosity-Driven Learning Framework for Sample-Efficient Reinforcement Learning [Paper]
• 🔶 IJCAI 2022 [TLDA] Don’t Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2022 [PIE-G] Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2022 Efficient Scheduling of Data Augmentation for Deep Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2022 Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels? [Paper]
• 🔶 NeurIPS 2022 [A2LS] Reinforcement Learning with Automated Auxiliary Loss Search [Paper][Torch Code]
• 🔶 NeurIPS 2022 [MLR] Mask-based Latent Reconstruction for Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2022 [SRM] Spectrum Random Masking for Generalization in Image-based Reinforcement Learning [Paper][Torch Code]
• 🔷 NeurIPS 2022 Deep Hierarchical Planning from Pixels. [Paper][TF Code]
• 🔷 NeurIPS 2022 Spotlight [Iso-Dream] Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models [Paper][Torch Code]
• 🔶 TPAMI 2022 [M-CURL] Masked Contrastive Representation Learning for Reinforcement Learning [Paper]
• 🔷 CoRL 2022 [DayDreamer] DayDreamer: World Models for Physical Robot Learning [Paper] [TF Code]
• 🔷 arXiv 2022.3 [DreamingV2] DreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction [Paper]

2021

• 🔶 ICLR 2021 Spotlight [DrQ] Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels [Paper][Torch Code]
• 🔶 ICLR 2021 [MixStyle] Domain Generalization with MixStyle [Paper][Torch Code]
• 🔶 ICLR 2021 [SPR] Data-Efficient Reinforcement Learning with Self-Predictive Representations [Paper][Torch Code]
• 🔷 ICLR 2021 [DreamerV2] Mastering Atari with Discrete World Models [Paper][TF Code][Torch Code]
• 🔶 ICML 2021 [SECANT] Self-Expert Cloning for Zero-Shot Generalization of Visual Policies [Paper] [Torch Code]
• 🔶 NeurIPS 2021 [PlayVirtual] Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learning [Paper][Torch Code]
• 🔶 NeurIPS 2021 [EXPAND] Widening the Pipeline in Human-Guided Reinforcement Learning with Explanation and Context-Aware Data Augmentation [Paper]
• 🔶 NeurIPS 2021 [SVEA] Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation [Paper] [Torch Code]
• 🔶 NeurIPS 2021 [UCB-DrAC] Automatic Data Augmentation for Generalization in Reinforcement Learning [Paper] [Torch Code]
• 🔷 ICRA 2021 [Dreaming] Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstruction [Paper]

2020

• 🔷 ICML 2020 [Plan2Explore] Planning to Explore via Self-Supervised World Models [Paper][TF Code][Torch Code]
• 🔶 ICML 2020 [CURL] CURL: Contrastive Unsupervised Representations for Reinforcement Learning [Paper] [Torch Code]
• 🔷 ICLR 2020 [DreamerV1] Dream to Control: Learning Behaviors by Latent Imagination [Paper][TF Code][Torch Code]

2018

• 🔷 NeurIPS 2018 Oral World Models [Paper]

Other Vision-Related Reinforcement Learning Papers

2024

• 🔷 ICLR 2024 Oral Predictive auxiliary objectives in deep RL mimic learning in the brain [Paper]
• 🔶 ICLR 2024 Oral [METRA] METRA: Scalable Unsupervised RL with Metric-Aware Abstraction [Paper] [Torch Code]
• 🔶 ICLR 2024 Spotlight Selective Visual Representations Improve Convergence and Generalization for Embodied AI [Paper] [Torch Code]
• 🔶 ICLR 2024 Spotlight Towards Principled Representation Learning from Videos for Reinforcement Learning [Paper] [Torch Code]

Technical Blog

• Can RL From Pixels be as Efficient as RL From State?

Contributors

Qi Wang Shanghai Jiao Tong University

GuoZheng Ma Nanyang Technological University

Yuan Pu Shanghai Artificial Intelligence Laboratory (OpenDILab)

(Back to top)