Skip to content
This repository has been archived by the owner on Mar 8, 2022. It is now read-only.

egrefen/impact-driven-exploration

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

This is an implementation of the method proposed in

RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments

by Roberta Raileanu and Tim Rocktäschel, published at ICLR 2020.

We propose a novel type of intrinsic reward which encourges the agent to take actions that result in significant changes to its representation of the environment state.

The code includes all the baselines and ablations used in the paper.

The code was also used to run the baselines in Learning with AMIGO: Adversarially Motivated Intrinsic Goals. See the associated repo for instructions on how to reproduce the results from that paper.

Citation

If you use this code in your own work, please cite our paper:

@inproceedings{
  Raileanu2020RIDE:,
  title={{RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments}},
  author={Roberta Raileanu and Tim Rockt{\"{a}}schel},
  booktitle={International Conference on Learning Representations},
  year={2020},
  url={https://openreview.net/forum?id=rkg-TJBFPB}
}

Installation

# create a new conda environment
conda create -n ride python=3.7
conda activate ride 

# install dependencies
git clone [email protected]:facebookresearch/impact-driven-exploration.git
cd impact-driven-exploration
pip install -r requirements.txt

Train RIDE on MiniGrid

cd impact-driven-exploration

OMP_NUM_THREADS=1 python main.py --model ride --env MiniGrid-ObstructedMaze-2Dlh-v0 

OMP_NUM_THREADS=1 python main.py --model ride --env MiniGrid-KeyCorridorS3R3-v0 \
--intrinsic_reward_coef 0.1 --entropy_cost 0.0005

Overview of RIDE

RIDE Overview

Results on MiniGrid

MiniGrid Results

Analysis of RIDE

Intrinsic Reward Heatmaps

State Visitation Heatmaps

Acknowledgements

Our vanilla RL algorithm is based on Torchbeast, which is an open source implementation of IMPALA.

License

This code is under the CC-BY-NC 4.0 (Attribution-NonCommercial 4.0 International) license.

About

impact-driven-exploration

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%