Skip to content

Latest commit

 

History

History
72 lines (56 loc) · 2.82 KB

README.md

File metadata and controls

72 lines (56 loc) · 2.82 KB
Raw

This is the repo that reproduce the results for continuous control domains presented in the paper "Learning Multi-Level Hierarchies with Hindsight" (ICLR 2019) in Pytorch. Original repo in Tensorflow is at https://github.com/andrew-j-levy/Hierarchical-Actor-Critc-HAC-. The repo is inspired from "https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch". The difference is that I use the domains (Ant-Four-Room, Ant-Reacher, UR5-Reacher, Inverted-Pendulum) in the original paper, while the other repo does not (it uses two custom and simpler domains - one of them is included in this repo as well).

Setup

  • Install pip3 install -e .
  • You will need MuJoCo to run

Training (replace the number of layers with 2 or 3)

python3 run_hac.py --n_layers 2 --env hac-inverted-pendulum-v0 --retrain --timesteps 2000000 --seed 0 --group 2-level

Visualizing the policy saved at results/logs/hac-inverted-pendulum-v0/2-levels/0

python3 run_hac.py --n_layers 2 --env hac-inverted-pendulum-v0 --test --show --timesteps 2000000 --seed 0 --group 2-level

  • Inverted-Pendulum (3 levels)

- Mountain-Car (2 and 3 levels)

- UR5-Reacher (2 and 3 levels)

- Ant-Four-Rooms (2 and 3 levels)

- Ant-Reacher (2 and 3 levels)

Learning Curves (logged by wandb)

  • Inverted-Pendulum

  • Mountain-Car

  • UR5-Reacher

  • Ant-Four-Rooms

  • Ant-Reacher

Saved policies are stored in saved_policies/

To replay a saved policy: create a folder results/logs/hac/hac-ant-reacher-v0/2-levels and copy a saved policies into the folder, e.g., copy the whole 0/ folder in saved_policies/ant-reacher-2-levels. Then run the following command (the seed value must also be the name of the folder copied over):

python3 run_hac.py --n_layers 2 --env hac-ant-reacher-v0 --test --show --timesteps 2000000 --seed 0 --group 2-level