In this repository you will find everything you need to build and train a rotary inverted pendulum (also known as a furuta pendulum). Watch the build video here.
- There are many (hundreds!) videos of rotary inverted pendulum on the internet but we couldn't find good documentation on how to build one and how to control one. This projects aims to make building a furuta pendulum affordable and reproducible.
- We wanted to learn about RL. Balancing a pendulum is kind of the "hello world" of the field, it seemed like a good starting point.
- Get the supplies and follow the hardware instructions
- Follow the software instructions
- Deploy pre-trained models, run experiments and train the robot
If you have any question feel free to open an issue or DM me @armand_dpl.
- Plug-in the robot
- Run
sudo dmesg | grep ttyin terminal to find which port is used for the device - run
python tests/interactive_robot_self_test.py
During this project we leveraged Weights and Biases MLOps tools to make our life easier. You can find our experiments, pre-trained models and reports on our dashboard. You can also read more about Training Reproducible Robots with W&B here.
To make this robot work we built on top of existing work!
- We got the encoder precision and the idea to use a direct drive motor from the Quanser Qube design.
- We re-used bits from Quanser's code. Notably:
- their VelocityFilter class to compute the angular speeds
- their GentlyTerminating wrapper to send a zero command to the robot at the end of each episode
- their rotary inverted pendulum simulation
- their ActionLimiter class
- The arm assembly is inspired by this YouTube video by Mack Tang.
- The visualization we use for the simulation is copy-pasted from https://github.com/angelolovatto/gym-cartpole-swingup
- We use the StableBaselines3 library to train the robot.
- We implemented tricks from Antonin Raffin's talk at RLVS 2021.
- HistoryWrapper and continuity cost
- gSDE
- We use code from Federico Bolanos to read the encoders counters.