Skip to content

Latest commit

 

History

History
215 lines (153 loc) · 7.79 KB

README.md

File metadata and controls

215 lines (153 loc) · 7.79 KB
Raw

Turtlebot4 tutorial

The following tutorial will help you get started with rai using Turtlebot4 simulation. The step by step video tutorial is available here.

Step by step instructions

  1. Install nav2 packages

       sudo apt install "ros-${ROS_DISTRO}-nav2-*"

  2. Clone and install rai

    1. Follow steps from 1. Setting up the workspace
    2. Then follow steps from 2. Build the project

  3. O3DE simulation setup

    1. Download turtlebot4 simulation binary matching your platform from :

      • Ubuntu 22.04 & ros2 humble: link
      • Ubuntu 24.04 & ros2 jazzy: link
      cd rai
# ubuntu 22.04 ~ humble
wget https://robotec-ml-roscon2024-demos.s3.eu-central-1.amazonaws.com/ROSCON_Release/Turtlebot4Tutorial_1.0.0_jammyhumble.zip

# ubuntu 24.04 ~ jazzy
wget https://robotec-ml-roscon2024-demos.s3.eu-central-1.amazonaws.com/ROSCON_Release/Turtlebot4Tutorial_1.0.0_noblejazzy.zip

    2. Unzip it:

      # for Ubuntu 22.04 jammy and ros2 humble
unzip -d src/examples/turtlebot4/simulation Turtlebot4Tutorial_1.0.0_jammyhumble.zip

# for Ubuntu 24.04 noble and ros2 jazzy
unzip -d src/examples/turtlebot4/simulation Turtlebot4Tutorial_1.0.0_noblejazzy.zip

  4. Setup your LLM vendor in config.toml and configure access keys as in docs/vendors.md. OpenAI or AWS Bedrock are recommended for models, since current local ollama models don't support vision & tool calling. ollama and llama:3.2 as an embedding model generate acceptable quality of robot's indentity in this tutorial creating identity

  5. Configure rai_whoami_node (based on "Your robot identity in RAI" tutorial):

    1. Create whoami package for turtlebot4 in src/examples/turtlebot4

      ./scripts/create_rai_ws.sh --name turtlebot4 --destination-directory src/examples

    TIP
    Skip steps 2-4 by downloading generated files here unzip them to src/examples/turtlebot4_whoami/description/generated with a command: unzip -d src/examples/turtlebot4_whoami/description turtlebot4_whoami_generated.zip

    1. Download official turtlebot4 data sheet into src/examples/turtlebot4_whoami/description/documentation

    2. Download image of turtlebot4 into src/examples/turtlebot4_whoami/description/images

    3. Create robot's identity. Run the parse_whoami_package. This will process the documentation, building it into a vector database, which is used by RAI agent to reason about its identity.

      ./scripts/parse_whoami_package.sh src/examples/turtlebot4_whoami
# you will be asked to press `y` to continue

    4. Build the workspace which now includes the new package

      deactivate # if poetry env is activated
colcon build --symlink-install

    5. Ensure turtlebot4_whoami package has been built:

      . ./setup_shell.sh
ros2 pkg list | grep turtlebot4

    Congratulations! Your rai_whoami_node is configured. In the following steps your RAI agent will assume a turtlebot4 personality.

  6. Run rai agent:

    ros2 launch ./src/examples/turtlebot4/turtlebot.launch.xml \
    game_launcher:=./src/examples/turtlebot4/simulation/TurtleBot4DemoGamePackage/TurtleBot4Demo.GameLauncher

    You should be able to see a simulation scene.

    TIP
    Press 1, 2 or 3 on the keyboard when simulation window to change the camera. Use W,A,S,D to move the camera.

  7. Open you internet browser and go to localhost:8501

  8. You can interact with your RAI agent through the chat. On the left you can communicate with the RAI HMI Agent. On the right you will see status of missions that were send to the RAI Node and are executed on your robot.

    • HMI Agent is responsible for human-robot interaction.
    • RAI Node is responsible for executing tasks on the robot.

    For example you can try the following prompts:

    • Testing robot's identity & RAG:
      • who are you?
      • what is the voltage of the battery and the diameter of the wheels?
    • Testing ROS 2 connection:
      • What are your ros2 interfaces?
      • What do you see?
      • Spin yourself left by 45 degrees and drive 1 meter forward

Troubleshooting

My robot doesn't have an identity.

  1. Query rai_whoami_node for the identity manually

    ros2 service call /rai_whoami_identity_service std_srvs/srv/Trigger

  2. Make sure all required files were generated correctly in the turtlebot4_whoami package and have similar sizes to those listed below.

    ls -sh src/examples/turtlebot4_whoami/description/
total 52K
4.0K documentation  4.0K images   28K index.faiss  8.0K index.pkl  4.0K robot_constitution.txt  4.0K robot_identity.txt

    You can also check the contents of robot_identity.txt file (it is generated by LLM, but should be similar to the one below).

    cat src/examples/turtlebot4_whoami/description/robot_identity.txt
I am a TurtleBot® 4, a robotics learning platform designed for education and
research in robotics. This next-generation mobile robot is built on the iRobot®
Create® 3 mobile base and comes fully assembled with ROS 2 installed and configured,
making it accessible for beginners and experienced developers alike.
...

    If files above are incorrect please check your vendor configuration is as described in docs/vendors.md and rerun point 4 of this tutorial.

Robot doesn't move in the simulation.

  1. Make sure you can see ros2 topic simulation binary is running

    1. Run the binary manually:

      cd src/examples/turtlebot4/simulation/
./TurtleBot4DemoGamePackage/TurtleBot4Demo.GameLauncher -bg_ConnectToAssetProcessor=0

    2. Check ros2 topics:

      ros2 topic list

    3. Verify if Turtlebot simulation binary publishes such topics:

      /camera_color_info
/camera_image_color
/clock
/cmd_vel
/parameter_events
/rosout
/scan
/tf

  2. Make sure navigation stack actions work correctly:

    1. Run binary as in the step above.

    2. Run navigation stack:

      cd src/examples/turtlebot4
./run-nav.bash

    3. Check available actions:

      ros2 action list

    4. Verify if such actions are available:

      /backup
/navigate_to_pose
/spin

    5. Try to run one of them from command line:

      ros2 action send_goal /spin nav2_msgs/action/Spin "{target_yaw: 3.14}"