Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
@RobRoyce
RobRoyce authored Aug 29, 2024
1 parent b6ff249 commit daf74bc
Showing 1 changed file with 33 additions and 154 deletions.
187 changes: 33 additions & 154 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,89 +1,41 @@
<!-- Header block for project --> <hr>
<div align="center">

[//]: # ( [INSERT YOUR LOGO IMAGE HERE &#40;IF APPLICABLE&#41;])
<!-- ☝️ Replace with your logo (if applicable) via ![](https://uri-to-your-logo-image) ☝️ -->
<!-- ☝️ If you see logo rendering errors, make sure you're not using indentation, or try an HTML IMG tag -->
<h1 align="center">ROSA - Robot Operating System Agent</h1>
<!-- ☝️ Replace with your repo name ☝️ -->
[//]: # " [INSERT YOUR LOGO IMAGE HERE (IF APPLICABLE)]"
<h1 align="center">ROSA - The ROS Agent 🤖</h1>
</div>
<pre align="center">
ROSA is an AI Agent designed to interact with ROS-based robotics systems<br>using natural language queries.
The ROS Agent (ROSA) is designed to interact with ROS-based<br>robotics systems using natural language queries. 🗣️🤖
</pre>

<div align="center">

![Static Badge](https://img.shields.io/badge/ROS_1-Noetic-blue)
![Static Badge](https://img.shields.io/badge/ROS_2-Humble|Iron|Jazzy-blue)
![PyPI - License](https://img.shields.io/pypi/l/jpl-rosa)
![ROS 1](https://img.shields.io/badge/ROS_1-Noetic-blue)
![ROS 2](https://img.shields.io/badge/ROS_2-Humble|Iron|Jazzy-blue)
![License](https://img.shields.io/pypi/l/jpl-rosa)
[![SLIM](https://img.shields.io/badge/Best%20Practices%20from-SLIM-blue)](https://nasa-ammos.github.io/slim/)

![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/nasa-jpl/rosa/ci.yml?branch=main&label=main)
![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/nasa-jpl/rosa/ci.yml?branch=dev&label=dev)
![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/nasa-jpl/rosa/publish.yml?label=publish)
![PyPI - Version](https://img.shields.io/pypi/v/jpl-rosa)
![PyPI - Downloads](https://img.shields.io/pypi/dw/jpl-rosa)
![Main Branch](https://img.shields.io/github/actions/workflow/status/nasa-jpl/rosa/ci.yml?branch=main&label=main)
![Dev Branch](https://img.shields.io/github/actions/workflow/status/nasa-jpl/rosa/ci.yml?branch=dev&label=dev)
![Publish Status](https://img.shields.io/github/actions/workflow/status/nasa-jpl/rosa/publish.yml?label=publish)
![Version](https://img.shields.io/pypi/v/jpl-rosa)
![Downloads](https://img.shields.io/pypi/dw/jpl-rosa)

</div>
<!-- Header block for project -->

> **Important:** 📚 Be sure to check out our [Wiki](https://github.com/nasa-jpl/rosa/wiki) (work-in-progress) for detailed documentation, guides, and FAQs!
> [!IMPORTANT]
> 📚 **New to ROSA?** Check out our [Wiki](https://github.com/nasa-jpl/rosa/wiki) for documentation, guides and FAQs!
ROSA is an AI agent that can be used to interact with ROS1 _and_ ROS2 systems in order to carry out various tasks.
It is built using the open-source [Langchain](https://python.langchain.com/v0.2/docs/introduction/) framework, and can
be adapted to work with different robots and environments, making it a versatile tool for robotics research and
development.
ROSA is your AI-powered assistant for ROS1 and ROS2 systems. Built on the [Langchain](https://python.langchain.com/v0.2/docs/introduction/) framework, ROSA helps you interact with robots using natural language, making robotics development more accessible and efficient.

## Features and Roadmap

- [x] Support for both ROS1 and ROS2
- [x] Generate system reports using fuzzy templates
- [x] Read, parse, and summarize ROS log files
- [x] Use natural language to run various ROS commands and tools, for example:
* "Give me a list of nodes, categorize them into `navigation`, `perception`, `control`, and `other`"
* "Show me a list of topics that have publishers but no subscribers"
* "Set the `/velocity` parameter to `10`"
* "Echo the `/robot/status` topic"
* "What is the message type of the `/robot/status` topic?"
- [x] Control the TurtleSim robot in simulation using ROSA
- [x] Easily adapt ROSA for your robot by adding new tools and prompts
- [ ] Use multi-modal models for vision, scene understanding, and more (in-progress)
- [ ] Web-based user interface with support for voice commands (in-progress)
- [ ] Text and speech modalities for human-robot interaction (in-progress)

### Support for Popular Robots

ROSA already supports any robot built with ROS1 or ROS2, but we are also working on custom agents for some popular
robots. These custom agents go beyond the basic ROSA functionality to provide more advanced capabilities and features.

- [x] Custom Agent for the [TurtleSim robot](http://wiki.ros.org/turtlesim) (see [turtle_agent](src/turtle_agent))
- [ ] Custom Agent for [NASA JPL's Open Source Rover](https://github.com/nasa-jpl/open-source-rover)
- [ ] Custom Agent for the [TurtleBot](https://www.turtlebot.com/)
- [ ] Custom Agent for the [Spot Robot](https://bostondynamics.com/products/spot/)

## Contents

* [Quick Start](#quick-start)
* [Changelog](#changelog)
* [FAQ](#frequently-asked-questions-faq)
* [Contributing Guide](#contributing)
* [License](#license)
* [Support](#support)

## Quick Start

This guide provides a quick way to get started with ROSA.
## 🚀 Quick Start

### Requirements
- Python 3.9+
- ROS Noetic or higher

1. Python 3.9 or higher
2. ROS Noetic (or higher)

**Note:** ROS Noetic uses Python3.8, but LangChain requires Python3.9 or higher. To use ROSA with ROS Noetic,
you will need to create a virtual environment with Python3.9 or higher and install ROSA in that environment.

### Setup Instructions

### Installation
```bash
pip3 install jpl-rosa
```
Expand All @@ -98,110 +50,38 @@ rosa = ROSA(ros_version=1, llm=llm)
rosa.invoke("Show me a list of topics that have publishers but no subscribers")
```

## TurtleSim Demo

We have included a demo that uses ROSA to control the TurtleSim robot in simulation. To run the demo, you will need to
have Docker installed on your machine.

The following video shows ROSA reasoning about how to draw a 5-point star, then executing the necessary commands to do
so.
For detailed information on configuring the LLM, please refer to our [Model Configuration Wiki page](https://github.com/nasa-jpl/rosa/wiki/Model-Configuration).

https://github.com/user-attachments/assets/77b97014-6d2e-4123-8d0b-ea0916d93a4e

### Setup and run

1. Clone this repository
2. Configure the LLM in `src/turtle_agent/scripts/llm.py`
3. Run the demo script: `./demo.sh`
4. Build and start the turtle agent:
## Adapting ROSA for Your Robot 🔧

```bash
catkin build && source devel/setup.bash && roslaunch turtle_agent agent
```
ROSA is designed to be easily adaptable to different robots and environments. You can create custom agents by either inheriting from the `ROSA` class or creating a new instance with custom parameters.

5. Run example queries: `examples`
For detailed information on creating custom agents, adding tools, and customizing prompts, please refer to our [Custom Agents Wiki page](https://github.com/nasa-jpl/rosa/wiki/Custom-Agents).

# Adapting ROSA for Your Robot

ROSA is designed to be easily adaptable to different robots and environments. To adapt ROSA for your robot, you
can either (1) create a new class that inherits from the `ROSA` class, or (2) create a new instance of the `ROSA` class
and pass in the necessary parameters. The first option is recommended if you need to make significant changes to the
agent's behavior, while the second option is recommended if you want to use the agent with minimal changes.
## TurtleSim Demo 🐢

In either case, ROSA is adapted by providing it with a new set of tools and/or prompts. The tools are used to interact
with the robot and the ROS environment, while the prompts are used to guide the agents behavior.
We have included a demo that uses ROSA to control the TurtleSim robot in simulation. To run the demo, you will need to have Docker installed on your machine. 🐳

## Adding Tools
The following video shows ROSA reasoning about how to draw a 5-point star, then
executing the necessary commands to do so.

There are two methods for adding tools to ROSA:

1. Pass in a list of @tool functions using the `tools` parameter.
2. Pass in a list of Python packages containing @tool functions using the `tool_packages` parameter.

The first method is recommended if you have a small number of tools, while the second method is recommended if you have
a large number of tools or if you want to organize your tools into separate packages.

**Hint:** check `src/turtle_agent/scripts/turtle_agent.py` for examples on how to use both methods.

## Adding Prompts

To add prompts to ROSA, you need to create a new instance of the `RobotSystemPrompts` class and pass it to the `ROSA`
constructor using the `prompts` parameter. The `RobotSystemPrompts` class contains the following attributes:

- `embodiment_and_persona`: Gives the agent a sense of identity and helps it understand its role.
- `about_your_operators`: Provides information about the operators who interact with the robot, which can help the agent
understand the context of the interaction.
- `critical_instructions`: Provides critical instructions that the agent should follow to ensure the safety and
well-being of the robot and its operators.
- `constraints_and_guardrails`: Gives the robot a sense of its limitations and informs its decision-making process.
- `about_your_environment`: Provides information about the physical and digital environment in which the robot operates.
- `about_your_capabilities`: Describes what the robot can and cannot do, which can help the agent understand its
limitations.
- `nuance_and_assumptions`: Provides information about the nuances and assumptions that the agent should consider when
interacting with the robot.
- `mission_and_objectives`: Describes the mission and objectives of the robot, which can help the agent understand its
purpose and goals.
- `environment_variables`: Provides information about the environment variables that the agent should consider when
interacting with the robot. e.g. $ROS_MASTER_URI, or $ROS_IP.

## Example

Here is a quick and easy example showing how to add new tools and prompts to ROSA:

```python
from langchain.agents import tool
from rosa import ROSA, RobotSystemPrompts
https://github.com/user-attachments/assets/77b97014-6d2e-4123-8d0b-ea0916d93a4e

For detailed instructions on setting up and running the TurtleSim demo, please refer to our [TurtleSim Demo Guide](https://github.com/nasa-jpl/rosa/wiki/Guide:-TurtleSim-Demo) in the Wiki.

@tool
def move_forward(distance: float) -> str:
"""
Move the robot forward by the specified distance.
:param distance: The distance to move the robot forward.
"""
# Your code here ...
return f"Moving forward by {distance} units."

## 📘 Learn More

prompts = RobotSystemPrompts(
embodiment_and_persona="You are a cool robot that can move forward."
)
- [🗺️ Roadmap](https://github.com/nasa-jpl/rosa/wiki/Feature-Roadmap)
- [🏷️ Releases](https://github.com/nasa-jpl/rosa/releases)
- [❓ FAQ](https://github.com/nasa-jpl/rosa/wiki/FAQ)

llm = get_your_llm_here()
rosa = ROSA(ros_version=1, llm=llm, tools=[move_forward], prompts=prompts)
rosa.invoke("Move forward by 2 units.")
```

## Changelog

See our [CHANGELOG.md](CHANGELOG.md) for a history of our changes.
See our [releases page](https://github.com/nasa-jpl/rosa/releases) for our key versioned releases.

## Frequently Asked Questions (FAQ)

Questions about ROSA? Please see our [FAQ](https://github.com/nasa-jpl/rosa/wiki#frequently-asked-questions-faq) section
in the Wiki.

## Contributing

Expand All @@ -226,7 +106,6 @@ Key points of contact are:
---

<div align="center">
ROSA: Robot Operating System Agent <br>
ROSA: Robot Operating System Agent 🤖<br>
Copyright (c) 2024. Jet Propulsion Laboratory. All rights reserved.
</div>

0 comments on commit daf74bc

Please sign in to comment.