ros2ai

AI Model for ROS 2.

Env Setup

Requestments

• ROS 2 Humble
• NVIDIA Driver 535.129.03
• CUDA 12.2
• torch==2.1.2
• transformers==4.37.0.dev0
• accelerate==0.25.0

Easy setup

sudo apt install git curl
git clone https://github.com/autowarefoundation/autoware.git -b main --single-branch
cd autoware
./setup-dev-env.sh

python3 -m pip install -U torch torchvision torchaudio git+https://github.com/huggingface/transformers accelerate

How To Use

Image-To-Text

blip_image_captioning_base

ros2 run ros2ai blip_image_captioning_base

blip_image_captioning_large

ros2 run ros2ai blip_image_captioning_large

pix2struct_textcaps_base

ros2 run ros2ai pix2struct_textcaps_base

vit_gpt2_image_captioning

ros2 run ros2ai vit_gpt2_image_captioning

Vision QA

vilt_b32_finetuned_vqa

ros2 run ros2ai vilt_b32_finetuned_vqa

Image-classification

resnet_50

ros2 run ros2ai resnet_50

zero-shot object detection

owlvit_base_patch16

ros2 run ros2ai owlvit_base_patch16

owlvit_base_patch32

ros2 run ros2ai owlvit_base_patch32

python3 src/ros2ai/ros2ai/vilt_b32_finetuned_vqa.py --subscription /blue/camera/image_raw