ROS Package bob_moondream_msgs 

Overview

This package contains the custom ROS 2 service definitions required for interacting with the bob_moondream node. See also:

vikhyatk/moondream2 at Huggingface

Provided ROS Interfaces

This package provides the following services:

`VisualQuery.srv`

This is the primary, unified service for all interactions with the moondream node.

string prompt
---
string response

Request:

string prompt: The text prompt for the operation (e.g., the question for a query, the object name for detection). Can be an empty string for captioning.

Response:

string response: The result of the operation, returned as a string. This could be a text caption, an answer, or a JSON-formatted list of detections/points.

Usage in Other Packages

To use these services in another ROS 2 package, you must add bob_moondream_msgs as a dependency.

1. Add Dependency in package.xml

Add the following line to the package.xml of your node’s package:

<depend>bob_moondream_msgs</depend>

2. Find Package in CMakeLists.txt

In the CMakeLists.txt of your node’s package, you need to find this package:

find_package(bob_moondream_msgs REQUIRED)

3. Import and Use in a Python Node

You can then import and use the service definitions in your Python code like any other ROS interface:

# Import the service definition from this package
from bob_moondream_msgs.srv import VisualQuery

# Example of creating a client for the service
self.my_client = self.create_client(VisualQuery, '/services/moondream')

Shell Service Call Example

You can test or interact with a node that provides these services directly from the command line using ros2 service call. The following examples assume a node is offering the VisualQuery service on the /services/moondream topic.

Example 1: Requesting a Caption

The prompt is not used and should be empty.

ros2 service call /services/moondream/caption bob_moondream_msgs/srv/VisualQuery "{prompt: ''}"

Example 2: Asking a Visual Question

Provides a question in the prompt.

ros2 service call /services/moondream/visual_query bob_moondream_msgs/srv/VisualQuery "{prompt: 'How many chairs are in the room?'}"

Example 3: Detecting an Object

ros2 service call /services/moondream/object_detection bob_moondream_msgs/srv/VisualQuery "{prompt: 'the green plant'}"

Example 4: Finding a Point of Interest

ros2 service call /services/moondream/pointing bob_moondream_msgs/srv/VisualQuery "{prompt: 'the corner of the table'}"

Building the Package

This package must be built before any other packages that depend on it.

# From the root of your workspace
colcon build --packages-select bob_moondream_msgs

ROS Package bob_moondream_msgs