bob_transformers.itt

This ROS Node integrates Huggingface Transformers capabilities into ROS.

Classes

ITTNode

Basic image to text ROS Node

Functions

main([args])

Module Contents

class bob_transformers.itt.ITTNode

Bases: bob_transformers.base.BaseNode

Basic image to text ROS Node

pub

configure()

Handle configure for lifecycle node.

messages_callback(msg: std_msgs.msg.String)

String message callback.

Example: Passing message array with one or more messages:

[
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "path": "/path/to/image.png"},
            {
                "type": "text",
                "text": "What do you see on that image"},
            ],
    }
]

Example: Passing as dict and configure max token:

{
    "max_new_tokens": 128,
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "path": "/path/to/image.png"
                },
                {
                    "type": "text",
                    "text": "What do you see on that image"
                }
            ]
        }
    ]
}

Example: Passing as simple string <image-path> [<num-max-tokens>] <text>:

/path/to/image.png What do you see on that image
/path/to/image.png 128 What do you see on that image

Args:

msg (String): The received topic message.

Raises:

Exception: raises a

bob_transformers.itt.main(args=None)