ROS Package bob_nviz

CI amd64 arm64

Ultra-lightweight, resource-efficient Software Renderer for ROS 2.

bob_nviz provides minimalist visual overlays for low-power devices (like ARM boards/Synology NAS) without requiring a GPU or heavy dependencies like SDL2 or FreeType. It renders directly to a raw BGRA buffer and outputs it to a FIFO pipe.

Key Features

  • CPU-Only Rendering: Zero GPU usage, ideal for headless servers or containers.

  • High-Performance Alpha Blending: Supports transparency for terminals and bitmaps using a specialized integer-based rendering engine (no floating-point overhead).

  • Minimal Dependencies: Requires only rclcpp and nlohmann_json.

  • Amiga-Style Aesthetic: 8x8 bitmap font with full UTF-8 to Latin-1 decoding.

  • Dynamic Layers: Manage multiple text terminals and bitmap canvases via JSON events.

  • Auto-Expiration: Elements can automatically disappear after a set time.

  • Latched State: The current layout is always available via events_changed.

Installation & Building

cd ~/ros2_ws/src
git clone https://github.com/bob-ros2/bob_nviz.git
cd ..
colcon build --packages-select bob_nviz
source install/setup.bash

Usage

Run the node (default output: /tmp/nano_fifo):

ros2 run bob_nviz nviz

Display the stream

Use ffplay to view the raw BGRA stream:

ffplay -f rawvideo -pixel_format bgra -video_size 854x480 -i /tmp/nano_fifo

ROS 2 API

Parameters

Parameter

Default

Description

width

854

Rendering width in pixels. (Env: NVIZ_WIDTH)

height

480

Rendering height in pixels. (Env: NVIZ_HEIGHT)

fps

30.0

Target frames per second. (Env: NVIZ_FPS)

fifo_path

/tmp/nano_fifo

Path to output raw BGRA pipe. (Env: NVIZ_FIFO_PATH)

queue_size

1000

ROS subscriber queue size. (Env: NVIZ_QUEUE_SIZE)

Topics

Topic

Type

Mode

Description

events

std_msgs/String

Sub

JSON commands to manage layers.

events_changed

std_msgs/String

Pub

Latched state of all active layers.

Dynamic

std_msgs/String

Sub

Topics for terminal content or bitmap data.

Dynamic Configuration

The node is controlled by sending JSON arrays to the events topic.

Common Fields

Every layer supports:

  • id (string, optional): Unique identifier.

  • area (array): [x, y, width, height]. Mandatory for creation.

  • action (string, optional):

    • add (default): Create or update an element.

    • remove: Remove element by id.

    • clear_all: Wipe the entire canvas (all layers).

1. String (Terminal)

Renders a rolling text terminal.

  • topic (string): ROS topic for incoming strings.

  • text (string, optional): Static text to display immediately.

  • title (string, optional): Title bar text.

  • mode (string):

    • default: Append incoming tokens (LLM style).

    • clear_on_new: Clear terminal before showing new message.

    • append_newline: Automatically add \n to every message.

  • font_size (int): Scaling factor (e.g., 16, 24, 32).

  • align (string): left (default), center, right.

  • text_color / bg_color: [R, G, B, A].

2. Bitmap (Canvas)

Displays raw image data (1-bit or 8-bit).

  • topic (string): Binary topic (std_msgs/UInt8MultiArray).

  • topic + "/hex": Hex-string topic (std_msgs/String).

  • depth (int): 1 (1-bit mask) or 8 (8-bit grayscale).

  • color: [R, G, B, A] for the foreground.

3. VideoStream (FIFO)

Displays raw video buffers from a pipe (headless streaming).

  • topic (string): Absolute path to the FIFO pipe (e.g., /tmp/stream_pipe).

  • source_width, source_height (int): Dimensions of the input raw frames.

  • encoding (string, optional): rgb (default) or bgr.

  • area (array): [x, y, w, h] for placement. (Note: Scaling is not supported; w/h should match source dimensions).

4. MarkerLayer (2D Projection)

Projects 3D ROS markers onto a 2D software plane (Front view: Y/Z plane). Optimized specifically for bob_face package 2D visualizations.

  • topic (string): ROS topic for visualization_msgs/MarkerArray.

  • area (array): [x, y, width, height] (drawing bounds).

  • scale (float): Mapping of ROS meters to pixels (Default: 1000.0).

  • offset_x / offset_y (float): Fine-tuning of the projection center.

  • title (string, optional): Title bar text.

  • exclude_ns (string, optional): Comma-separated list of marker namespaces to hide (e.g. "ir_sensors,collision").

5. Layer Actions

  • action: add (create/update) or remove (delete by id).

  • id: Unique identifier for the layer (mandatory for updates/removals).

Examples (ros2 topic pub)

Add a scrolling LLM Terminal

ros2 topic pub --once /eva/events std_msgs/msg/String 'data: "[{\"action\":\"add\", \"type\":\"String\", \"id\":\"main\", \"title\":\"LLM Stream\", \"topic\":\"/eva/llm\", \"area\":[50, 50, 400, 300], \"mode\": \"default\"}]"'

Show an Auto-Expiring Alert

ros2 topic pub --once /eva/events std_msgs/msg/String 'data: "[{\"type\":\"String\", \"id\":\"alert\", \"text\":\"System Warning!\", \"expire\": 5.0, \"area\":[227, 200, 400, 50], \"align\":\"center\", \"text_color\":[255, 0, 0, 255]}]"'

Add a Bitmap Icon (Heart) via Hex

# Register bitmap
ros2 topic pub --once /eva/events std_msgs/msg/String 'data: "[{\"type\":\"Bitmap\", \"id\":\"heart\", \"area\":[20, 20, 16, 16], \"topic\":\"/ui/heart\", \"depth\": 1, \"color\":[255, 50, 50, 255]}]"'

# Send data
ros2 topic pub --once /ui/heart/hex std_msgs/msg/String "data: '0C301E783FfC7FfE7FfE7FfE3FfC1Ff80fF007E003C0018'"

Add a Video Stream Overlay (FIFO)

# Register a 320x240 video stream from a pipe (using BGR encoding for OpenCV compatibility)
ros2 topic pub --once /eva/events std_msgs/msg/String 'data: "[{\"type\":\"VideoStream\", \"id\":\"cam\", \"topic\":\"/tmp/overlay_pipe\", \"area\":[500, 20, 320, 240], \"source_width\":320, \"source_height\":240, \"encoding\":\"bgr\"}]"'

Add a Marker Visualization (e.g. Lidar or Bob Face)

ros2 topic pub --once /eva/events std_msgs/msg/String 'data: "[{\"type\":\"MarkerLayer\", \"id\":\"face\", \"topic\":\"/eva/face_markers\", \"area\":[10, 10, 300, 300], \"scale\":1500.0, \"title\":\"Bob Face\"}]"'

Remove a Layer

ros2 topic pub --once /eva/events std_msgs/msg/String 'data: "[{\"action\":\"remove\", \"id\":\"alert\"}]"'

Global Reset (Clear All)

ros2 topic pub --once /eva/events std_msgs/msg/String 'data: "[{\"action\":\"clear_all\"}]"'

Headless & Software Rendering

bob_nviz is optimized for headless operation. If you are running in a Docker container or on a NAS without a GPU, ensure the following environment variables are set to force software rendering via Mesa:

export DISPLAY=:99
export QT_QPA_PLATFORM=xcb
export LIBGL_ALWAYS_SOFTWARE=1
export GALLIUM_DRIVER=llvmpipe

Required system packages for headless Qt: xvfb, libxcb-cursor0, libgl1-mesa-dri.

Audio System

bob_nviz is designed to work in conjunction with the bob_audio package. To ensure a stable stream without disconnects, a constant audio “heartbeat” is required.

1. The Mixer Pipeline

The bob_audio mixer node combines multiple audio sources (TTS, System Sounds, Music) into a single master pipe (/tmp/audio_master_pipe). This pipe is then consumed by FFmpeg.

  • Silence Heartbeat: Ensures FFmpeg always receives audio frames, even if no ROS nodes are currently speaking.

  • Dynamic Mixing: Use ros2 topic pub to send audio data to the mixer’s input topics.

2. Feeding External Audio

To inject audio files into the running stream (e.g., background music), use the provided helper script:

./scripts/feed_audio.sh my_song.mp3

Twitch Streaming

bob_nviz excels at streaming from headless, low-power ARM devices (RPi, Synology, etc.) by using efficient raw-video pipes and FFmpeg.

1. Configuration

Streaming is configured via environment variables. The easiest way is to use a .env file based on the provided template:

cp .env.template .env
# Edit .env and add your TWITCH_STREAM_KEY
cat .env

Variable

Default

Description

NAMESPACE

-

Optional ROS 2 namespace (e.g., /eva) applied to all nodes.

TWITCH_STREAM_KEY

-

Your Twitch Stream Key (passed as environment).

TWITCH_STREAM_KEY_FILE

-

Path to a file containing the Stream Key (e.g., /run/secrets/twitch_key).

INGEST_SERVER

rtmp://live-fra.twitch.tv/app/

Twitch ingest server URL.

NVIZ_WIDTH

854

Video width (e.g., 480p, 720p).

NVIZ_HEIGHT

480

Video height.

NVIZ_FPS

30

Target framerate.

NVIZ_FIFO_PATH

/tmp/nano_fifo

Raw video pipe location.

AUDIO_MASTER_PATH

/tmp/audio_master_pipe

Mixed audio pipe location.

Note on Security: If both TWITCH_STREAM_KEY_FILE and TWITCH_STREAM_KEY are provided, the file-based key takes precedence. This is recommended for Docker Secrets or encrypted setups.

2. Starting the Stream

The start_stream.sh script orchestrates the nviz node, the audio mixer, and FFmpeg in one go.

export TWITCH_STREAM_KEY="your_key_here"
./scripts/start_stream.sh

The script will:

  1. Create the necessary Unix pipes.

  2. Start the bob_audio mixer with a silence heartbeat.

  3. Start the nviz visualization node.

  4. Run an FFmpeg loop that encodes and pushes the combined stream to Twitch.

3. Cleanup

To stop everything, simply use Ctrl+C. The script includes a trap to cleanly shut down all background ROS processes.