Installation and Building

The Buffer NG service is distributed as a Docker container, which simplifies its installation and usage. You can pull the image from the GitHub Container Registry.

Savant-RS Version

The Buffer NG service is part of the Savant-RS project. The Savant-rs version used in Savant must match the version of the Buffer NG service.

How to check the savant-rs version used in Savant

You can check the Savant-rs version used in Savant by running the following command:

SAVANT_VERSION=0.5.15
docker run -it ghcr.io/insight-platform/savant-adapters-gstreamer:$SAVANT_VERSION


============
== Savant ==
============

Savant Version 0.5.15
Savant RS Version 1.10.1
DeepStream Version 7.0

So, you can see that the version of savant-rs is 1.10.1. Thus, you need to use the same version of the Buffer NG service.

We support the following tags for the Buffer NG service:

latest - the latest version (do not use it, we use it for development purposes);
${SAVANT_RS}-rolling - the latest version of the Buffer NG service for the specific version of Savant-rs, e.g. 1.10.1-rolling;
$SAVANT_VERSION - the specific version matching the version of Savant, e.g. 0.5.15;
savant-latest - the latest version of the Buffer NG service for the latest version of Savant.

So, if you want to use the latest Savant, you probably need to use the ${SAVANT_RS}-rolling tag for the Buffer NG service. If you want to use the release version of Savant, you can use the $SAVANT_VERSION tag.

Image naming

X86-64: savant-buffer-ng-x86;
ARM64: savant-buffer-ng-arm64.

Using The Docker Container

Pull the Docker image:

docker pull ghcr.io/insight-platform/savant-buffer-ng-x86:savant-latest

# or for arm64

docker pull ghcr.io/insight-platform/savant-buffer-ng-arm64:savant-latest

Run the container with your configuration file:

docker run \
   -v /path/to/your/config.json:/opt/etc/configuration.json \
   -v /path/to/your/python/handlers:/opt/python \
   -v /path/to/buffer/storage:/tmp/buffer \
   ghcr.io/insight-platform/savant-buffer-ng-x86:savant-latest

# or for arm64

docker run \
   -v /path/to/your/config.json:/opt/etc/configuration.json \
   -v /path/to/your/python/handlers:/opt/python \
   -v /path/to/buffer/storage:/tmp/buffer \
   ghcr.io/insight-platform/savant-buffer-ng-arm64:savant-latest

Here, you need to:

Mount your configuration file into the container at /opt/etc/configuration.json
Mount your Python handler modules directory to make them accessible to the service
Mount a persistent volume for the buffer storage to ensure data persistence across container restarts
Ensure your configuration file references the correct paths for Python modules and buffer storage

The configuration file defines the ingress sources, egress destinations, buffer settings, and Python handler specifications.

Note

Sometimes, you may need extra Python modules to be installed in the container. In this situation, you use the image as a base image and install the modules in the container in the Dockerfile.

Environment Variables

The Buffer NG service supports environment variable substitution in configuration files. You can use the following pattern:

docker run \
   -e ZMQ_SRC_ENDPOINT="tcp://source:5555" \
   -e ZMQ_SINK_ENDPOINT="tcp://sink:5556" \
   -e BUFFER_PATH="/tmp/buffer" \
   -e BUFFER_LEN=1000000 \
   -e BUFFER_THRESHOLD_PERCENTAGE=90 \
   -e STATS_LOG_INTERVAL=60 \
   -e PYTHON_MODULE_ROOT=/opt/python \
   -v /path/to/your/config.json:/opt/etc/configuration.json \
   -v /path/to/your/python/handlers:/opt/python \
   -v /path/to/buffer/storage:/tmp/buffer \
   ghcr.io/insight-platform/savant-buffer-ng-x86:savant-latest

Building From Source

If you prefer to build the service from source, you can do so using Docker or Cargo:

Clone the Savant RS repository:

git clone https://github.com/insight-platform/savant-rs.git

Build the service with Docker:

docker build -t buffer-ng -f docker/services/Dockerfile.buffer_ng .

Build the service with Cargo:
```
cargo build --release -p buffer_ng
```
Note

Because of various reasons we use dynamic linking in savant-rs. Thus, you need to copy the dependencies to the distribution. Consult with our Docker-based build to find out how to do it.

Development Setup

For development purposes, you can run the Buffer NG service directly from the source:

LOGLEVEL=info cargo run -p buffer_ng services/buffer_ng/assets/configuration.json

This approach is useful when developing custom Python handlers or modifying the service configuration during development.

Note

Make sure you have Python development dependencies installed and the correct Python version (3.8+) available in your development environment. Also ensure you have adequate disk space for the buffer storage and that the buffer directory is writable.

Dependencies

The Buffer NG service has the following key dependencies:

RocksDB: For persistent message storage and high-performance key-value operations
ZeroMQ: For message transport and communication with upstream and downstream services
Python 3.10+: For Python handler support and extensibility
Savant-RS Core: For message handling and transport abstractions

These dependencies are automatically handled when using the Docker container or following the build instructions.