Skip to content

Batch Transform Component

The Batch Transform component provides the ability to batch messages on a topic and republish them on a different topic.

The size of the batch is determined by the BufferSize and Timeout configuration variables.

The batched data is published once the buffer is full, or the timeout has been reached.

Batch Transform Configuration Variables

The following table describes the configuration variables that can be used with the Batch Transform component:

Parameter Type Description Valid Values Required Y/N
Bus String The name of the Bus component to use The name of an existing Bus component Y
Scheduler String The Scheduler component to use The name of an existing Scheduler component Y
ThreadPool String The name of the Threadpool component to use The name of an existing Threadpool component N
Logger String The name of the Logger component to use The name of an existing Logger component N
Pattern String The pattern to match on the Bus component for subscription Default: # N
Topic String The name of the topic on which to publish data The name of an existing topic Y
Sync Boolean Synchronization flag to set on the published data Valid values are as follows:
  • true
  • false (default)
 N
Metadata String The data about the batched data N
QueueSize Unsigned Integer The size of the queue for the subscriber to hold messages Default: 4 N
BufferSize Unsigned Integer The number of messages to batch before publishing them Default: 100 N
Timeout Unsigned Integer The interval in milliseconds for which batching occurs before the batch is published Default: 0 N
Priority Integer The priority of the scheduler thread The range is system-dependent

For example on Linux it is usually between 1 and 100 with 1 being the lowest priority and 100 being the highest priority 		N
Cookie Integer The cookie value for the subscriber A valid cookie N

Configuration Example

The following example configures a Batch Transform component: 

{
  "Bus": "bus",
  "Scheduler": "sched",
  "ThreadPool": "pool",
  "Logger": "logger",
  "Pattern": "data/raw",
  "Topic": "data/export",
  "Sync": true,
  "Metadata": {
    "MQTT_TOPIC": "demo"
  },
  "QueueSize": 4,
  "BufferSize": 100,
  "Timeout": 0,
  "Priority": 99,
  "Cookie": 255
}

Back to top