Skip to content

Export to InfluxDB

Overview

This example describes how to configure an application service to use proprietary built-in function “InfluxDBSyncWrite“ to export EdgeX events to InfluxDB 1.8.x or 2.0.x. Edge Xpert uses InfluxDB 2.0.x by default. Please refer to Upgrade from InfluxDB 1.x to 2.0.

Note

You can also take advantage of EdgeXpert Manger to create app services, which is the graphical user interface (GUI) of EdgeXpert. Refer to Supporting Services - InfluxDB for the example.

Prerequisite

  1. Understand Edge Xpert Application Services
  2. Launch Edge Xpert core services
      edgexpert up
    
    or use --secret option to run in security mode
      edgexpert up --secret 
    
  3. Launch InfluxDB and obtain the information

    • Influx DB server URL
    • Influx Organization
    • Influx Bucket
    • measurement
    • measurement field data type
    • timestamp precision
    • authentication token

    Info

    For InfluxDB 2.0.x, the authentication token can be either specified or auto-generated. InfluxDB 2.0.x container will take the value of DOCKER_INFLUXDB_INIT_ADMIN_TOKEN environment variable as the token value during setup. If DOCKER_INFLUXDB_INIT_ADMIN_TOKEN environment variable is not found during setup, InfluxDB 2.0.x will automatically generate the token, and you can obtain the token value through docker exec <influxdb container name> influx auth list --user admin --json command. Edge Xpert sets DOCKER_INFLUXDB_INIT_ADMIN_TOKEN to custom-token by default in the compose files.

Configure an application service to export events to InfluxDB 2.0.x

Edge Xpert offers a proprietary built-in function “InfluxDBSyncWrite“ to convert EdgeX events to InfluxDB points and then synchronously write to InfluxDB 2.0.x. If Edge Xpert CLI utility is installed, a sample configuration can be found in /usr/share/edgexpert/examples/app-configurable/influxdb-write/configuration.toml for InfluxDB 2.0.x.

  1. Copy and save the sample configuration file as a seperate file (for example, influxdb-write.toml) to your working directory
  2. Revise this configuration per information as obtained from the step 3 of Prerequisite. See InfluxDBSyncWrite configuration details.

    [Writable.Pipeline.Functions.InfluxDBSyncWrite]
      [Writable.Pipeline.Functions.InfluxDBSyncWrite.Parameters]
        InfluxDBServerURL = 'http://localhost:8086'
        InfluxDBOrganization = 'my-org'
        InfluxDBBucket = 'my-bucket'
        InfluxDBMeasurement = 'readings'
        InfluxDBValueType = 'integer'
        InfluxDBPrecision = 'us'
        Authmode = 'token'
        Secretpath='influxdb'
        Skipverify = 'true'
        PersistOnError = 'false'
        StoreEventTags = 'false'
    
  3. Configure authentication token

    Revise Writable.InsecureSecrets.influxdb configuration as shown below:

    [Writable.InsecureSecrets.influxdb]
      path = 'influxdb'
      [Writable.InsecureSecrets.influxdb.Secrets]
        token = 'custom-token'
    

    Note

    When the Application Services are launched in secure mode, run the following command to add the InfluxDB authentication token value into secret store. For more details about storing secrets in the secret store, refer to Secret Store.

    # add InfluxDB token to secret store
    docker run -it --network="edgexpert_edgex-network" curlimages/curl:latest \
        -X POST http://<your-app-service-container-name>:59700/v2/secret \
        --data '{"apiVersion": "v2", "path": "influxdb", "secretData": [{"key": "token","value": "custom-token"}]}'
    

  4. Once configuration is ready, launch an application service with such configuration:

    # Under secure mode
    edgexpert up --secret app-service --path=<path your configuration file>
    
    # Under insecure mode
    edgexpert up app-service --path=<path your configuration file>
    

  5. Now, an application service is up and ready to export events to InfluxDB. To simulate the device sending readings from southbound, we can start up virtual device service:

    # Under secure mode
    edgexpert up --secret device-virtual
    
    # Under insecure mode
    edgexpert up device-virtual
    

  6. There are various way to monitor and query those events being wrote to InfluxDB, please refer to InfluxDB v2.0 documentation for more details. The following sample uses the Influx command line interface inside the InfluxDB 2.0.x docker container (for example, run command docker exec -it influxdb bash) to query all records that were inserted within the last five minutes under my-bucket :

    # run the following command inside the InfluxDB 2.0.x docker container 
    # influx query 'from(bucket:"my-bucket") |> range(start:-5m)'
    Result: _result
    Table: keys: [_start, _stop, _field, _measurement, device, event_id, resource_name]
                       _start:time                      _stop:time           _field:string     _measurement:string           device:string                       event_id:string    resource_name:string                                       _value:float                      _time:time  
    ------------------------------  ------------------------------  ----------------------  ----------------------  ----------------------  ------------------------------------  ----------------------  -------------------------------------------------  ------------------------------  
    2021-05-25T12:58:45.731852264Z  2021-05-25T12:59:45.731852264Z                   value                readings     Random-Float-Device  12318221-d676-4cd4-8cba-ca063476d619                 Float32  0.00000000000000000000000000000006814705742712731  2021-05-25T12:59:26.629823000Z  
    Table: keys: [_start, _stop, _field, _measurement, device, event_id, resource_name]
                       _start:time                      _stop:time           _field:string     _measurement:string           device:string                       event_id:string    resource_name:string                  _value:float                      _time:time  
    ------------------------------  ------------------------------  ----------------------  ----------------------  ----------------------  ------------------------------------  ----------------------  ----------------------------  ------------------------------  
    2021-05-25T12:58:45.731852264Z  2021-05-25T12:59:45.731852264Z                   value                readings     Random-Float-Device  32d13053-9b94-499f-91a9-479cb5a76f3a                 Float32               7771.3740234375  2021-05-25T12:58:56.628589000Z  
    Table: keys: [_start, _stop, _field, _measurement, device, event_id, resource_name]
                       _start:time                      _stop:time           _field:string     _measurement:string           device:string                       event_id:string    resource_name:string                                                                                                                                                                                                                                                                                                          _value:float                      _time:time  
    ------------------------------  ------------------------------  ----------------------  ----------------------  ----------------------  ------------------------------------  ----------------------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------  ------------------------------  
    2021-05-25T12:58:45.731852264Z  2021-05-25T12:59:45.731852264Z                   value                readings     Random-Float-Device  53ad763a-f3be-4e30-b28d-4595316e5672                 Float64  93725270000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000  2021-05-25T12:58:56.628204000Z  
    Table: keys: [_start, _stop, _field, _measurement, device, event_id, resource_name]
                       _start:time                      _stop:time           _field:string     _measurement:string           device:string                       event_id:string    resource_name:string                                                                                                                                                                                                                                                                                                           _value:float                      _time:time  
    ------------------------------  ------------------------------  ----------------------  ----------------------  ----------------------  ------------------------------------  ----------------------  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------  ------------------------------  
    2021-05-25T12:58:45.731852264Z  2021-05-25T12:59:45.731852264Z   
    

More configuration details about InfluxDBSyncWrite

InfluxDBSyncWrite
Uses the influxdb-client-go library to synchronously write EdgeX events to the InfluxDB v1.8.x or v2.0.x
Sample configuration:
[Writable.Pipeline.Functions.InfluxDBSyncWrite]
[Writable.Pipeline.Functions.InfluxDBSyncWrite.Parameters]
InfluxDBServerURL = 'http://localhost:8086'
InfluxDBOrganization = 'my-org'
InfluxDBBucket = 'my-bucket'
InfluxDBMeasurement = 'readings'
InfluxDBValueType = 'integer'
InfluxDBPrecision = 'us'
authmode = 'token'
secretpath='influxdb'
skipverify = 'true'
persistOnError = 'false'
Configuration Description
InfluxDBServerURL Mandatory configuration to specify the InfluxDB Server address.
InfluxDBOrganization The configuration setting to specify the InfluxDB Organization. For InfluxDB 2.0, an organization is a workspace for a group of users. An organization includes dashboards, tasks, buckets, and members. For InfluxDB 1.8.x, organization is not used, and you can simply ignore this setting. For more details about InfluxDB Organization, please refer to "Manage organizations" section in InfluxDB v2.0 documentation. Default value: my-org
InfluxDBBucket The configuration setting to specify InfluxDB bucket. A bucket is a named location where time series data is stored in InfluxDB 2.0. In InfluxDB 1.8+, each combination of a database and a retention policy (database/retention-policy) represents a bucket. Skip retention policy if the default retention policy should be used. For example, edgex/autogen or edgex. To understand more details about InfluxDB bucket, please refer to "bucket" in "Glossary" section in InfluxDB v2.0 documentation. Default value: my-bucket
InfluxDBMeasurement The configuration setting to specify InfluxDB measurement where EdgeX events shall write to. Measurement is the part of the InfluxDB data structure that describes the data stored in the associated fields. To understand more details about measurement, please refer to "measurement" in "Glossary" section in InfluxDB v2.0 documentation. Default value: readings
InfluxDBValueType The configuration setting to specify what value type of events shall be written into InfluxDB. As each InfluxDB measurement is restricted to accept one data type, you shall specify the corresponding data type that InfluxDBMeasurement will accept. InfluxDBValueType only accepts float, integer, string, or boolean. Default value: integer
InfluxDBPrecision The configuration setting to specify how much timestamp precision is retained in the InfluxDB points. InfluxDB will truncate all incoming timestamps to the given precision. Per InfluxDB documentation, InfluxDB suggests to use coarsest possible timestamp precision when writing data to InfluxDB to maximize performance, so you can adjust the precision per your scenarios. InfluxDBPrecision only accepts ns(nanoseconds), us(microseconds), ms(millieseconds), or s(seconds). For more details about InfluxDB precision, please refer to InfluxDB documentation. Default value: us
authmode Specify what authentication mode to write points to InfluxDB. As of release of Edge Xpert 1.7.1, authmode only accepts token for token authentication or none for no authentication. Default Value: token
secretpath The name of the path in secret provider to retrieve secrets.
skipverify Specify whether EdgeX app service should verify the server's certificate chain and host name. If skipverify is true, TLS accepts any certificate presented by the server and any host name in that certificate. In this mode, TLS is susceptible to man-in-the-middle attacks. This should be used only for testing.
PersistOnError Specify whether the events should be stored for later retry if encountering any error during message publish. If persistOnError is true and Store and Forward is enabled, the data will be stored for later retry. See Store and Forward for more details.
StoreEventTags Specify whether the events tags should be stored as InfluxDB tags. By default, InfluxDB tags only contain deviceName and resourceName. If StoreEventTags is true, the event tags added by Tags built-in functions will be stored to InfluxDB.
Back to top