Export to Azure IoT Hub

Overview

Microsoft Azure IoT Hub is a cloud service that acts as a central message hub for communication between an IoT application and its attached devices. To send data to Azure IoT Hub, Edge Xpert provides a proprietary XpertMQTTExport function, which is similar to MQTTExport of EdgeX Foundry but with additional feature enhancements. This article will take you through steps to create an Edge Xpert application service to export data as generated by virtual device service to Azure IoT Hub.

Prerequisites

1. Create an IoT Hub using the Azure portal

2. Prepare an X.509 Certificate

   • For testing purpose only, you can generate X509 self-signed certificate rsa_cert.pem and key rsa_private.pem with OpenSSL by running the following command:

       openssl req -x509 -sha1 -nodes -newkey rsa:2048 -keyout rsa_private.pem -days 365 -out rsa_cert.pem
     

3. Register the X.509 Certificate to the IoT Hub

   • Sign in to Azure Portal, switch to your Azure IOT Hub and then add a certificate by uploading your certificate .pem or .cer file as generated on step 2.

   

   • Once your certificate .pem or .cer file has been successfully uploaded, copy its thumbprint, which is required when adding the IoT device.

   

4. Add a new device with X.509 authentication to the IoT Hub

   • In your IoT hub navigation menu, select Devices, then click on Add Device to add a device in your IoT hub.

   

   • In Create a device page, provide a name for your new device, choose X.509 Self-Signed as authentication type and provide the Thumbprint as obtained from previous step for both Primary Thumbprint and Secondary Thumbprint.

   

5. Understand Edge Xpert Application Service

6. Install Edge Xpert
7. Launch Edge Xpert core services

Sample Configuration

The following code defines the pipeline function XpertMQTTExport:

[Writable.Pipeline.Functions.XpertMQTTExport]
  [Writable.Pipeline.Functions.XpertMQTTExport.Parameters]
  ExportMode = "AzureIoTHub"
  BrokerAddress = "tls://{AzureIoTHubName}.azure-devices.net:8883"
  # If 8883 port is blocked in your network, please consider to use MQTT over Websocket to connect with Azure IoT
  # Hub via 443 port (comment above BrokerAddress and uncomment following BrokerAddress)
  #BrokerAddress = "wss://{AzureIoTHubName}.azure-devices.net:443/$iothub/websocket"
  Topic = "devices/{DeviceID}/messages/events/"
  ClientId = "{DeviceID}"
  SecretPath = "azure"
  AuthMode = "clientcert"
  Qos = "0"
  AutoReconnect = "false"
  Retain = "false"
  SkipVerify = "false"
  PersistOnError = "false"

The following table describes the configuration parameters used to define the pipeline function XpertMQTTExport to send data to Azure IoT Hub:

tls://{AzureIoTHubName}.azure-devices.net:8883

wss://{AzureIoTHubName}.azure-devices.net:443/$iothub/websocket

devices/{DeviceID}/messages/events/

Configure an Application service to export events to Azure IoT Hub

1. By end of step 4 in Prerequisites, users shall expect to obtain IoT Hub name, Device ID, and X.509 Certificate/Private Key. Keep a note on these values, as users will need these values to authenticate with Azure IoT Hub.

2. Edge Xpert offers a proprietary built-in function XpertMQTTExport to send the events to Azure IoT Hub via MQTT, and the sample configuration is provided with Edge Xpert CLI utility. Once users complete the installation of Edge Xpert CLI utility, the sample configuration is available at /usr/share/edgexpert/examples/app-configurable/azureiothub-mqtt-export/. Copy and save the sample configuration file as a separate file, e.g. azureiothub-mqtt-export.toml, and then revise this configuration per values of IoT Hub Name, Device ID, and X.509 Certificate/Private Key as obtained earlier. For example, if users have IoT Hub Name: EdgeX, Device ID: TestDevice01, user shall revise XpertMQTTExport with following parameters:

   [Writable.Pipeline.Functions.XpertMQTTExport]
     [Writable.Pipeline.Functions.XpertMQTTExport.Parameters]
     ExportMode = "AzureIoTHub"
     BrokerAddress = "tls://EdgeX.azure-devices.net:8883"
     # If 8883 port is blocked in your network, please consider to use MQTT over Websocket to connect with Azure IoT
     # Hub via 443 port (comment above BrokerAddress and uncomment following BrokerAddress)
     #BrokerAddress = "wss://EdgeX.azure-devices.net:443/$iothub/websocket"
     Topic = "devices/TestDevice01/messages/events/"
     ClientId = "TestDevice01"
     SecretPath = "azure"
     AuthMode = "clientcert"
     Qos = "0"
     AutoReconnect = "false"
     Retain = "false"
     SkipVerify = "false"
     PersistOnError = "false"
   

   Note

   Users will find two sample configuration files under /usr/share/edgexpert/examples/app-configurable/azureiothub-mqtt-export/. If the application service will be running in insecure mode, use configuration-insecure.toml as the sample. If the application service will be running in secure mode, use configuration-secretstore.toml as the sample.

3. If the application service will be launched under insecure mode, secrets will be retrieved from Writable.InsecureSecrets configuration. Users will need to update Writable.InsecureSecrets.azure configuration with correct values of username, clientcert, and clientkey:

    [Writable.InsecureSecrets.azure]
      path = "azure"
      [Writable.InsecureSecrets.azure.Secrets]
      # When using XpertMQTTExport and ExportMode="AzureIoTHub", username, clientcert and clientkey are mandatory secrets.
      username = "EdgeX.azure-devices.net/TestDevice01/?api-version=2021-04-12"
      clientcert = '''-----BEGIN CERTIFICATE-----
      MIIB2zCCAYKgAwIBAgIQSgMseWua/pRIZuBAyzIuCzAKBggqhkjOPQQDAjAoMSYw
      -----END CERTIFICATE-----'''
      clientkey = '''-----BEGIN PRIVATE KEY-----
      MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDBW5VXLmNugcKA
      -----END PRIVATE KEY-----'''
   

   Note

   At the time of publication, Azure IoT Hub requires the value of username to be in the format of {AzureIoTHubName}.azure-devices.net/{DeviceID}/?api-version=2021-04-12. The value of clientcert must be the content of certificate when you set up the X.509 security in Azure IoT Hub. The value of clientkey must be the private key when you set up the X.509 security in Azure IoT Hub.

4. Once configuration is ready, users can launch the application service in secure mode using command below:

   $ edgexpert up --secret app-service --path=<path of the confgiuration file>
   

   If users prefer to launch the application service in insecure mode, use command below:

   $ edgexpert up app-service --path=<path of the confgiuration file>
   

5. If the application service is launched under secure mode on previous step, secrets will be retrieved from Vault secret store. Users need to add username, clientkey, and clientcert into Vault secret store. Please refer to EdgeX Foundry Storing Secrets for details about adding secrets into Vault secret store.

   Note

   At the time of publication, Azure IoT Hub requires the value of username to be in the format of {AzureIoTHubName}.azure-devices.net/{DeviceID}/?api-version=2021-04-12. The value of clientcert must be the content of certificate when you set up the X.509 security in Azure IoT Hub. The value of clientkey must be the private key when you set up the X.509 security in Azure IoT Hub.
   
   Since Edge Xpert V2.1, IOTech recommends users to add secrets through the POST secrets REST API of application service. Follow steps below to add secrets under secure mode.
   
   1. Inspect the IPAddress of the application service as launched in previous step:
   

   $ docker inspect --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' app-configuration-secretstore
   192.168.80.10
   

   2. Users may find it's difficult to properly add the content of X.509 PEM files in valid format in a REST request. Use following awk commands to convert the content of X.509 PEM files as a string:

   $ awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' TestDevice01_rsa_cert.pem
   -----BEGIN CERTIFICATE-----\nMIIDgzCCAmugAwIBAgIUXvDwsS+pW/W+hMRp675EcXsYwIgwDQYJKoZIhvcNAQEF\n-----END CERTIFICATE-----\n
   $ awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' TestDevice01_rsa_key.pem
   -----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDBW5VXLmNugcKA\n-----END PRIVATE KEY-----\n
   

   Make a note on the console output of both awk commands result.
   3. Insert the copied string into the payload of REST call to add secrets. Note that all necessary secrets must be added in a single REST call, otherwise the secrets content will be overwritten by REST calls afterwards. For example, following curl command adds three secrets--username,clientcert, and clientkey--under the secret path azure in the Vault secret store:

   curl -k -X POST -d '{ "apiVersion": "v2", "path": "azure", "secretData": [ {"key":"username","value":"EdgeX.azure-devices.net/TestDevice01/?api-version=2021-04-12" },{"key":"clientcert","value":"-----BEGIN CERTIFICATE-----\nMIIDgzCCAmugAwIBAgIUXvDwsS+pW/W+hMRp675EcXsYwIgwDQYJKoZIhvcNAQEF\n-----END CERTIFICATE-----\n"},{"key":"clientkey","value":"-----BEGIN PRIVATE KEY-----\nMIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDBW5VXLmNugcKA\n-----END PRIVATE KEY-----\n"}]}' http://192.168.80.10:59700/api/v2/secret
   

6. Now, an application service is up and ready to export events to Azure IoT Hub. To simulate the device sending readings from southbound, start up the virtual device service in secure mode using command below:

   $ edgexpert up --secret device-virtual
   

   If users prefer to launch the virtual device service in insecure mode, use command below:

   $ edgexpert up device-virtual
   

7. Azure IoT Hub allows users to see the number of sent messages. To do this in the Azure Portal, select Metrics from the Monitor menu options. Select Telemetry messages sent in the Metrics drop-down menu to see the number of sent messages:

8. If users would like to examine messages as received by Azure IoT Hub, please install Visual Studio Code and the Azure IoT Hub extension. After installation, right-click on the target device, and then choose Start Monitoring Built-In Event Endpoint. Ideally, users should expect to see similar messages as shown below in the Output window:

   [IoTHubMonitor] [5:01:50 PM] Message received from [TestDevice01]:
   {
     "apiVersion": "v2",
     "id": "3a2fe106-1c21-4b55-a729-89b012433f9d",
     "deviceName": "Random-Boolean-Device",
     "profileName": "Random-Boolean-Device",
     "sourceName": "Bool",
     "origin": 1646643709904223500,
     "readings": [
       {
         "id": "f0609e1d-e182-49f5-b07b-99ebb7d2b9af",
         "origin": 1646643709904223500,
         "deviceName": "Random-Boolean-Device",
         "resourceName": "Bool",
         "profileName": "Random-Boolean-Device",
         "valueType": "Bool",
         "value": "true"
       }
     ]
   }