Config & Registry
Introduction
The Edge Xpert registry and configuration service provides other Edge Xpert micro services with information about associated services within Edge Xpert (such as location and status) and configuration properties (i.e. - a repository of initialization and operating values). Today, Edge Xpert uses Consul by Hashicorp as its reference implementation configuration and registry providers. However, abstractions are in place so that these functions could be provided by an alternate implementation. In fact, registration and configuration could be provided by different services under the covers. For more, see the Configuration Provider and Registry Provider sections in this page.
Note
After Edge Xpert 2.1 the registry and configuration service (Consul) is disabled by default to save the computing resources. It can be enabled when the edgexpert up command includes consul service, for example, edgexpert up consul device-virtual.
Configuration
Common Configuration
The tables in each of the tabs below document configuration properties that are common to all services in the Edge Xpert platform. Service-specific properties can be found on the respective documentation page for each service.
Configuration Properties
Note
For Edge Xpert 2.0 the Logging and Startup sections have been removed. Startup has been replaced with the EDGEX_STARTUP_DURATION (default is 60 secs) and EDGEX_STARTUP_INTERVAL (default is 1 sec) environment variables.
Note
After Edge Xpert 2.1.3 the default value of EDGEX_STARTUP_DURATION is extended to 300 seconds and EDGEX_STARTUP_INTERVAL becomes 3 seconds to meet the requirements of some low power machines.
| Property | Default Value | Description |
|---|---|---|
entries in the Writable section of the configuration can be changed on the fly while the service is running if the service is running with the -cp/--configProvider flag |
||
| LogLevel | INFO | log entry severity level. Log entries not of the default level or higher are ignored. |
| InsecureSecrets | --- | This section a map of secrets which simulates the SecretStore for accessing secrets when running in non-secure mode. All services have a default entry for Redis DB credentials called redisdb |
Note
For Edge Xpert 2.0 the For Edge Xpert 2.0 the Writable.InsecureSecrets configuration section is new.
| Property | Default Value | Description |
|---|---|---|
| HealthCheckInterval | 10s | The interval in seconds at which the service registry(Consul) will conduct a health check of this service. |
| Host | localhost | Micro service host name |
| Port | --- | Micro service port number (specific for each service) |
| ServerBindAddr | '' (empty string) | The interface on which the service's REST server should listen. By default the server is to listen on the interface to which the Host option resolves (leaving it blank). A value of 0.0.0.0 means listen on all available interfaces. App & Device service do not implement this setting |
| StartupMsg | --- | Message logged when service completes bootstrap start-up |
| MaxResultCount | 1024* | Read data limit per invocation. *Default value is for core/support services. Application and Device services do not implement this setting. |
| MaxRequestSize | 0 | Defines the maximum size of http request body in bytes. 0 represents default to system max. Not all services actual implement this setting. Those that do not have a comment stating this fact. |
| RequestTimeout | 5s | Specifies a timeout duration for handling requests |
Note
For Edge Xpert 2.0 Protocol and BootTimeout have been removed. CheckInterval and Timeout have been renamed to HealthCheckInterval and RequestTimeout respectively. MaxRequestSize was added for all services.
| Property | Default Value | Description |
|---|---|---|
| configuration that govern database connectivity and the type of database to use. While not all services require DB connectivity, most do and so this has been included in the common configuration docs. | ||
| Host | localhost | DB host name |
| Port | 6379 | DB port number |
| Name | ---- | Database or document store name (Specific to the service) |
| Timeout | 5000 | DB connection timeout |
| Type | redisdb | DB type. Redis is the only supported DB |
| Property | Default Value | Description |
|---|---|---|
| this configuration only takes effect when connecting to the registry for configuration info | ||
| Host | localhost | Registry host name |
| Port | 8500 | Registry port number |
| Type | consul | Registry implementation type |
| Property | Default Value | Description |
|---|---|---|
| Each service has it own collect of Clients that it uses | ||
| Protocol | http | The protocol to use when building a URI to local the service endpoint |
| Host | localhost | The host name or IP address where the service is hosted |
| Port | 598xx | The port exposed by the target service |
Note
For Edge Xpert 2.0 the map keys have changed to be the service's service-key, i.e. Metadata changed to core-metadata
| Property | Default Value | Description |
|---|---|---|
these config values are used when security is enabled and SecretStore service access is required for obtaining secrets, such as database credentials |
||
| Type | vault | The type of the SecretStore service to use. Currenly only vault is supported. |
| Host | localhost | The host name or IP address associated with the SecretStore service |
| Port | 8200 | The configured port on which the SecretStore service is listening |
| Path | <service-key>/ |
The service-specific path where the secrets are kept. This path will differ according to the given service. |
| Protocol | http | The protocol to be used when communicating with the SecretStore service |
| RootCaCertPath | blank | Default is to not use HTTPS |
| ServerName | blank | Not needed for HTTP |
| TokenFile | /tmp/edgex/secrets/<service-key>/secrets-token.json |
Fully-qualified path to the location of the service's SecretStore access token. This path will differ according to the given service. |
| SecretsFile | blank | Fully-qualified path to the location of the service's JSON secrets file contains secrets to seed at start-up. |
| DisableScrubSecretsFile | false | Controls if the secrets file is scrubbed (secret data remove) and rewritten after importing the secrets. |
| Authentication AuthType | X-Vault-Token | A header used to indicate how the given service will authenticate with the SecretStore service |
Note
For Edge Xpert 2.0 the Protocol default has changed to HTTP which no longer requires RootCaCertPath and ServerName to be set. Path has been reduce to the sub-path for the service since the based path is fixed. TokenFile default value has changed and requires the service-key be used in the path.
| Property | Default Value | Description |
|---|---|---|
| The settings of controling CORS http headers | ||
| EnableCORS | false | Enable or disable CORS support. |
| CORSAllowCredentials | false | The value of Access-Control-Allow-Credentials http header. It appears only if the value is true. |
| CORSAllowedOrigin | "https://localhost" | The value of Access-Control-Allow-Origin http header. |
| CORSAllowedMethods | "GET, POST, PUT, PATCH, DELETE" | The value of Access-Control-Allow-Methods http header. |
| CORSAllowedHeaders | "Authorization, Accept, Accept-Language, Content-Language, Content-Type, X-Correlation-ID" | The value of Access-Control-Allow-Headers http header. |
| CORSExposeHeaders | "Cache-Control, Content-Language, Content-Length, Content-Type, Expires, Last-Modified, Pragma, X-Correlation-ID" | The value of Access-Control-Expose-Headers http header. |
| CORSMaxAge | 3600 | The value of Access-Control-Max-Age http header. |
Note
New for Edge Xpert 2.1 is the ability to enable CORS access to Edge Xpert microservices through configuration.
To understand more details about these HTTP headers, please refer to MDN Web Docs, and refer to CORS enabling to learn more.
Writable vs Readable Settings
Within a given service's configuration, there are keys whose values can be edited and change the behavior of the service while it is running versus those that are effectively read-only. These writable settings are grouped under a given service key. For example, the top-level groupings for edgex-core-data are:
- /edgex/core/2.0/edgex-core-data/Writable
- /edgex/core/2.0/edgex-core-data/Service
- /edgex/core/2.0/edgex-core-data/Clients
- /edgex/core/2.0/edgex-core-data/Databases
- /edgex/core/2.0/edgex-core-data/MessageQueue
- /edgex/core/2.0/edgex-core-data/Registry
- /edgex/core/2.0/edgex-core-data/SecretStore
Any configuration settings found in a service's Writable section may be changed and affect a service's behavior without a restart. Any
modifications to the other settings (read-only configuration) would require a restart.
Local Configuration
Because Edge Xpert may be deployed and run in several different ways, it is important to understand how configuration is loaded and from where it is sourced. Inside each service container there is a res directory (short for "resource"). There you will find the configuration files in TOML format that defines each service's configuration. A service may support several different configuration profiles, for example, an App Service Configurable has multiple configuration profiles. In this case, the configuration file located directly in the res directory should be considered the default configuration profile. Sub-directories will contain configurations appropriate to the respective profile.
As of the v1.8 release, Edge Xpert recommends using environment variable overrides instead of creating profiles to override some subset of config values. App Service Configurable is an exception to this as this is how it defined unique instances using the same executable.
If you choose to use profiles as described above, the config profile can be indicated using one of the following command line flags:
--profile / -p
Taking the Core Data and App Service Configurable services as an examples:
./core-datastarts the service using the default profile found locally./app-service-configurable --profile=rules-enginestarts the service using therules-engineprofile found locally
Note
Again, utilizing environment variables for configuration overrides is the recommended path. Config profiles, for the most part, are not used.
Seeding Configuration
When utilizing the centralized configuration management for the Edge Xpert micro services, it is necessary to seed the required configuration before starting the services. Each service has the built-in capability to perform this seeding operation. A service will use its local configuration file to initialize the structure and relevant values, and then overlay any environment variable override values as specified. The end result will be seeded into the configuration provider if such is being used.
In order for a service to seed/load the configuration to/from the configuration provider, use one of the following flags:
--configProvider / -cp
Again, taking the core-data service as an example:
./core-data -cp=consul.http://localhost:8500 will start the service using configuration values found in the provider or seed them if they do not exist.
Note
Environment overrides are also applied after the configuration is loaded from the configuration provider.
Configuration Structure
Configuration information is organized into a hierarchical structure allowing for a logical grouping of services, as well as versioning, beneath an "edgex" namespace at root level of the configuration tree. The root namespace separates Edge Xpert-related configuration information from other applications that may be using the same configuration provider. Below the root, sub-nodes facilitate grouping of device services, core/support/security services, app services, etc. As an example, the top-level nodes shown when one views the configuration registry might be as follows:
- edgex (root namespace)
- core (core/support/security services)
- devices (device services)
- appservices (application services)
Versioning
Incorporating versioning into the configuration hierarchy looks like this.
- edgex (root namespace)
- core (core/support/security services)
- 2.0
- core-command
- core-data
- core-metadata
- support-notifications
- support-scheduler
- sys-mgmt-agent
- 3.0
- 2.0
- devices (device services)
- 2.0
- device-mqtt
- device-virtual
- device-modbus
- 3.0
- 2.0
- appservices (application services)
- 2.0
- app-rules-engine
- 3.0
- 2.0
- core (core/support/security services)
Note
For Edge Xpert 2.0 the version number in the path is now 2.0 and the service keys are now used for the service names.
The versions shown correspond to major versions of the given services. For all minor/patch versions associated with a major version, the respective service keys live under the major version in configuration (such as 2.0). Changes to the configuration structure that may be required during the associated minor version development cycles can only be additive. That is, key names will not be removed or changed once set in a major version. Furthermore, sections of the configuration tree cannot be moved from one place to another. In this way, backward compatibility for the lifetime of the major version is maintained.
An advantage of grouping all minor/patch versions under a major version involves end-user configuration changes that need to be persisted during an upgrade. A service on startup will not overwrite existing configuration when it runs unless explicitly told to do so via the --overwrite / -o command line flag. Therefore if a user leaves their configuration provider running during an Edge Xpert upgrade any customization will be left in place. Environment variable overrides such as those supplied in the docker-compose for a given release will always override existing content in the configuration provider.
Configuration Provider
You can supply and manage configuration in a centralized manner by utilizing the -cp/--configProvider flag when starting a service. If the flag is provided and points to an application such as HashiCorp's Consul, the service will bootstrap its configuration into the provider, if it doesn't exist. If configuration does already exist, it will load the content from the given location applying any environment variables overrides of which the service is aware.
Registry Provider
The registry refers to any platform you may use for service discovery. For the Edge Xpert reference implementation, the default provider for this responsibility is Consul.
Introduction to Registry
The objective of the registry is to enable micro services to find and to communicate with each other. When each micro service starts up, it registers itself with the registry, and the registry continues checking its availability periodically via a specified health check endpoint. When one micro service needs to connect to another one, it connects to the registry to retrieve the available host name and port number of the target micro service and then invokes the target micro service. The following figure shows the basic flow.
Consul is the default registry implementation and provides native features for service registration, service discovery, and health checking. Please refer to the Consul official web site for more information:
Physically, the "registry" and "configuration" management services are combined and running on the same Consul server node.
Web User Interface
A web user interface is also provided by Consul. Users can view the available service list and their health status through the web user interface. The web user interface is available at the /ui path on the same port as the HTTP API. By default this is http://localhost:8500/ui. For more detail, please see: