Core Metadata
Introduction
The Core Metadata contains the information about the devices and sensors and how to communicate with them. This information is used by the other services such as Core Data, Core Command, etc.
Specifically, Core Metadata has the following capabilities:
- Manages information about the devices that are connected to and operated by Edge Xpert
- Accesses to the type and organization of data reported by the devices
- Accesses to how to control these devices
Although Core Metadata has this information, it does not perform the following activities:
- It is not responsible for the actual data collection from the devices, which is performed by Device Services and Core Data
- It is not responsible for issuing commands to the devices, which is handled by Core Command and the Device Services
Data Models
To understand Core Metadata, it is important to know the Edge Xpert data objects that manage it. Metadata stores its information in a local persistence database.by default, Redis is used, but a database abstraction layer allows other databases to be used as well.
Device Profile
Device profiles define general characteristics of devices, the data they provide, and the way they can be controlled. Think of a device profile as a template for a device type or classification. For example, a device profile for BACnet thermostats provides general characteristics for the types of data a BACnet thermostat sends, such as current temperature and humidity. It also defines what types of commands or actions Edge Xpert can send to the BACnet thermostat. Examples include actions that set the cooling or heating point. Device profiles are usually defined in YAML or JSON files and uploaded to Edge Xpert. See below for more details.
Device Profile Details
Metadata device profile object model
A device profile has a number of high level properties to give the profile context and identification. Its name field is required and must be unique in an Edge Xpert deployment. Other fields are optional - they are not used by device services but may be populated for informational purposes:
- Description
- Manufacturer
- Model
- Labels
Here is a general information section for a sample KMC 9001 BACnet thermostat device profile provided with the BACnet Device Service (you can find the profile in Github). Only the name is required in this section of the device profile. The name of the device profile must be unique in any Edge Xpert deployment. The manufacturer, model and labels are all optional bits of information that allow better queries of the device profiles in the system.
name: "BAC-9001"
manufacturer: "KMC"
model: "BAC-9001"
labels:
- "B-AAC"
description: "KMC BAC-9001 BACnet thermostat"
Labels provided a way to tag, organize or categorize the various profiles. They serve no real purpose inside of Edge Xpert.
A device resource (in the deviceResources section of the device profile) specifies a sensor value within a device that may be read from or written to either individually or as part of a device command (see below). Think of a device resource as a specific value that can be obtained from the underlying device or a value that can be set to the underlying device. In a thermostat, a device resource may be a temperature or humidity (values sensed from the devices) or cooling point or heating point (values that can be set/actuated to allow the thermostat to determine when associated heat/cooling systems are turned on or off). A device resource has a name for identification and a description for informational purposes.
The properties section of a device resource has also been greatly simplified. See details below.
Back to the BACnet example, here are two device resources. One will be used to get the temperature (read) the current temperature and the other to set (write or actuate) the active cooling set point. The device resource name must be provided and it must also be unique in any Edge Xpert deployment.
name: Temperature
description: "Get the current temperature"
isHidden: false
name: ActiveCoolingSetpoint
description: "The active cooling set point"
isHidden: false
Note
isHidden
is false by default when it is not specified. isHidden
indicates whether to expose the device resource to the Core Command.
The Device Service allows access to the device resources via REST endpoint. Values specified in the device resources section of the device profile can be accessed through the following URL patterns:
http://<device-service>:<port>/api/v2/device/name/<DeviceName>/<DeviceResourceName>
The attributes associated to a device resource are the specific parameters required by the Device Service to access the particular value. In other words, attributes are “inward facing” and are used by the Device Service to determine how to speak to the device to either read or write (get or set) some of its values. Attributes are detailed protocol and/or device specific information that informs the Device Service how to communication with the device to get (or set) values of interest.
Returning to the BACnet device profile example, below are the complete device resource sections for Temperature and ActiveCoolingSetPoint – inclusive of the attributes – for the example device.
-
name: Temperature
description: "Get the current temperature"
isHidden: false
attributes:
{ type: "analogValue", instance: "1", property: "presentValue", index: "none" }
-
name: ActiveCoolingSetpoint
description: "The active cooling set point"
isHidden: false
attributes:
{ type: "analogValue", instance: "3", property: "presentValue", index: "none" }
The properties of a device resource describe the value obtained or set on the device. The properties can optionally inform the Device Service of some simple processing to be performed on the value. Again, using the BACnet profile as an example, here are the properties associated to the thermostat's temperature device resource.
name: Temperature
description: "Get the current temperature"
attributes:
{ type: "analogValue", instance: "1", property: "presentValue", index: "none" }
properties:
valueType: "Float32"
readWrite: "R"
units: "Degrees Fahrenheit"
The valueType
property of properties gives more details about the value collected or set. In this case giving the details of the temperature value to be set. The value provides details such as the type of the data collected or set, whether the value can be read, written or both.
The following fields are available in the value property:
- valueType - Required. The data type of the value. Supported types are Bool, Int8 - Int64, Uint8 - Uint64, Float32, Float64, String, Binary, Object and arrays of the primitive types (ints, floats, bool). Arrays are specified as eg. Float32Array, BoolArray etc.
- readWrite - R, RW, or W indicating whether the value is readable or writable.
- units - gives more detail about the unit of measure associated with the value. In this case, the temperature unit of measure is in degrees Fahrenheit.
- min - minimum allowed value
- max - maximum allowed value
- defaultValue - a value used for PUT requests which do not specify one.
- base - a value to be raised to the power of the raw reading before it is returned.
- scale - a factor by which to multiply a reading before it is returned.
- offset - a value to be added to a reading before it is returned.
- mask - a binary mask which will be applied to an integer reading.
- shift - a number of bits by which an integer reading will be shifted right.
The processing defined by base, scale, offset, mask and shift is applied in that order. This is done within the SDK. A reverse transformation is applied by the SDK to incoming data on set operations (NB mask transforms on set are NYI)
Device commands (in the deviceCommands section of the device profile) define access to reads and writes for multiple simultaneous device resources. Device commands are optional. Each named device command should contain a number of get and/or set resource operations, describing the read or write respectively.
Device commands may be useful when readings are logically related, for example with a 3-axis accelerometer it is helpful to read all axes (X, Y and Z) together.
A device command consists of the following properties:
- name - the name of the command
- readWrite - R, RW, or W indicating whether the operation is readable or writable.
- isHidden - indicates whether to expose the device command to the Core Command service (optional and false by default)
- resourceOperations - the list of included device resource operations included in the command.
Each resourceOperation will specify:
- the deviceResource - the name of the device resource
- defaultValue - optional, a value to return when the operation does not provide one
- parameter - optional, a value that will be used if a PUT request does not specify one.
- mappings - optional, allows readings of String type to be re-mapped.
The device commands can also be accessed through a device service’s REST API in a similar manner as described for device resources.
http://<device-service>:<port>/api/v2/device/name/<DeviceName>/<DeviceCommandName>
If a device command and device resource have the same name, it will be the device command which is available.
Device
Data about actual devices is another type of information that the Core Metadata stores and manages. Each device managed by Edge Xpert Foundry registers with Core Metadata (via its owning Device Service). Each device must have a unique name associated to it.
Metadata stores information about a device (such as its address) against the name in its database. Each device is also associated to a device profile. This association enables Core Metadata to apply knowledge provided by the device profile to each device. For example, a thermostat profile would say that it reports temperature values in Celsius. Associating a particular thermostat (the thermostat in the lobby for example) to the thermostat profile allows Core Metadata to know that the lobby thermostat reports temperature value in Celsius.
Device Service
Metadata also stores and manages information about the Device Services. Device Services serve as Edge Xpert's interfaces to the actual devices and sensors.
Device Services are other microservices that communicate with devices via the protocol of that device. For example, a Modbus Device Service facilitates communications among all types of Modbus devices. Examples of Modbus devices include motor controllers, proximity sensors, thermostats, and power meters. Device Services simplify communications with the device for the rest of Edge Xpert.
When a Device Service starts, it registers itself with Core Metadata. When Edge Xpert provisions a new devices the device gets associated to its owning Device Service. That association is also stored in Core Metadata.
Metadata Device, Device Service and Device Profile Model
Metadata's Device Profile, Device and Device Service object model and the association between them
Device Provisioning
A device needs to be onboarded into a Device Service so that the Device Service knows the location of the device and how to communicate with it. This process is called device provisioning.
There are two parts to device provisioning:
- A device profile is added to Edge Xpert. The device profile contains information about the resources and commands that are available for that device
- A device connection is added to Edge Xpert. This specifies the physical addressing information of the device such as its IP-address or port number
Once these two requirements are met, Edge Xpert will be able to communicate with the device.
Device provisioning can be done in two ways:
- Static Provisioning
- Dynamic Provisioning
Static Provisioning
All of the Device Services support static provisioning of devices. For static provisioning the device profile and the connection information for the device is known before onboarding the device. First, the device profile is added to Edge Xpert and then the device can be added. These steps can be performed using the Edge Xpert REST APIs or the Edge Xpert Manager UI.
Dynamic Provisioning
Some Device Services support dynamic provisioning using automatic discovery. The Device Service is provided with the necessary information about where to look and general parameters for a device (or devices). While the Device Service is running, it will continually scan (on a configurable schedule) for new devices as specified by the discovery parameters. When it finds a device that matches these parameters it will automatically create and register device profiles for those devices, and automatically onboard the devices to the Edge Xpert platform.
The configuration about how and where to look for discoverable devices is specified by an Edge Xpert Provision Watcher.
Provision Watcher
A provision watcher is specific configuration information provided to a Device Service and then stored by Core Metadata. The provision watcher is given identifiers which are a map of key value pairs that identify the type of protocol and the specific locations and parameters used in the discovery scanning process. For example, this can include MAC addresses and/or IP addresses.
In addition to providing details about what devices to look for during a scan, a provision watcher may also contain “blocking” indentifiers, which define parameters about devices that should not be automatically provisioned if they are found. This allows the scope of a device scan to be narrowed or allows specific devices to be avoided. Examples of how this is used can be found in Automatic Discovery
Metadata's provision watcher object model
Data Dictionary
Property | Description |
---|---|
The Core Metadata base structure for common information needed to make a request to an Edge Xpert target. | |
Type | REST or MQTT |
Host | Target's address string - such as an IP address |
Port | Port for the target address |
Property | Description |
---|---|
Structure extending BaseAddress, used to make a request of Edge Xpert targets via REST. | |
Path | URI path beyond the host and port |
HTTPMethod | Method for connecting (i.e. POST) |
Property | Description |
---|---|
Structure extending BaseAddress, used to make a request of Edge Xpert targets via MQTT. | |
Publisher | Publisher name |
User | User id for authentication |
Password | Password of the user for authentication |
Topic | Topic for message bus |
QoS | Quality of service level for message publishing; value 0, 1, or 2 |
KeepAlive | Maximum time interval in seconds with no comms before closing |
Retained | Flag to have the broker store the last rec'd message for future subscribers |
AutoReconnect | Indication to reconnect on failed connection |
ConnectTimeout | Maximum time interval the client will wait for the connection to the MQTT server to be established |
Property | Description |
---|---|
AutoEvent supports auto-generated events sourced from a Device Service | |
Interval | How often the specific resource needs to be polled. |
OnChange | indicates whether the Device Service will generate an event only |
SourceName | the name of the resource in the device profile which describes the event to generate |
Property | Description |
---|---|
The object that contains information about the state, position, reachability, and methods of interfacing with a Device; represents a registered device participating within the Edge Xpert ecosystem | |
Id | Uniquely identifies the device, a UUID for example |
Description | Provide further description about this device |
Name | Name for identifying a device |
AdminState | Admin state (LOCKED/UNLOCKED) |
OperatingState | Operating state (UP/DOWN) |
Protocols | A map of supported protocols for the given device |
LastConnected | Time (milliseconds) that the device last provided any feedback or responded to any request |
LastReported | Time (milliseconds) that the device reported data to the core microservice |
Labels | Labels applied to the device to help with searching |
Location | Device Service specific location (interface{} is an empty interface so it can be anything) |
ServiceName | Associated Device Service - One per device |
ProfileName | Associated Device Profile - Describes the device |
AutoEvents | A list of auto-generated events coming from the device |
Tags | Define the tags which can be added to the events produced from this device in the App Service |
Property | Description |
---|---|
represents the attributes and operational capabilities of a device. It is a template for which there can be multiple matching devices within a given system. | |
Id | Uniquely identifies the device, a UUID for example |
Description | Provide further description about this device profile |
Name | Name for identifying a device profile |
Manufacturer | Manufacturer of the device |
Model | Model of the device |
Labels | Labels used to search for groups of profiles |
DeviceResources | deviceResource collection |
DeviceCommands | collect of deviceCommand |
Property | Description |
---|---|
The atomic description of a particular protocol level interface for a class of Devices; represents a value on a device that can be read or written | |
Description | Provide further description about this device resource |
Name | Name for identifying a device resource |
Tags | Define the tags which can be added to the events produced from this device resource in the App Service |
Properties | List of associated properties |
Attributes | List of associated attributes |
Property | Description |
---|---|
represents a service that is responsible for proxying connectivity between a set of devices and the Edge Xpert core services; the current state and reachability information for a registered Device Service | |
Id | Uniquely identifies the Device Service, a UUID for example |
Name | Name for identifying a Device Service |
LastConnected | Time (milliseconds) that the device last reported data to the core |
LastReported | Time (milliseconds) that the Device Service reported data to the core microservice |
Labels | Labels applied to the Device Service for search or other identification needs |
BaseAddress | address (MQTT topic, HTTP address, serial bus, etc.) for reaching the service |
AdminState | Device Service Admin State (LOCKED/UNLOCKED) |
Property | Description |
---|---|
The transformation and constraint properties for a device resource. | |
ValueType | Type of the value |
ReadWrite | Read/Write Permissions set for this property |
Minimum | Minimum value that can be get/set from this property |
Maximum | Maximum value that can be get/set from this property |
DefaultValue | Default value set to this property if no argument is passed |
Mask | Mask to be applied prior to get/set of property |
Shift | Shift to be applied after masking, prior to get/set of property |
Scale | Multiplicative factor to be applied after shifting, prior to get/set of property |
Offset | Additive factor to be applied after multiplying, prior to get/set of property |
Base | Base for property to be applied to, leave 0 for no power operation (i.e. base ^ property: 2 ^ 10) |
Assertion | Required value of the property, set for checking error state. Failing an assertion condition will mark the device with an error state |
MediaType | A string value used to indicate the type of binary data if valueType=binary |
Property | Description |
---|---|
The Core Metadata used by a Service for automatically provisioning matching Devices. | |
Id | Uniquely identifies the provision watcher, a UUID for example |
Name | Name for identifying of the provision watcher |
Labels | Labels applied to the provision watcher for search or other identification needs |
Identifiers | Set of key value pairs that identify property (MAC, HTTP,...) and value to watch for (00-05-1B-A1-99-99, 10.0.0.1,...) |
BlockingIdentifiers | Set of key-values pairs that identify devices which will not be added despite matching on Identifiers |
ProfileName | Name of the device profile that should be applied to the devices available at the identifier addresses |
ServiceName | Name of the Device Service that new devices will be associated to |
AdminState | Administrative state for new devices - either unlocked or locked |
AutoEvents | Associated auto events to this watcher |
Configuration Properties
Please refer to the general Common Configuration documentation for configuration properties common to all services. Below are only the additional settings and sections that are not common to all Edge Xpert Services.
Property | Default Value | Description |
---|---|---|
Properties used by the service to access the database | ||
Name | 'Core Metadata' | Document store or database name |
Property | Default Value | Description |
---|---|---|
Configuration to post device changes through the notifiction service | ||
PostDeviceChanges | false | Whether to send out notification when a device has been added, changed, or removed |
Content | 'Meatadata notice: ' | Start of the notification message when sending notification messages on device change |
Sender | 'Core Metadata' | Sender of any notification messages sent on device change |
Description | 'Metadata change notice' | Message description of any notification messages sent on device change |
Label | 'Core Metadata' | Label to put on messages for any notification messages sent on device change |