Skip to content

Database

Edge Xpert's reference database (for sensor data, Core Metadata and all things that need to be stored in a database) is Redis.

Redis is an open source (BSD licensed), in-memory data structure store, used as a database in Edge Xpert. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis is durable and uses persistence only for state recovering; the only data Redis operates on is in-memory.

Memory Utilization

Redis uses a number of techniques to optimize memory utilization. Antirez and Redis have written a number of articles on the underlying details (see the list below) and those strategies has continued to evolve. When thinking about your system architecture, consider how long data will be living at the edge and consuming memory (physical or physical + virtual).

On-disk Persistence

Redis supports a number of different levels of on-disk persistence. By default, snapshots of data are persisted every 60 seconds or after 1000 key changes. In addition to increasing the frequency of snapshots, there is support for only recording additional files for each database write. See Redis Persistence for a detailed discussion of how to balance these options.

Redis supports setting memory usage limits and policies for how to handle situations where memory cannot be allocated for write operations. For configuration options, see the MEMORY MANAGEMENT section of Redis Configuration File Examples. Since Edge Xpert and Redis do not currently communicate about data eviction, you will need to use the Edge Xpert scheduler to control memory usage instead of Redis's eviction policy.

Note

Edge Xpert provides a configuration called EventMaxLimit under the Writable section to limit the Events stored in the database to prevent memory exhuastion. The default value is 1000, which can be easily by using Writable_EventMaxLimit environment variable in the Core Data, see Core Data Configuration Properties for more details.

Note

EventMaxLimit has been deprecated since v2.3.6 and it's replaced by the Data Retention mechanism, see Core Data Retention for more details.