Skip to content

Database

Edge Xpert's reference implementation database (for sensor data, metadata and all things that need to be persisted in a database) is Redis.

Redis is an open source (BSD licensed), in-memory data structure store, used as a database in Edge Xpert. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis is durable and uses persistence only for recovering state; the only data Redis operates on is in-memory.

Memory Utilization

Redis uses a number of techniques to optimize memory utilization. Antirez and Redis Labs have written a number of articles on the underlying details (see the list below) and those strategies has continued to evolve. When thinking about your system architecture, consider how long data will be living at the edge and consuming memory (physical or physical + virtual).

On-disk Persistence

Redis supports a number of different levels of on-disk persistence. By default, snapshots of the data are persisted every 60 seconds or after 1000 keys have changed. Beyond increasing the frequency of snapshots, append only files that log every database write are also supported. See Redis Persistence for a detailed discussion on how to balance the options.

Redis supports setting a memory usage limit and a policy on what to do if memory cannot be allocated for a write. See the MEMORY MANAGEMENT section of Redis configuration file example for the configuration options. Since Edge Xpert and Redis do not currently communicate on data evictions, you will need to use the Edge Xpert scheduler to control memory usage rather than a Redis eviction policy.

Note

Edge Xpert provides a configuration called EventMaxLimit under the Writable section to limit the Events stored in the database to prevent the memory exhuastion. The default value is 1000, and it can be easily adjusted by using Writable_EventMaxLimit environment variable in the core-data service. Please see Core Data Configuration Properties for more details.

Back to top