Database

Edge Xpert's reference database (for sensor data, Core Metadata and all things that need to be stored in a database) is Redis.

Redis is an open source (BSD licensed), in-memory data structure store, used as a database in Edge Xpert. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. Redis is durable and uses persistence only for state recovering; the only data Redis operates on is in-memory.

Memory Utilization

Redis uses a number of techniques to optimize memory utilization. Antirez and Redis have written a number of articles on the underlying details (see the list below) and those strategies has continued to evolve. When thinking about your system architecture, consider how long data will be living at the edge and consuming memory (physical or physical + virtual).

• Redis streams as a pure data structure
• About Redis Sets memory efficiency
• Redis RAM Ramifications
• Redis Memory Optimization

On-disk Persistence

Redis supports a number of different levels of on-disk persistence. By default, snapshots of data are persisted every 60 seconds or after 1000 key changes. In addition to increasing the frequency of snapshots, there is support for only recording additional files for each database write. See Redis Persistence for a detailed discussion of how to balance these options.

Redis supports setting memory usage limits and policies for how to handle situations where memory cannot be allocated for write operations. For configuration options, see the MEMORY MANAGEMENT section of Redis Configuration File Examples. Since Edge Xpert and Redis do not currently communicate about data eviction, you will need to use the Edge Xpert scheduler to control memory usage instead of Redis's eviction policy.