- Real-time apps (Uber, DoorDash) require extremely low-latency access to frequently changing data: location updates, ride/delivery requests, driver statuses, session state, and pricing calculations.
- They must process thousands of updates per second, remain available 24/7, preserve critical transactional state across failures, scale with demand, and keep state consistent across clients.
- Speed: ultra-low read/write latency to handle frequent updates (e.g., driver locations).
- Reliability & Availability: continuous operation across infrastructure failures.
- Durability: no loss of transactional data (ride requests, matches) during failures.
- Scalability: capacity to absorb traffic spikes without performance degradation.
- Consistency: clients observe a single, up‑to‑date view of state.
- In-memory performance: stores dataset primarily in memory for microsecond to low‑millisecond reads and writes—ideal for real‑time updates.
- High availability: Multi‑AZ replication with automatic failover keeps clusters online when an Availability Zone fails.
- Durability: writes are append‑only to a distributed transaction log that is durably stored and replicated across AZs for recovery and minimal data loss.
- Scalability: horizontal scaling via shards and replicas to meet growing throughput and capacity needs.
- Strong consistency during failover: designed replication and transaction log mechanics reduce the risk of stale reads after failover.
- Redis compatibility: supports the Redis data model and protocol, so most existing Redis clients and tools work with minimal changes.
- AWS handles provisioning, patching, and routine operations—reducing administrative overhead.
- MemoryDB can replace separate cache + persistent store architectures when you need both in‑memory performance and durable storage semantics.
- Primary / Replica model: each shard hosts a primary (handles reads/writes) and one or more replicas (read scaling and failover candidates).
- Distributed transaction log: the primary appends updates to a transaction log that is durably stored and replicated across AZs; replicas consume the log to replicate updates reliably.
- Sharding: datasets are partitioned across multiple shards so the cluster can scale horizontally and distribute load.
- Transaction logs + snapshots: MemoryDB combines a distributed transaction log with periodic or on-demand snapshots. Together they enable point‑in‑time recovery and seeding of new clusters.
- Snapshots are stored durabley (e.g., in Amazon S3) and can be used to restore a cluster to a previous state or to create a copy for testing.
When to choose MemoryDB
- Choose MemoryDB when you need:
- Very low latency (in‑memory) plus durable storage semantics.
- Multi‑AZ high availability and strong guarantees during failover.
- Redis compatibility so you can reuse existing Redis client code.
- Example: a ride‑sharing backend that cannot afford to lose transactional state during outages.
- Choose ElastiCache when you primarily need caching and can tolerate data loss or rehydration from another persistent store. It’s a cost‑effective option for read scaling and short‑lived session caches.
- MemoryDB’s stronger durability and multi‑AZ replication typically increase cost compared with ElastiCache used as a cache.
- Consider cost vs. risk: evaluate the business impact of losing ephemeral state versus the ongoing cost of a durable in‑memory store.
- Common patterns used with MemoryDB:
- Session store:
HSET session:<id> userId 123 lastSeen <timestamp> - Real‑time location updates:
GEOADD drivers <lon> <lat> driver:<id> - Matchmaking queue:
LPUSH rides:requests <requestId>/BRPOP rides:requests 0
- Session store:
- Because MemoryDB is Redis‑compatible, existing Redis commands and client libraries continue to work.
MemoryDB is Redis‑compatible, so most Redis clients and commands work unchanged. You can migrate or reuse Redis-based code with minimal changes while gaining durability and Multi‑AZ availability.
Evaluate failure modes and recovery procedures before switching to MemoryDB. Strong durability reduces data loss risk, but application-level handling for edge cases (split‑brain, propagation lag, or replica promotion) is still important.
- AWS MemoryDB for Redis documentation: https://docs.aws.amazon.com/memorydb/latest/devguide/what-is-memorydb.html
- Redis project: https://redis.io/
- AWS ElastiCache for Redis: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/WhatIs.html
- Architectural patterns for real‑time systems
- Designing for eventual consistency vs. strong consistency
- Capacity planning for in‑memory datastores