Turning up Reliability on Database Services

Welcome, fellow Solutions Architects — Michael Forrester here. This lesson focuses on designing for reliability for database services on AWS, with practical patterns for RDS, Aurora, Redshift, DynamoDB, caching and search services, and other managed datastores. Throughout, the emphasis is on how architecture choices (Multi‑AZ, read replicas, proxies, backups, and managed features) affect availability, durability, and operational overhead.

Managed (serverless) services reduce your operational responsibility for availability and durability. When you manage compute (EC2), you also accept more reliability tasks (patching, failover automation, backups). Use managed services where appropriate to shift operational burden to AWS.

Key quick reference: service model vs responsibility

Service model	Typical customer responsibility	Recommended reliability controls
EC2-based databases (self-managed)	OS patching, replication, backups, failover	Automation (IaC), monitoring, multi-AZ/region replication
Managed RDS / Redshift / DynamoDB / Aurora	Schema, queries, capacity tuning, backups config	Use Multi‑AZ, read replicas, automated backups, monitoring
Serverless (Aurora Serverless / DynamoDB On-Demand / Redshift Serverless)	Data modeling, request throttling, cold-start handling	Autoscaling configs, proxies, pre-warming, buffering

Relational databases (SQL) are transactional and consistent by design. We begin with Amazon RDS (Relational Database Service — excluding Aurora in the Aurora section).

A presentation slide titled "RDS (main)" showing the AWS shared-responsibility model for Infrastructure, Container and Managed services, with stacked bars indicating which security and management tasks are customer vs AWS responsibilities. At the bottom are arrows showing "More Customer Responsibility → Less" and "More Customizable → Less Customizable."

RDS: Read replicas, Multi‑AZ, and failover patterns

One effective reliability pattern is to add read replicas and route read-only traffic to them while keeping writes on the primary. Use separate DNS endpoints for clarity — for example:

read.myapplication.companyname.com → reader endpoints
readwrite.myapplication.companyname.com → primary/write endpoint

An Amazon RDS architecture diagram showing a central writer DB instance replicating to two reader DB instances across different availability zones. Clients connect to the writer for read/write access and to the readers for read-only access.

Read replicas use asynchronous replication and can offload significant read traffic from the writer (in some systems up to ~80% of reads). They’re also useful for scaling read-heavy workloads and for rolling upgrades or migrations: Read-replica upgrade/migration flow:

Create a read replica of the production primary.
Upgrade the replica to the target engine/version.
Promote the upgraded replica to standalone master.
Repoint application read/write traffic to the new primary.

A flowchart showing an Amazon RDS upgrade workflow: client applications talk to a v5.6 production master, a v5.6 read replica is created and upgraded to v5.7 and then promoted to standalone, after which applications are reconnected.

For high availability (HA) on RDS engines like MySQL, PostgreSQL, or SQL Server, use Multi‑AZ deployments. Multi‑AZ provides a primary in one AZ and a synchronous standby in another AZ with automatic failover.

A slide asking what deployment strategy a Solutions Architect should use for an IBM DB2 database on Amazon RDS, with four numbered options displayed. The top recommendation shown is to configure a Multi‑AZ RDS deployment for synchronous replication and automatic failover.

Important distinctions:

Standard RDS Multi‑AZ: synchronous replication to a standby; standby usually not readable; failover is automatic and transparent to the RDS endpoint.
Aurora: storage-level replication that supports readable replicas and faster failovers; different behaviour from standard RDS.

For read-heavy applications (e‑commerce, analytics dashboards), combine read replicas and Multi‑AZ where appropriate. Distribute read replicas across AZs or regions to improve performance and resilience, keeping in mind asynchronous replicas may lag slightly behind the writer.

An infographic slide describing how a Solutions Architect can use Amazon RDS read replicas to improve an e‑commerce app’s performance and resiliency. It lists four numbered recommendations (create read replicas across AZs, convert to Multi‑AZ, increase primary instance size, and implement RDS Proxy) to handle high read traffic.

When you have a primary and a synchronous standby (Multi‑AZ), every committed write is copied to the standby, and the primary waits for acknowledgement from the standby — providing strong durability and automatic failover for writes. For critical transactional systems (finance, payments) that cannot tolerate manual DNS changes during failover, Multi‑AZ is often preferred to manual promotion of read replicas because failover is automatic and transparent to the DB endpoint.

A technical diagram of Amazon RDS Multi‑AZ architecture showing a primary writer instance, two readable standby instances across availability zones, reader and writer endpoints, and Amazon EBS storage. It also illustrates log replication and data flow between the writer, readers, and EBS.

Clarification: always verify engine-specific behavior (e.g., SQL Server, Oracle, PostgreSQL) — not all engines expose readable standbys for Multi‑AZ.

An infographic slide asking "What is the key advantage of using Amazon RDS Multi-AZ deployments with a SQL Server engine..." and showing four numbered answer boxes. The choices describe benefits such as synchronous replication to a standby for high availability and minimal downtime, read replicas for load balancing, and cross-region replication.

RDS Proxy and improved resiliency

RDS Proxy sits between applications and databases to pool and reuse connections, reduce connection storms, and keep clients connected during failover. It also integrates with AWS Secrets Manager for credential rotation and supports read-only and read-write endpoints for supported engines. Operational benefits:

Fewer open connections on DB during spikes
Smoother failovers with client connection continuity
Centralized credential rotation via Secrets Manager

An infographic slide describing a pattern where API Gateway obtains database credentials from Secrets Manager and connects to RDS Proxy, which routes to read-only and primary endpoints for reads and writes.

Monitor RDS Proxy metrics (active client connections, database connections, average connection duration) to tune proxy capacity and detect anomalies.

An AWS architecture diagram titled "RDS Proxy" showing a Load Test VPC with EC2 workers and an API Gateway connecting to a Proxy VPC containing an AWS Lambda, Amazon RDS Proxy (read-only) and an Amazon Aurora reader endpoint. It also shows integrations with AWS Secrets Manager, X-Ray, CloudWatch, STS/IAM and an alternative "No Proxy VPC" path for comparison.

Observability and operational tooling for RDS

Automated snapshots and point-in-time recovery (PITR) are essential for durability.
Use Enhanced Monitoring when you need OS-level metrics (one-second granularity available) in addition to standard CloudWatch metrics.
Use Performance Insights for SQL-level diagnostics: top queries, wait events, and tuning opportunities.
Export database logs (error, audit) to CloudWatch Logs for retention, alerting, and metric extraction.
Use RDS Event Notifications forwarded to SNS/EventBridge for automated incident workflows.

A complex diagram with EKS clusters, RDS replicas, ElastiCache, OpenSearch clusters, SNS/SQS and cross-region replication visualized as primary and standby zones with replication flows.

RDS Enhanced Monitoring provides deep OS-level insights; Performance Insights targets SQL diagnostics.

A colorful infographic asking which AWS service should be used to continuously monitor an Amazon RDS instance and receive operational notifications. It lists four numbered options: Amazon CloudWatch Alarms, AWS Config with SNS, AWS CloudTrail, and AWS X-Ray.

A presentation slide outlining how a Solutions Architect can use Enhanced Monitoring to improve resiliency for an Amazon RDS instance. Four numbered colorful boxes list actions like enabling OS-level metrics at one-second granularity, real-time resource adjustments, automatic scaling, and one-second snapshots.

Performance Insights helps diagnose slow queries and locking/contention patterns. Example queries (illustrative) that such tools surface for tuning:

-- Example queries Performance Insights might surface for tuning
WITH cte AS (
  SELECT id FROM authors LIMIT ?
)
UPDATE authors s
SET email = ?
FROM cte
WHERE s.id = cte.id;

SELECT COUNT(*)
FROM authors
WHERE id < (SELECT MAX(id) - ? FROM authors)
  AND id > (SELECT MAX(id) - ? FROM authors);

DELETE FROM authors
WHERE id < (SELECT MAX(id) - ? FROM authors)
  AND id > (SELECT MAX(id) - ? FROM authors);

A slide-style infographic explaining how Amazon RDS Performance Insights helps optimize a critical RDS instance, with a headline question at the top. Below are four numbered rounded boxes listing specific benefits like monitoring dashboards, scaling, direct schema changes, and automated recommendations.

Example RDS event payload (for alert automation) — include these events in SNS topics or EventBridge rules:

{
  "metaData": {
    "version": "2014-07-01",
    "sent": "2021-08-10T05:12:52.000+0000",
    "source": {
      "accountId": "123456789012",
      "region": "us-east-1",
      "source": "aws.rds",
      "SourceIdentifier": "dbtest1",
      "SourceArn": "arn:aws:rds:us-east-1:123456789012:db:dbtest1",
      "SourceType": "DB_INSTANCE",
      "EventID": "RDS-EVENT-0087",
      "time": "2021-08-10T05:12:52Z"
    },
    "tags": {
      "name": "dbtest1"
    }
  },
  "keyData": {
    "programName": "RDS",
    "eventName": "RDS Event Notification - ['notification']"
  },
  "baseData": {
    "occurred": "2021-08-10T05:12:52.000+0000",
    "State": "Alarm",
    "summary": "RDS DB Instance Event - DB identifier: dbtest1 EventID: RDS-EVENT-0087",
    "message": "Message: DB instance stopped"
  }
}

A slide explaining how a Solutions Architect can use Amazon RDS Event Notifications to improve resiliency. It lists four actions — send SNS alerts for events, auto-create read replicas on failover, periodically report performance to DB admins, and trigger backups to S3 when write-heavy activity is detected.

RDS also provides an RDS Recommendations page with engine-specific best-practice suggestions (instance sizing, engine upgrades, encryption, parameter changes) — use this alongside Trusted Advisor.

An infographic explaining how a Solutions Architect can use Amazon RDS Recommendations to keep RDS instances resilient. It lists four steps: enable automatic best-practice configurations, regularly review and apply suggested changes, configure real-time alerts, and use recommendations to predict incidents and trigger failover.

Amazon Aurora

Aurora is a cloud-native engine compatible with PostgreSQL or MySQL. It separates compute and storage: the storage layer is distributed across AZs (typically 6 copies) and continuously backs up to S3. This design enables lightweight compute replicas and fast failovers.

Aurora Global Database provides asynchronous cross-region replication for DR and global read scaling.

A diagram of an Amazon Aurora database architecture showing a primary cluster with writer and read replicas spread across availability zones. A remote read-only cluster is shown on the right receiving asynchronous replication from the primary.

Advanced Aurora options:

Multi‑master Aurora: multiple writer instances across AZs for continuous write availability.
Aurora Serverless: autoscaling compute with Aurora Capacity Units (ACUs) — configure min/max ACUs (min cannot be zero).

For heavy or unpredictable workloads, consider:

Pre-warming replicas;
Blue/green deployments (promote green cluster after testing);
Provisioned clusters + RDS Proxy for connection management.

An infographic slide with a question at the top and four numbered colored boxes describing Aurora Serverless features for unpredictable workloads. The boxes list automatic scaling, multi-master read/write endpoints, pre-provisioned compute for peaks, and cross-region replication.

A diagram of Amazon Aurora’s architecture showing applications connecting through a router fleet to active DB instances backed by distributed DB storage. A warm pool of additional database capacity is shown for on‑the‑fly scaling.

To mitigate cold starts in Aurora Serverless:

Pre-warm clusters with periodic dummy queries,
Use provisioned clusters with replicas for predictable peaks,
Trigger ACU scaling programmatically (Lambda + CloudWatch alarms),
Use RDS Proxy to manage client connection spikes.

RDS Proxy (revisited)

RDS Proxy reduces DB connection overhead and improves failover behavior by decoupling connections from application clients. It supports Secrets Manager integration for credential rotation.

Monitor the proxy and tune timeouts, connection pooling, and IAM auth where possible.

Amazon Redshift (data warehouse)

Redshift supports provisioned clusters (leader + compute nodes) and a serverless option. For reliability:

Use multiple compute nodes (within a region) and automated cross-region snapshots for DR.
Redshift Serverless auto-scales compute in response to workload peaks.
For global replication or DR, use cross-region snapshots or data archiving to S3.

A diagram illustrating Amazon Redshift cross-region snapshots and cluster restore. It shows a source Redshift cluster in one AWS region sending snapshots to a target region where a target cluster is created, with apps connecting.

Redshift integrates with ETL pipelines, Lambda, and streaming services — design for loose coupling using queues, S3, and event-driven components.

A detailed AWS Redshift architecture diagram showing numbered layers and data flows across availability zones and regions. It depicts EKS control/data planes, ElastiCache, OpenSearch, RDS/PostgreSQL, and backup/replication to S3 and a secondary region.

An infographic slide asking what strategy a Solutions Architect should use to establish global replication for an Amazon Redshift data warehouse. It lists four numbered options: cross-region snapshots, streaming replication between clusters, Redshift read replicas, and AWS DataSync tasks.

NoSQL and Serverless Datastores

DynamoDB

DynamoDB is a fully managed, serverless NoSQL datastore with automatic replication across AZs. Key reliability patterns:

Use on‑demand capacity for unpredictable spikes or provisioned capacity + Auto Scaling for predictable workloads.
Use Global Tables for multi-region, multi-master asynchronous replication.
Buffer ingest (Kinesis, SQS) to absorb spikes and ensure reliability.

An AWS architecture slide showing DynamoDB Global Tables replicated between two regions (us-east-1 and us-west-2). Each region has an API Gateway and Lambda, with Amazon Route 53 and Application Recovery Controller health checks handling routing.

Global secondary indexes (GSIs) have independent capacity and must be considered in autoscaling policies.

An infographic slide posing a question about how a Solutions Architect can configure an Amazon DynamoDB table to handle unpredictable workloads without disruption. It shows four numbered options including provisioned capacity, on‑demand capacity, enabling DAX, and scheduled scaling.

A slide titled "DynamoDB" showing an Auto Scaling settings panel with read and write capacity options, target utilization, and minimum/maximum provisioned capacity fields. The caption notes autoscaling is important for table and global secondary index performance and availability.

DynamoDB Streams capture item-level changes for asynchronous processing (Lambda, Kinesis, ETL), enabling event-driven architectures and safe downstream processing.

A diagram of an AWS DynamoDB data pipeline showing mock reviews flowing from Amazon S3 through Lambda into DynamoDB, with DynamoDB Streams triggering Lambda for sentiment analysis via Amazon Comprehend. Aggregated results are sent through Kinesis Firehose into S3 (Parquet) and queried by Amazon Athena.

DAX (DynamoDB Accelerator)

DAX is an in-memory caching layer for DynamoDB that reduces read latency for eventually consistent reads. It is managed and replicates across nodes for fault tolerance.

A diagram of AWS DynamoDB Accelerator (DAX) architecture showing an Amazon EC2 instance running an application and DAX client connecting to a DAX cache cluster, which sits between the app and Amazon DynamoDB inside a VPC.

OpenSearch (search & analytics)

OpenSearch uses shards and replica shards for durability and scaling. If a primary shard or node fails, reads route to replicas and replicas can be promoted to primary. Ensure:

Adequate replica counts,
Snapshot indices to S3 regularly,
Monitor cluster health and shard allocation.

A diagram of an OpenSearch logging pipeline showing Producers (application, security and service logs, traces and metrics) feeding Collectors (Kinesis agent, CloudWatch agent, Beats, fluentbit/fluentd) and Aggregators (Kinesis Data Firehose, MSK/Kafka, S3, Logstash) into Amazon OpenSearch Service. On the right are dashboards (Kibana and OpenSearch Dashboards) for visualization.

OpenSearch Serverless decouples compute from storage and stores index data in S3; it handles node management and scaling for you.

An AWS architecture diagram for an OpenSearch data pipeline showing various data sources flowing through ingestion services (Kinesis, DMS, DataSync, MSK) into S3 zones (Raw, Indexing, Clean, Enrich) and processing components (EMR, Glue, Lambda, Comprehend, SageMaker). Processed data is stored in Amazon OpenSearch Service and DynamoDB and exposed to a visualization/dashboard.

A diagram titled "OpenSearch Serverless" showing an architecture with separate Indexing and Search sections containing compute units, indexes, and arrows indicating data flow. Index data is stored in Amazon S3 and read by the search compute units for queries.

Open-source-based and In‑memory Stores

ElastiCache (Redis & Memcached)

ElastiCache adds a caching layer to reduce DB load and improve response times:

Memcached: simple sharded cache, no built-in replication.
Redis: replication, persistence options, replication groups with automatic failover.

Use replication groups with replicas across AZs for HA. Push engine logs and slow logs to CloudWatch for diagnostics.

A diagram of AWS ElastiCache architecture showing an application accessing a Redis cluster with primary and replica nodes across AZs.

A slide titled "ElastiCache (Redis or Memcached)" showing the AWS ElastiCache console Logs tab where Slow logs and Engine logs are enabled, formatted as JSON and sent to CloudWatch (redis-app). The slide caption at the bottom reads "ElastiCache also has Engine logs."

The image shows an ElastiCache (Redis or Memcached) monitoring dashboard with multiple graphs (Bytes Used, Current Items, CPU Utilization, Evictions, Swap Usage) split by node. A caption below notes that ElastiCache monitoring includes both AWS standard and application metrics.

Keep ElastiCache engines up-to-date with service updates and plan maintenance windows for minimal disruption.

An infographic showing a question about how an online gaming company should manage service updates for their Amazon ElastiCache for Redis clusters. It lists four answer choices: manually install updates, use an AWS Lambda to apply updates, use the ElastiCache dashboard to apply engine upgrades, or rely entirely on AWS automatic updates.

MemoryDB for Redis

MemoryDB provides an in-memory durable Redis-compatible store with synchronous replication across AZs and multi-AZ failover for stronger durability than standard caching solutions.

MemoryDB is ideal when services require Redis semantics plus durability (e.g., session stores with persistence requirements).

DocumentDB, Keyspaces, Neptune, and other specialized datastores

DocumentDB (MongoDB-compatible)

DocumentDB uses a distributed storage volume (similar to Aurora), replicates across AZs, and backs up to S3. It supports reader/cluster endpoints and global clusters for cross-region DR.

An AWS architecture diagram showing Amazon DocumentDB deployed as a global cluster with primary and secondary clusters replicated across US‑EAST‑1 and US‑WEST‑2. Users access the system via Amazon Route 53, regional Elastic Load Balancers, and application servers (read/write in primary, read in secondary).

Amazon Keyspaces (Cassandra-compatible)

Keyspaces is a managed, serverless Cassandra-compatible datastore with partitioning and replication across AZs. It handles provisioning, patching, and failure recovery and exposes TLS+CQL endpoints.

An infographic slide asking "How does Amazon Keyspaces (for Apache Cassandra) utilize distributed storage to manage large‑scale, globally distributed databases?" It shows four numbered answer boxes describing different architectures (master‑slave, multi‑AZ partitioning, centralized storage, and peer‑to‑peer distributed storage).

A diagram titled "Keyspaces (for Apache Cassandra)" showing Cassandra clients (drivers, cqlsh, console) connecting via TLS/CQL to Amazon Keyspaces. The Keyspaces service then replicates data to multiple encrypted storage nodes for redundancy.

Monitor Keyspaces using CloudWatch and CloudTrail for operational and security telemetry.

A presentation slide titled "Designing for Reliability – Keyspaces (for Apache Cassandra)" showing a line chart of write/successful request latency over time for different operations. Below the chart is a caption stating keyspaces are highly managed and that CloudWatch and CloudTrail will contain relevant metrics and logs.

Amazon Neptune (graph DB)

Neptune is a managed graph database with clustered storage across AZs. Deploy replicas across AZs for HA and use the cluster endpoint for client connections. Use S3 bulk load for large imports and monitor cluster health.

An AWS architecture diagram for Amazon Neptune (neo4j) showing a VPC with private subnets containing a Neptune primary (and optional replica), a Neptune cluster endpoint, and a Neptune notebook. An S3 bulk-load bucket is connected via an S3 VPC endpoint, and an organizational graph SME icon is shown below.

Immutable and Time-series Databases

Amazon QLDB (Quantum Ledger Database)

QLDB is an append-only ledger with a verifiable history of changes. It offers high durability and AZ replication, but because it is append-only, design ingestion with buffering (Kinesis, SQS) or retries to handle transient failures and to decouple producers.

An AWS architecture diagram labeled "QLDB" showing an API Gateway connected to several Lambda functions (CreateWallet, AddFunds, WithdrawFunds, GetFunds) that interact with a QLDB wallet database, Kinesis streaming and DynamoDB for transaction history. Downstream ETL components (AWS Glue, S3, Athena, Redshift) are shown for data transformation and analytics.

Amazon Timestream

Timestream is purpose-built for time-series data (IoT, telemetry). It replicates across AZs and is optimized for high-ingest workloads. Best practices:

Implement client-side retries with exponential backoff,
Buffer ingest using Kinesis or IoT Core,
Monitor ingest rates, query durations, and memory usage.

An AWS architecture diagram titled "Timestream" showing IoT devices feeding data through AWS IoT Greengrass/IoT Core into Amazon Kinesis, AWS Lambda, Amazon Timestream and S3. The pipeline connects to analytics and visualization services like Amazon QuickSight, Grafana, SageMaker and a custom application.

A slide describing a manufacturing company using Amazon TimeStream and asking which metrics to prioritize for tracking incoming data. It lists four metrics: write input records (ingested records), query duration, memory usage, and active connections.

Summary and recommended patterns

This lesson covered AWS datastores and reliability strategies across relational, data warehouse, NoSQL, caching, search, document, graph, immutable, and time-series stores. Repeatable patterns include:

Prefer managed/serverless services to shift operational burden to AWS when possible.
Use Multi‑AZ for synchronous replication and automatic failover (RDS engines, MemoryDB, Neptune).
Use read replicas for read scaling and use promotion workflows or blue/green for upgrades.
Use RDS Proxy or caching layers (DAX, ElastiCache) to decouple connection management and reduce failovers.
Implement automated backups, snapshots, cross‑region replication or global tables for DR.
Buffer and queue ingestion (Kinesis, SQS) to prevent cascading failures and enable retries.
Instrument everything: CloudWatch, Enhanced Monitoring, Performance Insights, CloudTrail, and export logs to CloudWatch Logs/S3.

Be aware of replication models and lag: asynchronous replicas can lag and are not a substitute for synchronous Multi‑AZ for critical writes. Test failover and recovery procedures regularly.

Introduction

Services - Networking

Services - Storage

Services - Compute

Services - Database

Services - Application Integration

Services - Data and ML

Services - Migration and Transfer

Services - Management and Governance

Services - Security

Bringing it all together

Designing for Security

Designing for Reliability

Designing for Performance

Designing for Cost-Optimization