Skip to main content
In this lesson we review the AWS services most relevant to incident detection, processing, notification, remediation, and investigation. These services typically integrate to form end-to-end incident response workflows: detect an event, process and enrich it, notify stakeholders or systems, automatically remediate where appropriate, and investigate root causes. At a high level, there are five categories and 12 core services you should be familiar with:
  • Detection: find events and signals that indicate something happened.
  • Processing: transform, buffer, or run logic in response to events.
  • Notification: route information to people and systems.
  • Remediation: automatically resolve or mitigate issues.
  • Investigation: analyze and explore incidents and their causes.
Primary building blocks include AWS Health, EventBridge, CloudTrail, and CloudWatch — these form the foundation of most incident-response architectures.

​
Key building blocks

  • AWS Health — service and account-relevant health events.
  • Amazon EventBridge — central event bus with rules, routing, and transformations.
  • AWS CloudTrail — records management and optional data-plane API activity.
  • Amazon CloudWatch — metrics, logs, and traces (the observability stack).
These building blocks are commonly combined with compute, orchestration, and systems-management services to build detection and response flows. The remainder of this article breaks the categories down, highlights common integration patterns, and calls out limits, latency, and cost considerations.

​
Detection layer

The detection layer includes global service status and account-level signals:
  • Global service status → AWS Health.
  • Account and resource events → EventBridge rules and custom event buses.
  • API activity → CloudTrail (management and optional data events).
  • Metrics, logs, and traces → CloudWatch Alarms, Logs, and X‑Ray traces.
EventBridge can intercept and route events early, before downstream rule logic runs. CloudTrail delivers to S3 in an eventual manner, so account for delivery latency in real-time designs.
A slide titled "Event Source Service Limits" showing five colored cards that list limits: unlimited AWS Health API calls, 10K EventBridge rules, 1 multi-region CloudTrail, 5 CloudTrail trails, and 5K CloudWatch alarms.
Detection design tip: evaluate signal latency and regional scope for each source. Use EventBridge for low-latency routing and streaming services (Kinesis) for real-time processing. For non-real-time forensic data, rely on CloudTrail + S3.
Important latency considerations:
  • CloudTrail-to-S3 is typically eventual — expect a ~5–15 minute delivery window; not suitable for sub-minute automation.
  • For near-real-time or sub-second responses, rely on EventBridge or streaming (Kinesis).

​
Event processing services

Event processing provides compute, orchestration, buffering, and streaming:
  • AWS Lambda — event-driven compute for short-lived tasks (max 15 minutes).
  • AWS Step Functions — orchestrated workflows; Standard Workflows support long-running executions (up to 1 year), Express Workflows are optimized for high-volume, short-duration workloads (up to 5 minutes).
  • Amazon SQS — durable queueing (Standard or FIFO) for decoupling and buffering asynchronous workloads.
  • Amazon Kinesis Data Streams — high-throughput, low-latency streaming for real-time processing.
Choose Kinesis when throughput and low latency exceed EventBridge’s intended scale.
A slide titled "Event Processing Services" showing icons and names for four AWS offerings: AWS Lambda, AWS Step Functions, AWS SQS, and AWS Kinesis. Each service has short notes about capabilities (e.g., Lambda: event-driven compute with 15-minute max run; Step Functions: workflows up to 1 year or express up to 5 minutes; SQS: standard/FIFO queues; Kinesis: real-time streaming with configurable retention).

​
Notification services

Notification services distribute alerts and findings to humans and systems:
  • Amazon SNS — multi-protocol fan-out (Email, SMS, HTTP/S, Lambda, SQS, mobile push) with subscription filtering.
  • EventBridge targets — sophisticated routing and transformation to many targets; preferred when complex routing or transformations are required.
  • Amazon SES — optimized for formatted or high-volume email with templates and reputation management.
Use EventBridge when you need advanced routing, transformations, or high fan-out with rule-based targeting.
A slide titled "Notification Services – Amazon SNS" summarizing Amazon Simple Notification Service's multi-protocol notification delivery, supported protocols (Email, SMS, HTTPS, Lambda, SQS, mobile push) and message-filtering features. A pink icon with a funnel-and-nodes graphic appears on the left.
A presentation slide titled "Notification Services – Amazon SES" with a red icon showing an email/network symbol on the left. A callout on the right says, "Email-only notification is optimized for detailed reports and formatted content."

​
Automated remediation

Remediation combines orchestration, compute, and systems management:
  • AWS Lambda — ad-hoc/custom remediation actions (subject to the 15-minute runtime limit).
  • AWS Systems Manager Automation — runbooks and operational automations for standardized tasks.
  • AWS Config Remediation — automatic remediation for compliance-driven rules.
  • AWS Step Functions — coordinate multi-step or approval-based remediation workflows.
Lambda functions are limited to a 15-minute maximum execution time. For long-running or durable stateful workflows, prefer Step Functions Standard or Systems Manager Automation.
A slide titled "Automated Remediation Services" showing three AWS offerings—AWS Systems Manager Automation, AWS Config Remediation, and AWS Lambda—each with an icon and short feature bullets describing their capabilities.

​
Investigation and analysis

Use observability and management tools to investigate incidents and perform root-cause analysis:
  • CloudWatch Logs Insights — interactive log queries and ad-hoc investigation.
  • AWS X‑Ray — distributed tracing and latency analysis for microservices.
  • Systems Manager OpsCenter — aggregated operational items and remediation tracking.
  • CloudWatch ServiceLens — visualize service health, metrics, and traces together.
  • OpenSearch — powerful log analytics and dashboards for complex queries.
A presentation slide titled "Investigation and Analysis Services" showing four AWS tools—CloudWatch Logs Insights, AWS X‑Ray, Systems Manager OpsCenter, and CloudWatch ServiceLens—with icons and short feature bullets beneath each.

​
Common service integration patterns

Below are common architectural patterns and how AWS services typically chain together:
  • Event-driven remediation
    • Source (e.g., AWS Config) → EventBridge → Systems Manager Automation → CloudWatch Logs (results) → OpsCenter (lifecycle tracking)
  • Fan-out / multi-destination notification
    • Alarm → SNS topic → PagerDuty / Lambda / SQS / Email / SMS
  • Approval-based automation (human-in-the-loop)
    • EventBridge → Step Functions (approval via SNS/console/API) → on approval, trigger Systems Manager Automation or Lambda
  • Cross-account invocation
    • Use EventBridge cross-account event buses or assume-role patterns (sts:AssumeRole) and resource-based policies to invoke Systems Manager runbooks across accounts. Test IAM roles and policies thoroughly.
  • High-volume processing
    • CloudTrail or events → EventBridge → Kinesis Data Streams → Lambda consumers or processing apps → DynamoDB or other stores → CloudWatch dashboards
Permissions and service quotas are common failure points in cross-account and high-scale architectures — plan and test IAM roles and quotas early.
A slide titled "Cross-Account Architecture Considerations" with two columns: "IAM Role Requirements" and "Service Quotas." The IAM column lists permissions (Systems Manager sts:AssumeRole, EventBridge PutEvents, CloudWatch Logs PutSubscriptionFilter) and the quotas column lists limits (EventBridge rules 10,000/account/region, Systems Manager 100 concurrent automations, Lambda concurrency 1,000).

​
Cost optimization and operational concerns

Keep these cost and operational best practices in mind:
  • Centralize logging to simplify retention policies and cost management.
  • Minimize cross-region data transfer to reduce network egress costs.
  • EventBridge is commonly used as a cross-account aggregation backbone; cross-account EventBridge routing does not itself add an extra cross-account charge.
  • Tier CloudWatch log retention (hot → warm → cold) and archive older data to lower-cost storage (S3 Glacier) when appropriate.
  • Understand default quotas (e.g., Lambda concurrency, Step Functions limits, SQS FIFO TPS without batching, Systems Manager automation concurrency) and request quota increases early.
A presentation slide titled "Cross-Account Architecture Considerations – Cost Optimization" showing three colored panels labeled "Centralized Logging", "Cross-Region Charges", and "EventBridge Cross-Account" with short notes about cost impacts beneath each.

​
Service selection decision tree (summary)

When choosing services, evaluate these core questions:
  • Detection: what is the signal type?
    • Infrastructure events → AWS Health
    • Service state changes → EventBridge
    • API calls → CloudTrail (management or data events)
    • Metrics, logs, traces → CloudWatch
  • Processing: runtime, scale, and durability requirements?
    • Custom logic under 15 minutes → Lambda
    • Multi-step serverless workflows → Step Functions
    • High-volume, real-time stream processing → Kinesis Data Streams
    • Simple asynchronous buffering → SQS
  • Remediation:
    • Standard operational runbooks → Systems Manager Automation
    • Complex or custom remediation → Lambda or Step Functions
    • Compliance-driven enforcement → AWS Config Remediation + Automation
  • Investigation & analysis:
    • Log analytics → CloudWatch Logs Insights or OpenSearch
    • Distributed tracing → AWS X‑Ray
    • Aggregated operational items → Systems Manager OpsCenter
    • Security findings → Security Hub (security-focused detections)
You can use the quick reference table below to map categories to services and primary use cases.

​
Key takeaways

  • Core service categories: detection, processing, notification, remediation, and investigation.
  • Watch critical limits and latencies (e.g., Lambda 15-minute execution limit; CloudTrail→S3 delivery ~5–15 minutes).
  • Common patterns to design for: event-driven remediation, multi-destination notifications (fan-out), approval-based automation, and cross-account monitoring.
  • Cost and operations: centralize logging, tier retention, optimize compute runtimes, and consider EventBridge for cross-account aggregation.
  • Always validate IAM roles and service quotas early when building cross-account or high-scale solutions.
A presentation slide titled "Key Takeaways" summarizing AWS service categories and critical service limits. It also shows example patterns like event-driven remediation, multi-destination notification, approval-based automation, cross-account monitoring, and a brief cost-optimization note.
This is the final lesson covering Incident Response and Remediation.

Watch Video