> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Demo Real Time Log Analysis With Insights

> Guide showing how to use AWS CloudWatch Live Tail and Logs Insights to monitor, query, and analyze Lambda logs in near real time and for historical analysis.

This guide demonstrates how to perform near real-time log monitoring and deeper analysis using AWS CloudWatch Log Analytics (Live Tail) and CloudWatch Logs Insights. You'll learn how to start a Live Tail to watch logs as they arrive and how to run and refine Logs Insights queries to filter, search, and aggregate log data from Lambda and other services.

## Open CloudWatch Log Analytics

1. In the AWS Console, open the CloudWatch service.
2. In the left-hand navigation, under "Logs", select **Log Analytics**.
3. Set the data source to **Log groups**, then choose the Lambda log group you want to inspect (for this demo it is `/aws/lambda/xray-demo`).

![A screenshot of the AWS CloudWatch Log Analytics console with the log group picker open, showing Lambda log groups (e.g., /aws/lambda/xray-demo) and filters/time-range controls. The AWS console header and account/region bar are visible across the top.](https://mintcdn.com/kodekloud-c4ac6d9a/SSy1aL2mVSdeEi1p/images/AWS-Certified-DevOps-Engineer-Professional/Domain-4-Monitoring-and-Logging/Demo-Real-Time-Log-Analysis-With-Insights/cloudwatch-log-analytics-lambda-groups.jpg?fit=max\&auto=format\&n=SSy1aL2mVSdeEi1p\&q=85\&s=bb35d1dd02783a19419a283d9bd4590b)

Click outside the log group picker to confirm your selection.

## Start Live Tail (near real-time)

Live Tail streams new log events as they are ingested into CloudWatch Logs so you can observe activity in near real-time. With the Lambda log group selected, start Live Tail and then invoke the function (for example, via the Lambda console **Test** button) to generate new events — they should appear immediately.

CloudWatch provides a SOURCE block to scope the Live Tail session. Example SOURCE query used internally by the console:

```text theme={null}
SOURCE "arn:aws:logs:us-east-2:635573991785:log-group:/aws/lambda/xray-demo" START=-1w END=0s |
fields @timestamp, @message
| sort @timestamp desc
| limit 10000
```

Sample Live Tail output (timestamps and RequestIds shortened):

```text theme={null}
@timestamp                            @message
2026-09-30T00:54:29.721Z             INIT_START Runtime Version: python:3.14.mainline.v85 Runtime Version ARN: arn:aws:lambda:us-east-2::runtime:2c75a493ba...
2026-09-30T00:54:29.814Z             START RequestId: f838e87e-ccf6-4d6f-b105-9066ae0b9fb4 Version: $LATEST
2026-09-30T00:54:29.817Z             END RequestId: f838e87e-ccf6-4d6f-b105-9066ae0b9fb4
2026-09-30T00:54:29.817Z             REPORT RequestId: f838e87e-ccf6-4d6f-b105-9066ae0b9fb4 Duration: 1.68 ms Billed Duration: 93 ms Memory Size: 128 MB Max...
```

Live Tail is best for immediate troubleshooting (errors, cold starts, runtime diagnostics). Use it to validate deployments or reproduce issues in real time. For historical analysis, aggregates, or large-scale searching, switch to Logs Insights.

## Analyze logs with Logs Insights

Open Logs Insights in a new tab so Live Tail can continue running while you query stored log events. Make sure your Logs Insights query is scoped to the same log group and the correct time range.

Basic query to list recent events (timestamp and message), newest first, limited to 20:

```text theme={null}
fields @timestamp, @message
| sort @timestamp desc
| limit 20
```

If you prefer an explicit SOURCE clause (the console will auto-add it when you pick a log group), use:

```text theme={null}
SOURCE logGroups(namePrefix: [], class: "STANDARD") START=-1w END=0s |
fields @timestamp, @message
| sort @timestamp desc
| limit 20
```

Example result table (sample output):

```text theme={null}
@timestamp                        @message
2026-09-30T00:55:20.433Z          REPORT RequestId: ecc383c2-cdde-485e-b74f-3937cf3f487d Duration: 1.67 ms Billed Duration: 2 ms Memory Size: 128 MB
2026-09-30T00:55:20.432Z          END RequestId: ecc383c2-cdde-485e-b74f-3937cf3f487d
2026-09-30T00:55:20.417Z          START RequestId: ecc383c2-cdde-485e-b74f-3937cf3f487d Version: $LATEST
2026-09-30T00:55:19.489Z          END RequestId: 7933bf08-3906-4d1f-a88a-39347b904131
```

### Common Logs Insights clauses and examples

| Clause | Purpose | Example |
| - | - | - |
| `fields` | Select fields/columns to show | `fields @timestamp, @message` |
| `filter` | Narrow results with pattern or condition | `\| filter @message like /error/i` |
| `sort` | Order results | `\| sort @timestamp desc` |
| `limit` | Restrict number of rows returned | `\| limit 20` |
| `stats` | Aggregate results (count, avg, sum, etc.) | `\| stats count() by bin(1m)` |

### Filtering for specific events

Filter for Lambda `START` lines:

```text theme={null}
fields @timestamp, @message
| filter @message like /START/
| sort @timestamp desc
| limit 20
```

Filter for errors (case-insensitive):

```text theme={null}
fields @timestamp, @message
| filter @message like /error/i
| sort @timestamp desc
| limit 20
```

If a query returns "0 matches," there were no log events matching that pattern in the selected time range for the chosen log group.

Ensure your IAM principal has the required CloudWatch Logs permissions (e.g., `logs:StartQuery`, `logs:GetQueryResults`, `logs:FilterLogEvents`, and `logs:Tail`) to run Logs Insights queries and Live Tail. Live Tail streams recent ingestion — if no events appear, check Lambda invocation, log retention, and permissions.

## Quick reference and takeaway

* Live Tail: near real-time streaming of log events as they are ingested — ideal for immediate troubleshooting and validation.
* Logs Insights: interactive query language to search, filter, and aggregate stored logs for historical analysis and dashboards.
* Build concise queries with `fields`, `filter`, `sort`, and `limit`. Add `stats` and `bin()` for aggregations over time.

Combined example: use SOURCE, filter for `START`, and list recent matches:

```text theme={null}
SOURCE logGroups(namePrefix: [], class: "STANDARD") START=-1w END=0s |
fields @timestamp, @message
| filter @message like /START/
| sort @timestamp desc
| limit 20
```

## Links and references

* AWS CloudWatch Logs Insights Overview: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html)
* CloudWatch Logs Insights Query Syntax: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL\_QuerySyntax.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html)
* CloudWatch Live Tail documentation: [https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/live-tail.html](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/live-tail.html)

With Live Tail for immediate visibility and Logs Insights for deeper queries, you can quickly investigate Lambda behavior, surface errors, and validate performance after deployments.

> **[Watch Video](https://learn.kodekloud.com/user/courses/aws-certified-devops-engineer-professional/module/15ed9473-60f4-466a-b7b2-3d9cb829e847/lesson/2c09296c-2ed4-4f0c-8226-facc8e13649f)**

> **[Practice Lab](https://learn.kodekloud.com/user/courses/aws-certified-devops-engineer-professional/module/15ed9473-60f4-466a-b7b2-3d9cb829e847/lesson/04c2c3fb-bc3f-477a-818c-f0295b89f2dc)**


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.