Demo Metric Filter

1. Create IAM policies and roles
2. Launch an EC2 instance with the IAM role
3. Generate sample application logs
4. Push logs to CloudWatch Logs
5. Define a metric filter for HTTP 404 errors
6. View metrics and create an alarm
7. Clean up resources

​

1. Create an IAM Policy

1. Open the IAM console and select Policies > Create policy.
2. Under Service, choose EC2 and enable All EC2 permissions.

3. Click Add permissions, search for CloudWatch, and select All CloudWatch permissions.
4. Add CloudWatch Logs with All actions allowed.

5. Click Next, name the policy metric-filter-demo, then Create policy.

​

2. Create an IAM Role

1. In the IAM console, go to Roles > Create role.
2. Choose EC2 as the trusted entity, then attach the metric-filter-demo policy.
3. Name the role metric-filter-role and Create role.

​

3. Launch an EC2 Instance

1. Navigate to the EC2 console and click Launch instance.
2. Provide a name tag, select an AMI, choose or create a key pair, and configure a security group.

3. Under Advanced details > IAM instance profile, select metric-filter-role.
4. Scroll down and click Launch instance.

5. Verify the instance state is running.

​

4. Generate Application Logs

sudo su -
cd ~

cat > generate_all.sh << 'EOF'
#!/bin/bash
echo '[' > events_all.json
for i in {1..50}; do
  ts=$(date +%s%3N)
  code=$(( (RANDOM % 5) * 100 + 200 ))
  printf '{"timestamp": %d, "message": "GET /endpoint HTTP/1.1\" %d"}' \
    "$ts" "$code" >> events_all.json
  [ $i -lt 50 ] && echo ',' >> events_all.json
done
echo ']' >> events_all.json
EOF
chmod +x generate_all.sh

./generate_all.sh
ls -l events_all.json
tail -n 5 events_all.json

​

5. Create a CloudWatch Logs Group & Stream

1. In the CloudWatch console, go to Logs > Log groups > Create log group.
   • Name: application-404-error-tracker
2. Select the new group and click Create log stream.
   • Name: hostname

aws logs put-log-events \
  --log-group-name application-404-error-tracker \
  --log-stream-name hostname \
  --log-events file://events_all.json

​

6. Define a Metric Filter

1. In CloudWatch, open Logs > Log groups, select application-404-error-tracker, and click Create metric filter.
2. Enter this pattern to extract the HTTP status code:

[_, _, _, _, status_code]

3. Test against the hostname stream to validate matches.

4. Click Next and configure the metric:

• Filter name: HTTP404Filter
• Metric namespace: MyNamespace
• Metric name: ApacheNotFoundErrorCount
• Metric value: 1
• Default value: 0

5. Review settings and click Create metric filter.

​

7. Push Additional Logs

./generate_all.sh
aws logs put-log-events \
  --log-group-name application-404-error-tracker \
  --log-stream-name hostname \
  --log-events file://events_all.json

​

8. View Metric & Create an Alarm

1. In CloudWatch, go to Metrics > MyNamespace > ApacheNotFoundErrorCount.
2. Select the metric and click Create alarm.
3. Set a threshold (for example, when > 1 events in 5 minutes) and configure a notification (SNS, email, etc.).

​

9. Summary & Cleanup

• Created a custom IAM policy and role for EC2 & CloudWatch Logs
• Launched an EC2 instance with the IAM role attached
• Generated and pushed application logs to CloudWatch Logs
• Defined a metric filter for HTTP 404 errors
• Viewed the metric and configured an alarm

​

Links & References

• AWS IAM Policies
• AWS CloudWatch Logs Documentation
• AWS CLI put-log-events