Skip to main content
Welcome — this guide covers Amazon Kinesis and how it enables real-time streaming data ingestion and processing for analytics, storage, and media workloads. Kinesis is a fully managed streaming platform from AWS. Think of it as a river of data: producers write records into streams and consumers read and process them in real time. The Kinesis family includes multiple services optimized for different streaming scenarios — general-purpose data streams, managed delivery into storage and analytics, time-encoded media, and stream SQL analytics. Primary Kinesis products:
  • Kinesis Data Streams — low-latency, custom data streams for real-time processing.
  • Kinesis Data Firehose — fully managed delivery streams that buffer, optionally transform, and load streaming data into storage and analytics destinations.
  • Kinesis Video Streams — ingestion and storage of time-encoded media (video, audio, and similar sensor data).
  • Kinesis Data Analytics — run SQL queries on streaming data for real-time filtering, aggregation, or enrichment before delivery.
A slide showing Amazon Kinesis branding with a large purple Kinesis icon on the left and four smaller purple icons inside a dotted box. The smaller icons are labeled Amazon Kinesis Video Streams, Data Streams, Data Firehose, and Data Analytics.
Quick product comparison Useful links and references:

​
Kinesis Data Streams

Kinesis Data Streams is the original Kinesis service designed for real-time, high-throughput streaming use cases. Producers write records into a stream, and multiple consumers independently read and process those records. Typical producer/consumer implementations use the Kinesis Client Library (KCL) or AWS SDKs. Key characteristics
  • Millisecond-level ingestion and processing latency.
  • Capacity scaling via shards: each shard provides a fixed read/write capacity; change capacity by resharding.
  • Ideal for clickstreams, telemetry, financial transactions, and gaming events.
  • Integrates with Lambda, EC2, containers, Kinesis Data Analytics, EMR, and custom consumers.
Example AWS CLI commands Create a stream with 2 shards:
Put a single record (base64-encoded data):
Increase shard count (on-demand scaling):
Developer tooling
  • Kinesis Client Library (KCL) for checkpointing and shard management.
  • AWS SDKs for direct put/get operations.
  • Enhanced fan-out for dedicated consumer throughput (reduces consumer throttling).
A diagram titled "Amazon Kinesis Data Streams" showing various input sources (laptop, webcam, mobile, IoT chip) feeding into a central Kinesis Data Streams icon. Arrows lead from Kinesis to output services: Amazon EC2, Lambda, EMR, and Kinesis Data Analytics.

​
Kinesis Data Firehose

Kinesis Data Firehose is a fully managed service that reliably delivers streaming data to destinations such as Amazon S3, Amazon Redshift, Amazon OpenSearch Service, and Splunk. Firehose handles buffering, optional transformation (via Lambda), compression, and encryption before delivery. When to choose Firehose
  • You want a low-ops pipeline to land streaming data into storage or analytics systems.
  • You prefer managed buffering and automatic scaling rather than managing shards.
  • You need built-in transformation and format conversion (e.g., JSON to Parquet via Lambda).
Firehose behavior and tuning
  • Firehose buffers data by size and/or time before delivery; typical minimum buffer interval is 60 seconds for S3.
  • Configure buffering, compression, and retry behavior per destination.
  • Supports Lambda-based transformations for simple ETL prior to delivery.
Example Firehose creation via AWS CLI (simplified):
A diagram of Amazon Kinesis Data Firehose showing various input sources (web, mobile, camera, IoT/device) flowing into a Firehose delivery stream. The Firehose forwards data to outputs like Amazon S3, Amazon RedShift, Amazon OpenSearch Service, and Splunk.
Kinesis Data Firehose is ideal for managed, low-operations delivery pipelines into data lakes and analytics systems. Choose Firehose for simple ingestion-to-storage workflows; choose Data Streams when you need the lowest possible latency and custom stream consumers.

​
Kinesis Video Streams

Kinesis Video Streams is built for ingesting, storing, and processing time-encoded media (video and audio) from devices like cameras, phones, and sensors. It fragments media into durable, timestamped chunks and provides APIs and SDKs for producers and consumers to read and process media streams. Primary capabilities
  • Durable, time-ordered storage for media fragments with multi-AZ replication.
  • Producer SDKs for camera and device ingestion (optimized for media).
  • Consumer APIs for playback, retrieval, and integration with ML services.
  • Integrations with Rekognition for video analytics, SageMaker for model training, Lambda for event-driven processing, and S3 for long-term archival.
Common use cases
  • Live video streaming and analytics (security, monitoring, smart cities).
  • Media ingestion for ML inference (object detection, activity recognition).
  • Storing and replaying sensor data with time context.
A diagram titled "Amazon Kinesis Video Streams" showing cameras and mobile devices as producers feeding into Amazon Kinesis Video Streams in the center. On the right are consumer/destination services including Amazon Rekognition, Amazon SageMaker, Amazon S3, and AWS Lambda.
Producers and consumers for Kinesis Video Streams follow a similar model to Data Streams: producer SDKs handle ingestion, and consumer libraries or AWS services pull and analyze media fragments.

​
Features and operational considerations

A slide titled "Features" showing five numbered cards with colorful circular icons. The cards list: Real-Time Data Processing, Scalability, Data Durability and Availability, Integration with AWS Services, and Custom Application Building.
Across the Kinesis family, consider these common capabilities and operational trade-offs:
  • Real-time vs near-real-time: Data Streams target millisecond latency for real-time processing. Video Streams supports low-latency media ingestion. Firehose is optimized for near-real-time delivery with buffering in seconds to minutes.
  • Scalability: Data Streams scale via shards and resharding; Firehose manages throughput automatically and requires tuning of buffer sizes and destination settings.
  • Durability and availability: Kinesis replicates data across multiple Availability Zones for high durability and availability.
  • Integration: Kinesis integrates tightly with Lambda, S3, Redshift, OpenSearch Service, Rekognition, SageMaker, EMR, and other AWS services — enabling end-to-end analytics and ML workflows.
  • Developer experience: Use KCL and SDKs for custom consumers and producers; Firehose reduces operational complexity when your goal is reliable delivery to AWS storage and analytics services.
Operational best practices
  • Monitor shard metrics (GetRecords.IteratorAgeMilliseconds, PutRecord/PutRecords metrics) and scale shards before you hit throughput limits.
  • Use enhanced fan-out for multiple high-throughput consumers.
  • Configure encryption (KMS), access control (IAM), and retention periods appropriate to your compliance needs.
  • For Firehose, tune buffer size/time and use Lambda transformations when you need simple ETL without running your own consumers.
Kinesis is a mature AWS-managed streaming platform that covers a broad set of real-time ingestion and processing needs — from telemetry and logs to video and ML-driven media analysis. Select the Kinesis product that matches your latency requirements, data type, and destination goals. Questions? Let us know.

Watch Video