Skip to main content
Welcome — in this lesson you’ll provision an EC2 instance, install a single-node KRaft-based Kafka broker+controller, and prepare the host for Kafka Connect so you can later sync topic data to S3 using an S3 connector. This guide covers:
  • Creating an IAM role for SSM access
  • Launching an EC2 instance with the role attached
  • Connecting via Session Manager (browser shell)
  • Installing Java and Kafka
  • Formatting KRaft metadata storage and configuring server.properties
  • Opening the Kafka port and starting the broker
Prerequisites
  • An AWS account with permission to create IAM roles and EC2 instances.
  • Basic familiarity with the AWS Console and SSH/Session Manager.
  • Browser access for the Session Manager shell (no SSH key required for this demo).

​
Create an IAM role

Create an IAM role for EC2 that allows Session Manager (SSM) access:
  1. In the AWS Console, search for and open IAM.
  2. Click Roles → Create role.
  3. Select EC2 as the trusted entity.
  4. Click Next.
  5. Attach at minimum the AmazonSSMManagedInstanceCore managed policy. This enables Session Manager connectivity.
  6. Name the role (example: Kafka S3 Demo) and create it.
This is the EC2 instance trust policy that corresponds to the role you just created:

​
Launch an EC2 instance

Launch an EC2 instance and attach the IAM role you created. Recommended configuration for this demo:
  1. In the AWS Console, open EC2 → Instances → Launch Instance.
  2. Configure the instance using the recommended settings above.
  3. Under Advanced details, select the IAM role (Kafka S3 Demo) you created.
  4. Launch the instance.
The image shows a portion of the AWS EC2 management console, where a user is configuring the settings for launching an EC2 instance. The settings include network, firewall, and storage configurations, along with a summary of the instance details and free tier information.

​
Connect to the instance using Session Manager

Because the EC2 instance has the SSM role attached, you can open a browser shell without an SSH key.
  1. Go to EC2 → Instances.
  2. Select the instance and click Connect.
  3. Choose Session Manager and click Connect to open a browser-based shell.
Session Manager is convenient for demos and secure access: no inbound SSH port or key pairs are required. Ensure your instance has the SSM agent installed (most recent AMIs include it by default) and the attached IAM role has AmazonSSMManagedInstanceCore.

​
Switch to root, download Kafka, and inspect files

Once connected, become root and move to the home directory:
Download and extract a Kafka binary release (example uses Kafka 3.0.0 with Scala 2.13):
The Kafka distribution contains config, bin, and libs — Kafka Connect is bundled, so no separate Connect install is required. Connector-specific JARs (for S3) will be added later.

​
Install Java

KRaft and Kafka need a JDK. Check if Java is installed:
Install Amazon Corretto 8 (or another supported JDK) if Java is missing:

​
Format storage for KRaft metadata

KRaft stores metadata locally and requires initializing the storage directory with a cluster UUID. Generate a UUID:
Use the UUID to format the storage path referenced in your KRaft config (adjust the path if you extracted Kafka elsewhere):

​
Edit the KRaft server.properties

Open config/kraft/server.properties and update the settings required for a single-node KRaft cluster. Key items:
  • Enable both broker and controller roles: process.roles=broker,controller
  • Assign a node id: node.id=1
  • Define controller quorum voters for a single-node cluster
  • Bind listeners to all interfaces (0.0.0.0)
  • Set inter.broker.listener.name
  • Advertise the EC2 public IP so remote clients can connect
Example entries to add or modify in server.properties:
Replace <EC2_PUBLIC_IP> with your instance’s public IP (copy from the EC2 console) and save the file.

​
Open the Kafka broker port in the security group

Before starting the broker, allow inbound traffic on TCP port 9092 in your instance’s security group so clients can reach Kafka.
For production or organizational environments, never open Kafka to the entire internet. Restrict inbound rules to specific IP ranges (for example, your office IP or VPN CIDR). Allowing 0.0.0.0/0 exposes your cluster to attacks.
The image shows an AWS console interface for editing inbound rules, displaying security group settings for allowing specific traffic types and port ranges. There is a warning about rules allowing access from all IP addresses.

​
Start the KRaft Kafka service

Start Kafka with the KRaft configuration:
The server log streams to the terminal. Review the output for successful controller and broker startup messages and confirm that listeners bind to ports 9092 and 9093. Watch for fatal errors — if you see errors, check the server.properties values and the storage format step.
The image shows an AWS EC2 management console with details of a running instance named "kafka_s3_demo," including its state, type, and IP addresses.

​
What’s next

With a single-node KRaft broker+controller up and reachable, the next lessons will cover:
  • Downloading and installing an S3 connector (Confluent S3 connector or a community connector)
  • Preparing the connector configuration (including AWS credentials and S3 bucket settings)
  • Running Kafka Connect and syncing topic data to S3
Links and references That’s it for this lesson — in the follow-up article we’ll install and configure the S3 connector and run a demo sync from a Kafka topic to S3.

Watch Video