> ## Documentation Index
> Fetch the complete documentation index at: https://notes.kodekloud.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Kafka Security

> Overview of Kafka security best practices using a banking example, covering encryption in transit and at rest, authentication and authorization, schema use, and access controls for sensitive event data.

Hello and welcome back.

In this lesson we’ll explain core Kafka security concerns using a concise banking example and show practical controls you can apply. This article covers the common threats and recommended defenses so teams can make informed choices about confidentiality, integrity, and access control for event data.

## Example scenario

Imagine a banking application that produces events to a Kafka cluster:

* When a customer logs in, the app produces an event to the topic `login-events`.
* When a payment occurs, the app produces an event to the topic `card-payment-event`.

Various internal consumers (microservices, analytics pipelines, or partner systems) read these topics. Because these topics may contain sensitive customer information, we must answer several security questions:

* Are events protected while in transit from producers/consumers to brokers?
* Who is consuming these events and are they authorized to read them?
* How is the data stored on broker disks protected?

## Producers and consumers

* Producers: banking app writing to `login-events` and `card-payment-event`.
* Consumers: internal data consumers that read one or both topics.

From a security perspective you should ensure:

* Encryption for data in transit (TLS).
* Strong client authentication and authorization (SASL / OAuth + ACLs or RBAC).
* Protection of data at rest (disk-level encryption or application-level encryption).

## Data in transit: TLS (encryption and client auth)

Encrypt connections between clients and brokers using TLS. For mutual TLS, enable client certificate authentication so brokers verify producer/consumer identities.

Example server-side configuration snippets (server.properties):

```properties theme={null}
# Enable TLS listener
listeners=SSL://broker1:9093
advertised.listeners=SSL://broker1:9093
ssl.keystore.location=/var/private/ssl/kafka.server.keystore.jks
ssl.keystore.password=<keystore-password>
ssl.key.password=<key-password>
ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password=<truststore-password>
ssl.client.auth=required
```

Client-side properties for an SSL/TLS producer/consumer:

```properties theme={null}
security.protocol=SSL
ssl.truststore.location=/etc/ssl/kafka.client.truststore.jks
ssl.truststore.password=<truststore-password>
ssl.keystore.location=/etc/ssl/kafka.client.keystore.jks
ssl.keystore.password=<keystore-password>
ssl.key.password=<key-password>
```

Notes:

* Use strong cipher suites and rotate certificates regularly.
* Consider standard PKI (internal CA or cloud-managed) to simplify certificate management.

## Authentication and authorization: SASL, OAuth, ACLs, RBAC

Kafka supports multiple authentication mechanisms:

* SASL/SCRAM for username/password.
* SASL/GSSAPI for Kerberos.
* SASL/OAUTHBEARER for OAuth2 tokens.

Combine authentication with authorization controls (ACLs or RBAC) to enforce least privilege.

Example: grant a user produce permission to `card-payment-event` and consume permission to `login-events` using kafka-acls.sh:

```bash theme={null}
# Allow user 'payments-producer' to produce to 'card-payment-event'
kafka-acls.sh --authorizer-properties zookeeper.connect=zk:2181 \
  --add --allow-principal User:payments-producer --operation Write --topic card-payment-event

# Allow user 'analytics-consumer' to read from 'login-events'
kafka-acls.sh --authorizer-properties zookeeper.connect=zk:2181 \
  --add --allow-principal User:analytics-consumer --operation Read --topic login-events
```

If you use a managed Kafka (Confluent Cloud, MSK, etc.), you may have RBAC constructs and cloud IAM integration instead of raw ACLs.

Best practices:

* Enforce least privilege per topic and per client.
* Use short-lived tokens (OAuth) or rotate SCRAM credentials regularly.
* Log and audit ACL changes and authentication events.

## Data at rest: disk encryption and application-level encryption

Protect broker storage:

* Use OS-level disk encryption or cloud provider-managed encryption (e.g., AWS EBS encryption, Azure Disk Encryption).
* Enable full-disk encryption for on-prem clusters.
* For extremely sensitive fields, apply application-level or field-level encryption before producing to Kafka.

Remember: encryption-at-rest protects against physical disk compromise, but application-level encryption is required to protect sensitive fields from authorized Kafka consumers that should not see raw values.

## Schemas and serialization

Use schema registries and structured serialization (Avro, Protobuf) to:

* Enforce schema compatibility.
* Prevent accidental schema drift and certain misuse patterns.

However, schema enforcement does not provide confidentiality—serialization is orthogonal to encryption and access control.

<Frame>
  <img src="https://mintcdn.com/kodekloud-c4ac6d9a/lPmuTD3Rx7FZuv6W/images/Event-Streaming-with-Kafka/Deep-Dive-into-Kafka-Beyond-the-Basics/Kafka-Security/kafka-security-diagram-data-transit-storage.jpg?fit=max&auto=format&n=lPmuTD3Rx7FZuv6W&q=85&s=74067cd4c8d0e74ab51d141dd100c3da" alt="The image is a diagram about Kafka Security, highlighting potential security questions related to data in transit, data storage, and consumer access to topics. It includes components like topics &#x22;LoginEvents&#x22; and &#x22;CardPaymentEvent,&#x22; and consumers." width="1920" height="1080" data-path="images/Event-Streaming-with-Kafka/Deep-Dive-into-Kafka-Beyond-the-Basics/Kafka-Security/kafka-security-diagram-data-transit-storage.jpg" />
</Frame>

## Summary — three primary layers to secure

1. Data in transit — ensure TLS between clients and brokers and consider client certificate authentication.
2. Authentication & authorization — ensure only allowed clients can access specific topics (SASL/Kerberos/OAuth + ACLs or RBAC).
3. Data at rest — ensure stored data on broker disks is encrypted (disk encryption or application-level encryption).

|  Security Layer | Controls / Features                                                     | Example                                                                                  |
| --------------: | ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| Data in transit | TLS, mutual TLS, TLS ciphers                                            | `listeners=SSL://...` and `ssl.client.auth=required`                                     |
|    Auth / AuthZ | SASL/SCRAM, GSSAPI (Kerberos), OAuth, Kafka ACLs or RBAC                | `kafka-acls.sh --add --allow-principal User:alice --operation Read --topic login-events` |
|    Data at rest | Disk-level encryption, cloud-managed encryption, field-level encryption | EBS encryption, LUKS, application-side encryption of PII                                 |

## Quick security checklist

* [ ] Enable encryption-in-transit (TLS) for all listeners.
* [ ] Require client authentication for production clusters.
* [ ] Use SCRAM/Kerberos/OAuth for client authentication and integrate with IAM where possible.
* [ ] Apply topic-level ACLs or RBAC and follow least privilege.
* [ ] Encrypt broker disks or use provider-managed encryption.
* [ ] Consider application-level encryption for highly sensitive fields.
* [ ] Enforce schemas (Avro/Protobuf) to reduce data-quality issues and accidental misuse.
* [ ] Enable logging and auditing for authentication and ACL changes.

<Callout icon="lightbulb" color="#1CB2FE">
  When designing Kafka security for sensitive data, combine encryption-in-transit, strong authentication/authorization, and encryption-at-rest. Consider least-privilege access for consumers and evaluate whether additional protections (such as field-level encryption or tokenization) are required for highly sensitive fields.
</Callout>

Specific Kafka features and configurations you can use to implement each layer of protection will be covered in a future article.

References and further reading

* [Kafka Security Documentation](https://kafka.apache.org/documentation/#security)
* [TLS (Transport Layer Security) — Wikipedia](https://en.wikipedia.org/wiki/Transport_Layer_Security)
* [SASL — Wikipedia](https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer)
* [SCRAM — Wikipedia](https://en.wikipedia.org/wiki/Salted_Challenge_Response_Authentication_Mechanism)
* [Kerberos — MIT](https://web.mit.edu/kerberos/)
* [OAuth2 — OAuth.net](https://oauth.net/)
* [Avro](https://avro.apache.org/), [Protobuf](https://protobuf.dev/)

<CardGroup>
  <Card title="Watch Video" icon="video" cta="Learn more" href="https://learn.kodekloud.com/user/courses/event-streaming-with-kafka/module/9aa104e8-faa5-4099-977f-71744306b99d/lesson/1686796f-850f-4a9e-a360-996cbeab8364" />
</CardGroup>