Skip to main content
Hello, and welcome back. Earlier we discussed why Kafka infrastructure requires security. In this lesson we’ll cover the practical ways to secure a Kafka cluster: authenticating clients, enforcing access control, and protecting data at rest. These controls together reduce the blast radius of compromised services and help you meet compliance requirements. Authentication methods Before applications or microservices interact with Kafka, they must be authenticated. Common authentication approaches include username/password (SASL/SCRAM), Kerberos (SASL/GSSAPI), and certificate-based TLS (mutual TLS or mTLS). Each option has trade-offs in deployment complexity, operational overhead, and suitability for different environments.
The image is an informative graphic about authentication methods, highlighting Username/Password, Kerberos, and SSL/TLS, each with brief descriptions of their characteristics and uses.
  • Username/password (SASL/SCRAM): Simple to configure and common for development or small deployments. Works well with existing user stores but should be paired with network/TLS protections in production.
  • Kerberos (SASL/GSSAPI): Enterprise-grade identity verification with strong single sign-on capabilities. Great for large, security-conscious organizations but requires running and maintaining a Kerberos infrastructure.
  • TLS certificates (mutual TLS / mTLS): Certificate-based authentication using PKI. Widely adopted in production due to strong identity guarantees and compatibility with certificate rotation and automation tooling.
Authentication method comparison
If you don’t have engineers experienced with Kerberos, prefer TLS-based certificate authentication (mTLS) for production. It provides strong identity assurance without the operational overhead of Kerberos.
Access control (ACLs) Authentication proves who is connecting. After that, you must control what each principal can do. Kafka uses Access Control Lists (ACLs) to grant or deny operations such as READ, WRITE (PRODUCE), CREATE, DELETE, and cluster-level actions on resources like topics and consumer groups. Example ACL rules might look like:
The image illustrates examples of Access Control Lists (ACLs) for controlling user actions, such as producing or consuming data, on specific topics. It provides three examples of users with different permissions related to event topics.
A few practical notes:
  • Apply least privilege: grant each principal only the exact operations required (e.g., PRODUCE to a specific topic, not cluster-wide WRITE).
  • Use distinct principals: assign unique usernames or certificate subjects per service to simplify ACL management and audits.
  • Separate consumer and admin privileges: consumers typically need READ; avoid giving consumer principals DELETE or admin rights.
  • Manage ACLs programmatically: store ACL intents in version control and apply them via automation to avoid drift.
Data at rest Kafka brokers persist messages to disk. Kafka does not encrypt data at rest by itself, so rely on the underlying OS or storage provider for disk encryption. Options include filesystem-level encryption, full-disk OS encryption, or storage-provider encryption (for example, encrypted block storage).
The image illustrates data protection methods for keeping messages safe at rest, focusing on disk encryption through operating system encryption, hardware-level encryption, and Kafka's at-rest encryption.
Example (AWS): if your Kafka cluster runs on EC2 instances with EBS volumes, enable EBS encryption (see AWS docs) so the volumes storing Kafka logs are encrypted at rest. Managed Kafka services (e.g., Amazon MSK, Confluent Cloud) typically offer encryption-at-rest as a configurable option. Putting it together
  • Authenticate every client (producer or consumer) using an appropriate mechanism: SASL/SCRAM, Kerberos, or mTLS.
  • Enforce ACLs so authenticated principals can perform only permitted actions (produce to specific topics, consume from specific topics, etc.).
  • Protect data at rest via OS or storage-level encryption (disk, volume, or managed-service options).
  • Keep staging and QA environments as close to production as practical: use the same auth model and similar ACLs to catch security misconfigurations early.
Apply strict ACLs and certificate-based authentication in QA and production to mirror production behavior. For simple local development you can loosen restrictions, but avoid wide-open access in shared QA environments.
The image illustrates Kafka security measures, including message transit security, data at rest security, user access control, and consumer authentication. It shows a flow of data topics like "LoginEvents" and "CardPaymentEvent" between producers and consumers.
Quick checklist
  • Enable TLS for client-broker traffic and consider mTLS for strong client identity.
  • Choose an authentication mechanism that matches your operational expertise.
  • Implement least-privilege ACLs and automate their application.
  • Ensure storage volumes are encrypted at rest and manage keys securely.
  • Monitor and audit Kafka principals and ACL changes.
Further reading and references That is it for this lesson.

Watch Video