Vault Security Hardening

In this guide, we explore best practices for production hardening a Vault deployment. Following HashiCorp’s Vault Security Model and defense-in-depth principles, you’ll learn how to securely configure your platform, operating system, and Vault instances. This is a conceptual overview—ideal for certification preparation—focusing on secure configurations rather than OS-level demonstrations.

We’ll cover three main areas:

1. General recommendations (platform & deployment)
2. Operating system recommendations (Linux/Windows)
3. Vault-specific recommendations

1. General Recommendations

1.1 Deployment Model

Deploy Vault with minimal resource sharing—single tenancy is ideal. Moving from physical hardware to VMs to containers increases the attack surface. If you’re using Kubernetes, VMware, or cloud, dedicate separate clusters or nodes for Vault.

For example, isolate your Vault servers in a dedicated Kubernetes namespace or cluster, apart from application pods:

1.2 Restrict Node Access

Direct SSH/RDP or kubectl exec/docker exec on Vault nodes should be prohibited. Use the Vault API/CLI from a hardened jump box. For secure interactive sessions, consider HashiCorp Boundary.

# Avoid direct exec into Vault pods:
# Use remote Vault CLI instead:
vault status

1.3 Limit Services on Vault Nodes

Run only Vault (plus telemetry or log agents such as Splunk, Sumo Logic, or Datadog). Fewer services mean fewer firewall rules and less risk of binary tampering or memory exposure.

1.4 Permit Only Required Firewall Ports

Restrict ingress to only Vault’s and Consul’s necessary ports. By default:

1.5 Allow Outbound Connections to Backends

Vault requires outbound connectivity to its secret backends (e.g., cloud APIs, databases, LDAP, object storage).

1.6 Simplify with Integrated Storage

Using Vault’s Raft-based Integrated Storage reduces the number of open ports—only 8200 (client) and 8201 (Raft). No Consul ports are needed.

Or view the Raft replication topology:

1.7 Immutable Upgrades

Adopt immutable infrastructure: destroy unhealthy nodes and spin up new ones via automation. Leverage Consul Autopilot or Raft Autopilot to maintain quorum. Always add and replicate to new nodes before decommissioning old ones to avoid data loss.

2. Operating System Recommendations

2.1 Run Vault as a Non-Root User

Create a dedicated vault user and restrict directory ownership:

useradd --system --home /opt/vault vault
chown -R vault:vault /opt/vault/data

2.2 Secure Critical Directories and Files

Audit and enforce strict permissions on binaries, configs (/etc/vault.d), plugins, service definitions, logs, and snapshots:

chmod -R 740 /etc/vault.d

2.3 Protect the Storage Backend

When using Consul storage, enable ACLs, enforce TLS, limit node access, and verify hostnames in consul.hcl.

2.4 Disable Shell History

Prevent sensitive data from ending up in shell history:

echo 'set +o history' >> /etc/profile

2.5 Configure SELinux or AppArmor

Enable SELinux/AppArmor to align with CIS/DISA benchmarks and enhance OS-level protection.

2.6 Turn Off Core Dumps

Disabling core dumps prevents leakage of memory contents and encryption keys in the event of a crash.

2.7 Protect and Audit the Vault Service File

Monitor the systemd or service definition file for unauthorized changes, which could indicate binary tampering.

2.8 Frequent OS Patching

Regularly update Vault hosts. With immutable architectures, rebuild nodes using tools like Packer, Satellite, or Spacewalk.

2.9 Disable Swap

Prevent sensitive data from being written to disk by disabling swap or using mlock on Linux.

3. Vault-Specific Recommendations

3.1 Enforce TLS Everywhere

Configure trusted TLS certificates in vault.hcl (except for local development).

3.2 Secure Consul Backend

For Consul storage, enforce TLS, trusted certs, ACLs, and gossip encryption (generate with consul keygen).

3.3 Enable Auditing

Activate one or more audit devices, send logs to a centralized server, archive them, and configure alerts for critical events.

3.4 Avoid Clear-Text Credentials

Never embed access keys or HSM credentials in vault.hcl. Use environment variables or cloud IAM roles/MSIs instead.

# ❌ Do not store clear-text AWS credentials
seal "awskms" {
  region     = "us-east-1"
  kms_key_id = "1234abcd-..."
  access_key = "AKIAIOSFODNNEXAMPLE"
  secret_key = "wJalrXUtnFEMI/K7MDENG/bPxRficEXAMPLEKEY"
}

3.5 Upgrade Vault Frequently

Track new releases at releases.hashicorp.com/vault. Regular updates deliver security fixes and new cipher suites.

3.6 Discontinue Root Tokens

Root tokens bypass ACLs and never expire. Revoke the root token after setup:

vault token revoke <root-token>

Generate a new one later via unseal/recovery keys if necessary.

3.7 Verify Binary Integrity

Download Vault binaries from HashiCorp and validate checksums before use.

3.8 Disable Unused UI

If you don’t need the Vault web UI, disable it in vault.hcl:

ui = false

3.9 Encrypt Gossip Protocol

Consul gossip traffic isn’t protected by TLS—use a 32-byte key generated with consul keygen and rotate with consul keyring.

encrypt = "generated-32-byte-key"

3.10 Secure Unseal/Recovery Keys

During initialization, use PGP keys (e.g., via Keybase) so each operator receives an encrypted unseal key. Store keys offline and distribute among team members—losing them results in irreversible data loss.

3.11 Minimize Token and Lease TTLs

Define the smallest TTLs needed and set maximum TTLs to prevent runaway renewals. This also reduces load on the storage backend via garbage collection.

3.12 Follow the Principle of Least Privilege

Grant tokens only the permissions required. Separate policies for applications and users, limit wildcards (*, +), and consider templated policies for consistency.

3.13 Perform Regular Backups

Automate and schedule snapshots, then test restores:

# Open Source Raft snapshot
vault operator raft snapshot save daily.snap

# Enterprise auto-snapshot
vault write sys/storage/raft/snapshot-auto/config/daily

3.14 Integrate with Identity Providers

Leverage your existing IdP (AD, Okta, etc.) to manage user authentication. When a user is disabled upstream, Vault access is revoked automatically.

4. Monitoring and Alerting

Centralize your Vault audit logs in a SIEM or log collector (Splunk, Sumo Logic, Datadog) and set up alerts for critical events:

• Root token usage or creation
• Policy modifications
• Auth method creations
• Transit configuration changes or key deletions
• Audit log failures

References

• Vault Security Best Practices
• HashiCorp Vault Documentation
• Kubernetes Security