Introduction to Consul KV Store

The Consul Key/Value (KV) Store is a centralized repository for storing configuration parameters, metadata, and arbitrary data objects. Built into Consul, it’s always enabled and ready for use, although leveraging it remains optional. Data is replicated across all server nodes—voters, non-voters, and read replicas—ensuring high availability and fault tolerance.

The image is an informational slide about a centralized Key/Value store, explaining its features and use cases, particularly in storing configuration parameters and metadata. It highlights its distributed architecture, installation with Consul, and accessibility by server and client agents.

Distributed Architecture and High Availability

Replicates data across all Consul server nodes (voting, non-voting, read replicas)
Maintains redundancy even if one or more nodes fail
Accessible by server and client agents, as well as external clients with a valid ACL token (when ACLs are enabled)

Note

Consul KV Store is designed strictly for key/value operations, not as a full database or file system.

What the Consul KV Store Is Not

The image explains what a Key/Value store is not, highlighting that it is not a full-featured database, not encrypted, lacks a directory structure, and is stored in a single datacenter without replication.

Limitation	Explanation
Not a full database	Lacks complex queries and advanced features (e.g., DynamoDB)
Not encrypted by default	Stored in plaintext—use Vault for sensitive data
No directory hierarchy	Forward slashes (`/`) in keys are part of the name, not actual folders
Single-datacenter replication	Replicates only within one datacenter, not across regions

Warning

Consul KV Store data is not encrypted by default. For secrets and sensitive information, use HashiCorp Vault.

Object Size Limitation

Each key/value object is limited to 512 KB.

The image provides additional information about Consul K/V, highlighting an object size limitation of 512KB and backup and recovery options using the `consul snapshot save` command and Consul snapshot agent for Enterprise.

Backup and Recovery

Consul supports snapshot-based backup and restore for KV data:

# Backup all KV data
consul snapshot save backup.snap

# Restore KV data from a snapshot
consul snapshot restore backup.snap

In Consul Enterprise, the Consul Snapshot Agent provides automated, policy-driven backups.

Note

See the Consul Backup and Restore guide for advanced options.

Designing the KV Structure

Collaborate with your teams to plan a KV hierarchy that meets current requirements and future growth.

The image is a slide titled "Designing the K/V Structure," providing guidelines on designing a key/value structure, emphasizing collaboration, alignment with teams, and consideration of current and future use cases. It features colorful text and a pixelated design on the right side.

Example 1: SDLC-Based Structure

Organize keys by environment:

k8s/
- dev/
- qa/
- staging/
- production/

Example keys:

k8s/staging/app3/api-key
k8s/staging/app3/certificate

Example 2: Team-Based Structure

Group keys by team and service:

The image is a diagram illustrating the design of a key/value (K/V) structure based on teams, with categories like cloud, automation, data, and apps, and subcategories such as chef, aws, TFE, and app1-3. It includes specific parameters like account numbers and API keys.

cloud/
- cloud/aws/account-number
- cloud/aws/account-name
apps/
- apps/app1/param1
- apps/app2/param2

Tailor your structure to align with application teams and infrastructure needs.

Links and References

Watch Video

Watch video content