HashiCorp Certified: Vault Operations Professional 2022

Build Fault Tolerant Vault Environments

Demo Disaster Recovery DR Replication

In this guide, you’ll configure Disaster Recovery (DR) replication across two Vault Enterprise clusters. We assume both single-node clusters are already initialized, unsealed, and you’re logged in as root:

  • Primary (10.1.101.199, tan background)
  • Secondary (10.1.101.108, white background)

DR replication requires Vault Enterprise; it is not supported in the open-source edition.


1. Check Current Replication Status

On the primary (10.1.101.199), confirm DR and performance replication are disabled:

ec2-user@ip-10-1-101-199 vault]$ vault read sys/replication/status
Key         Value
---         -----
dr          map[mode:disabled]
performance map[mode:disabled]

2. Enable DR Replication on Primary

Enable DR replication in primary mode:

ec2-user@ip-10-1-101-199 vault]$ vault write -force sys/replication/dr/primary/enable
WARNING! The following warnings were returned from Vault:
* This cluster is being enabled as a primary for replication. Vault will be unavailable for a brief period and will resume service shortly.

Note

Enabling primary DR replication causes a short downtime. Plan accordingly for production environments.

Verify that DR is now primary:

ec2-user@ip-10-1-101-199 vault]$ vault read sys/replication/status
Key         Value
---         -----
dr          map[cluster_id:65b5025a-b5ee-67de-6df6-6b2033dc75ed mode:primary state:running ...]
performance map[mode:disabled]

Or view as JSON:

ec2-user@ip-10-1-101-199 vault]$ vault read -format=json sys/replication/status | jq
{
  "data": {
    "dr": {
      "cluster_id": "65b5025a-b5ee-67de-d6f6-6b2033dc75ed",
      "mode": "primary",
      "state": "running",
      "known_secondaries": [],
      ...
    },
    "performance": {"mode":"disabled"}
  }
}

3. Generate a Secondary Token

Still on the primary, create a one-time wrapping token for the secondary:

ec2-user@ip-10-1-101-199 vault]$ vault write -f sys/replication/dr/primary/secondary-token id=secondary-dallas
Key                   Value
---                   -----
wrapping_token        [C|JeHAiOjE2NTM0MTIxOTIsImlhdC1hIGB...]
wrapping_token_ttl    30m
wrapping_creation_path sys/replication/dr/primary/secondary-token

Copy the wrapping_token value (valid for 30 minutes).


4. Enable DR Replication on Secondary

Switch to the secondary (10.1.101.108) and join it to the primary:

[root@ip-10-1-101-108 vault]# vault write -f sys/replication/dr/secondary/enable token=<WRAPPING_TOKEN>
WARNING! The following warnings were returned from Vault:
* Vault has successfully found secondary information; it may take a while to perform setup tasks. Vault will be unavailable until these tasks and initial sync complete.

Warning

Enabling DR on a secondary will wipe any existing data. Be sure this node is dedicated for DR replication.


5. Verify DR Status on Secondary

On the secondary, confirm replication is active:

[root@ip-10-1-101-108 vault]# vault read -format=json sys/replication/status | jq
{
  "data": {
    "dr": {
      "cluster_id": "65b5025a-b5ee-67de-6b2033dc75ed",
      "mode": "secondary",
      "state": "stream-wals",
      "connection_state": "ready",
      "secondary_id": "secondary-dallas",
      "known_primary_cluster_addr": ["https://10.1.101.199:8201"],
      ...
    },
    "performance": {"mode":"disabled"}
  }
}

A DR secondary does not serve client requests—most paths are disabled.


6. Verify DR Status on Primary

Back on the primary, list known secondaries:

[root@ip-10-1-101-199 vault]# vault read -format=json sys/replication/status | jq
{
  "data": {
    "dr": {
      "mode": "primary",
      "state": "running",
      "known_secondaries": ["secondary-dallas"],
      "secondaries": [
        {
          "api_address": "http://10.1.101.108:8200",
          "cluster_address": "https://10.1.101.108:8201",
          "connection_status": "connected",
          "node_id": "secondary-dallas",
          ...
        }
      ]
    },
    "performance": {"mode":"disabled"}
  }
}

7. DR Replication in the UI

Primary Cluster Dashboard

  1. Log in to the primary UI and go to Status → Disaster Recovery.
    The image shows a web interface for HashiCorp Vault, displaying the "Secrets Engines" section with a "cubbyhole" engine listed. The status menu on the right indicates the server is unsealed and operational.

  2. Click Disaster Recovery to view replication details:
    The image shows a web interface for disaster recovery management, indicating the cluster's state as "running" and listing one known secondary connection.

  3. Under Manage, you can disable replication, force a re-index, or demote this primary:
    The image shows a web interface for managing disaster recovery settings, with options to disable replication, recover, reindex, and demote a cluster.

Secondary Cluster Dashboard

On the secondary UI, you’ll land directly in the DR dashboard (no login prompt). It shows the cluster in stream-wals state:

The image shows a "Disaster Recovery" dashboard from a Vault application, displaying the status and details of a secondary cluster's connection and replication settings. It includes information about the primary cluster address, connection state, and replication set ID.

Under Manage, you can promote this secondary or generate an operational token:

The image shows a web interface for promoting a disaster recovery cluster in Vault, with fields for entering a DR operation token and an optional primary cluster address.


Command Summary

StepCommandDescription
1. Check Statusvault read sys/replication/statusView current DR & performance mode
2. Enable Primary DRvault write -force sys/replication/dr/primary/enableActivate DR on primary
3. Generate Secondary Tokenvault write -f sys/replication/dr/primary/secondary-tokenCreate a one-time token for secondary
4. Enable Secondary DRvault write -f sys/replication/dr/secondary/enableConnect secondary to primary
5. Verify Secondaryvault read -format=json sys/replication/statusConfirm secondary is streaming WALs
6. Verify Primaryvault read -format=json sys/replication/statusList connected secondaries

Watch Video

Watch video content

Practice Lab

Practice lab

Previous
Enable and Configure Disaster Recovery DR Replication