This tutorial covers viewing a Consul cluster, adding a client agent, and removing an agent to maintain a healthy service mesh.
In this tutorial, you’ll learn how to view your current Consul cluster, add a new client agent, and remove an agent—either gracefully or forcefully. This ensures your service mesh remains healthy and up to date.
On your server node (consul-node-a), verify the existing members and Raft peers:
Copy
# List all cluster membersconsul members# Check Raft peer statusconsul operator raft list-peers
Example output:
Copy
Node Address Status Type Build Protocol DC Segmentconsul-node-a 10.0.101.110:8301 alive server 1.9.3+ent 2 us-east-1 <all>consul-node-b 10.0.101.248:8301 alive server 1.9.3+ent 2 us-east-1 <all>
Copy
Node ID Address State Voter RaftProtocolconsul-node-a 9655caa5-8d6d-bb5b-b087-df7acc277d60 10.0.101.110:8300 leader true 3consul-node-b 7d69d582-7308-b1e8-ff51-8c4899f2df43 10.0.101.248:8300 follower true 3
consul-node-a is the Raft leader, while consul-node-b is a follower. Maintaining at least three servers is recommended for high availability.
Node Address Status Type Build Protocol DC Segmentweb-server-01 10.0.101.177:8301 alive client 1.9.3+ent 2 us-east-1 <default>
Join the cluster
Copy
consul join 10.0.101.110
Copy
Successfully joined cluster by contacting 10.0.101.110
Confirm membership across the cluster
Copy
consul members
All nodes:
Copy
Node Address Status Type Build Protocol DC Segmentconsul-node-a 10.0.101.110:8301 alive server 1.9.3+ent 2 us-east-1 <all>consul-node-b 10.0.101.248:8301 alive server 1.9.3+ent 2 us-east-1 <all>web-server-01 10.0.101.177:8301 alive client 1.9.3+ent 2 us-east-1 <default>
For automatic retries, add a retry_join block in your client config. You can also leverage cloud auto-join or gossip keys—see Consul auto-join for details.
==> Graceful leave complete, shutting down agent...
On any server, you’ll see the client marked as left:
Copy
consul members
Copy
Node Address Status Type Build Protocol DC Segmentconsul-node-a 10.0.101.110:8301 alive server 1.9.3+ent 2 us-east-1 <all>consul-node-b 10.0.101.248:8301 alive server 1.9.3+ent 2 us-east-1 <all>web-server-01 10.0.101.177:8301 left client 1.9.3+ent 2 us-east-1 <default>
If a client is unresponsive or destroyed, use force-leave with pruning:
Copy
consul force-leave --prune web-server-01
Copy
consul members
Copy
Node Address Status Type Build Protocol DC Segmentconsul-node-a 10.0.101.110:8301 alive server 1.9.3+ent 2 us-east-1 <all>consul-node-b 10.0.101.248:8301 alive server 1.9.3+ent 2 us-east-1 <all>
force-leave is destructive and should only be used when an agent cannot leave gracefully. It immediately prunes the node from the membership list.