Skip to content

Instantly share code, notes, and snippets.

@slackpad

slackpad/bootstrap.md

Last active September 27, 2020 17:57
Show Gist options
  • Save slackpad/d89ce0e1cc0802c3c4f2d84932fa3234 to your computer and use it in GitHub Desktop.
Save slackpad/d89ce0e1cc0802c3c4f2d84932fa3234 to your computer and use it in GitHub Desktop.
Download ZIP
Bootstrapping Consul Servers with Version 8 ACLs Enabled (v0.8.1)
Raw
bootstrap.md

NOTE - An updated and more complete example can be found here.

Bootstrapping Consul Servers with Version 8 ACLs Enabled (v0.8.x)

Start a new Consul server

Here's acl.json:

{
  "acl_datacenter": "dc1",
  "acl_master_token": "root",
  "acl_default_policy": "deny"
}

Start the server. Note that we get the expected ACL errors since the server can't register itself with the catalog yet:

$ ./consul agent -server -data-dir=/tmp/consul-node-1 -bootstrap -config-file=acl.json
==> WARNING: Bootstrap mode enabled! Do not enable unless necessary
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v0.8.1'
           Node ID: '774af59f-23d8-9255-a00c-066067a5db52'
         Node name: 'workpad.local'
        Datacenter: 'dc1'
            Server: true (bootstrap: true)
       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.183.189.160 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2017/05/05 08:19:28 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:10.183.189.160:8300 Address:10.183.189.160:8300}]
    2017/05/05 08:19:28 [INFO] raft: Node at 10.183.189.160:8300 [Follower] entering Follower state (Leader: "")
    2017/05/05 08:19:28 [INFO] serf: EventMemberJoin: workpad.local 10.183.189.160
    2017/05/05 08:19:28 [INFO] consul: Adding LAN server workpad.local (Addr: tcp/10.183.189.160:8300) (DC: dc1)
    2017/05/05 08:19:28 [INFO] serf: EventMemberJoin: workpad.local.dc1 10.183.189.160
    2017/05/05 08:19:28 [INFO] consul: Handled member-join event for server "workpad.local.dc1" in area "wan"
    2017/05/05 08:19:34 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2017/05/05 08:19:34 [INFO] raft: Node at 10.183.189.160:8300 [Candidate] entering Candidate state in term 2
    2017/05/05 08:19:34 [INFO] raft: Election won. Tally: 1
    2017/05/05 08:19:34 [INFO] raft: Node at 10.183.189.160:8300 [Leader] entering Leader state
    2017/05/05 08:19:34 [INFO] consul: cluster leadership acquired
    2017/05/05 08:19:34 [INFO] consul: New leader elected: workpad.local
    2017/05/05 08:19:34 [INFO] consul: member 'workpad.local' joined, marking health alive
    2017/05/05 08:19:35 [WARN] agent: Service 'consul' registration blocked by ACLs
    2017/05/05 08:19:35 [WARN] agent: Node info update blocked by ACLs
    2017/05/05 08:19:58 [ERR] agent: coordinate update error: Permission denied
    ...

Create a token for the server

$ curl \
    --request PUT \
    --data \
'{
  "Name": "Server Token",
  "Type": "client",
  "Rules": "node \"workpad.local\" { policy = \"write\" } service \"consul\" { policy = \"write\" }"
}' http://127.0.0.1:8500/v1/acl/create?token=root

{"ID":"fe3b8d40-0ee0-8783-6cc2-ab1aa9bb16c1"}

Configure server with token

Update acl.json with the token from the previous step:

{
  "acl_datacenter": "dc1",
  "acl_master_token": "root",
  "acl_agent_token": "fe3b8d40-0ee0-8783-6cc2-ab1aa9bb16c1",
  "acl_default_policy": "deny"
}

Stop and start the Consul server. Note that now it can register itself and the consul service with no ACL errors:

./consul agent -server -data-dir=/tmp/consul-node-1 -bootstrap -config-file=acl.json
==> WARNING: Bootstrap mode enabled! Do not enable unless necessary
==> Starting Consul agent...
==> Consul agent running!
           Version: 'v0.8.1'
           Node ID: '774af59f-23d8-9255-a00c-066067a5db52'
         Node name: 'workpad.local'
        Datacenter: 'dc1'
            Server: true (bootstrap: true)
       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600)
      Cluster Addr: 10.183.189.160 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2017/05/05 08:25:18 [INFO] raft: Initial configuration (index=1): [{Suffrage:Voter ID:10.183.189.160:8300 Address:10.183.189.160:8300}]
    2017/05/05 08:25:18 [INFO] raft: Node at 10.183.189.160:8300 [Follower] entering Follower state (Leader: "")
    2017/05/05 08:25:18 [INFO] serf: EventMemberJoin: workpad.local 10.183.189.160
    2017/05/05 08:25:18 [WARN] serf: Failed to re-join any previously known node
    2017/05/05 08:25:18 [INFO] consul: Adding LAN server workpad.local (Addr: tcp/10.183.189.160:8300) (DC: dc1)
    2017/05/05 08:25:18 [INFO] serf: EventMemberJoin: workpad.local.dc1 10.183.189.160
    2017/05/05 08:25:18 [WARN] serf: Failed to re-join any previously known node
    2017/05/05 08:25:18 [INFO] consul: Handled member-join event for server "workpad.local.dc1" in area "wan"
    2017/05/05 08:25:25 [ERR] agent: failed to sync remote state: No cluster leader
    2017/05/05 08:25:26 [WARN] raft: Heartbeat timeout from "" reached, starting election
    2017/05/05 08:25:26 [INFO] raft: Node at 10.183.189.160:8300 [Candidate] entering Candidate state in term 3
    2017/05/05 08:25:26 [INFO] raft: Election won. Tally: 1
    2017/05/05 08:25:26 [INFO] raft: Node at 10.183.189.160:8300 [Leader] entering Leader state
    2017/05/05 08:25:26 [INFO] consul: cluster leadership acquired
    2017/05/05 08:25:26 [INFO] consul: New leader elected: workpad.local
    2017/05/05 08:25:28 [INFO] agent: Synced service 'consul'
    ...
@EugenMayer
Copy link

i created a complete automation of such a stack start with all aspects including policy: deny

https://github.com/EugenMayer/consul-docker-stability-tests/tree/master/acls

i also had to run into the same inconvinience

https://github.com/EugenMayer/consul-docker-stability-tests/blob/master/acls/bin/server_acl_agent_token.sh#L3

This setup does:

  1. bootstrap 1 server, enables ACL lockdown
  2. configures no anon access and generates an acl_token for the agent clients
  3. it shares the acl_token with the client1 and client2 which start but wait for the server to provide them a token so they can register

Those scripts usually should go into an own image which just adds those and FROM consul and thats about it.

Hope that helps somebody, its more or less the above, just with full automation and 0 interaction on initial stack start and any further start

@Nmishin
Copy link

Nmishin commented Feb 25, 2020

Hi @slackpad,

Could you please help me there - I right understand that after I switched from acl_enforce_version_8 from false to true one Consul ACL master token was split to the three different tokens: acl_master_token, acl_agent_token and acl_agent_master_token?

And for the initial bootstrap of Consul cluster I need to use acl_agent_master_token and after that need to switch to the acl_master_token?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment