Created
October 31, 2019 21:20
-
-
Save alena1108/3b1e4b052646ecfff909b81c0bd6a3f9 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**What kind of request is this (question/bug/enhancement/feature request):** bug | |
**Steps to reproduce (least amount of steps as possible):** | |
- Enable custom config in an exiting rke cluster. | |
- Edit cluster, Add this in the YAML file | |
``` | |
services: | |
kube-api: | |
secrets_encryption_config: | |
enabled: true | |
custom_config: | |
apiVersion: apiserver.config.k8s.io/v1 | |
kind: EncryptionConfiguration | |
resources: | |
- resources: | |
- secrets | |
providers: | |
- aescbc: | |
keys: | |
- name: k-jgxw9 | |
secret: QkNEQTRGRUMyMTExOUM4M0FBNjY4MTlDNTBDQTY4RkE= | |
- identity: {} | |
``` | |
- On clicking Save, cluster is provisioned again, goes into Updating state and comes up Active. | |
- The secrets are seen encrypted on the etcd | |
``` | |
/ # ETCDCTL_API=3 etcdctl get /registry/secrets/default/test1 |hexdump -C | |
00000000 2f 72 65 67 69 73 74 72 79 2f 73 65 63 72 65 74 |/registry/secret| | |
00000010 73 2f 64 65 66 61 75 6c 74 2f 74 65 73 74 31 0a |s/default/test1.| | |
00000020 6b 38 73 3a 65 6e 63 3a 61 65 73 63 62 63 3a 76 |k8s:enc:aescbc:v| | |
00000030 31 3a 6b 2d 6a 67 78 77 39 3a 4d f0 d2 16 f4 3f |1:k-jgxw9:M....?| | |
00000040 c5 3e aa 4d b5 de 08 0f da 06 02 40 d0 ef 5e 0b |.>.M.......@..^.| | |
00000050 e6 ba df 24 44 1e 17 13 75 86 fe a2 d2 83 6e e1 |...$D...u.....n.| | |
00000060 7d da 3c 77 1b 20 68 68 ab dd f0 82 3e b8 f9 42 |}.<w. hh....>..B| | |
00000070 a1 60 93 49 1f 0f 95 90 ba 26 d6 bf d4 2b 16 41 |.`.I.....&...+.A| | |
00000080 ff df 6a 88 a8 8d 99 29 45 30 72 f8 3b 81 fe 7d |..j....)E0r.;..}| | |
00000090 46 f1 47 85 27 86 7b aa 38 36 36 70 5e 07 c2 e6 |F.G.'.{.866p^...| | |
000000a0 f3 17 a0 14 c2 2a 78 88 cf b9 78 61 7f 05 78 e2 |.....*x...xa..x.| | |
000000b0 73 46 a1 e0 c3 eb 35 72 da b0 6d e9 cc 3f ae 50 |sF....5r..m..?.P| | |
000000c0 6c 45 46 e4 82 d7 2a ba 7e 3f a0 d6 b0 75 b8 b0 |lEF...*.~?...u..| | |
000000d0 6c 04 ef a3 24 23 3a 9a 67 c5 51 60 c4 60 55 81 |l...$#:.g.Q`.`U.| | |
000000e0 61 4f 07 34 97 8f c6 03 45 0f 05 df a4 3e 11 74 |aO.4....E....>.t| | |
000000f0 0f 1f cf 36 27 dd 1d 3f 07 06 68 01 24 b9 bd 6e |...6'..?..h.$..n| | |
00000100 7d 55 7c 7b 3e 24 ef 72 38 51 8c a2 03 33 ed af |}U|{>$.r8Q...3..| | |
00000110 94 bb 02 03 59 56 87 80 97 fc 4a 07 39 d8 8a c2 |....YV....J.9...| | |
00000120 1b 44 5f e8 e6 25 3b 44 0c c7 4b 87 d0 d9 c8 22 |.D_..%;D..K...."| | |
00000130 55 18 f6 bf f1 46 70 19 de de 2b e3 bb ac 44 4c |U....Fp...+...DL| | |
00000140 5e 6b 46 ec 48 78 18 fc 6f 20 71 7b f7 f4 e3 04 |^kF.Hx..o q{....| | |
00000150 ab 34 79 c9 97 37 33 43 3d c2 7c bf 72 34 1d 1e |.4y..73C=.|.r4..| | |
00000160 95 aa 1f 64 7b 22 a3 e6 c6 da 38 70 35 5f 05 70 |...d{"....8p5_.p| | |
00000170 a8 f5 ab 38 92 cf b5 92 d9 f9 3b db 71 0d 24 8a |...8......;.q.$.| | |
00000180 a8 01 34 87 68 f7 ba 6e e9 4c 0a |..4.h..n.L.| | |
``` | |
- `encryption.yaml` file on the control plane node is | |
``` | |
/etc/kubernetes/ssl# cat encryption.yaml | |
apiVersion: apiserver.config.k8s.io/v1 | |
kind: EncryptionConfiguration | |
resources: | |
- providers: | |
- aescbc: | |
keys: | |
- name: k-jgxw9 | |
secret: QkNEQTRGRUMyMTExOUM4M0FBNjY4MTlDNTBDQTY4RkE= | |
- identity: {} | |
resources: | |
- secrets | |
``` | |
- Now add new key to the config, in the first place in the keys list. Old key will be second in the list | |
- `encryption.yaml` file: | |
``` | |
root@santest115c1:/etc/kubernetes/ssl# cat encryption.yaml | |
apiVersion: apiserver.config.k8s.io/v1 | |
kind: EncryptionConfiguration | |
resources: | |
- providers: | |
- aescbc: | |
keys: | |
- name: k-abcde | |
secret: AbCDETRGRUMyMTExOUM4M0FBNjY4MTlDNTBDQTY4RkE= | |
- name: k-jgxw9 | |
secret: QkNEQTRGRUMyMTExOUM4M0FBNjY4MTlDNTBDQTY4RkE= | |
- identity: {} | |
resources: | |
- secrets | |
``` | |
- The secrets continue to be encrypted with **`k-jgxw9`** key | |
- Remove the old key from the config. Click on Save | |
- After the cluster comes up active, `encryption.yaml` file: | |
``` | |
root@santest115c1:/etc/kubernetes/ssl# cat encryption.yaml | |
apiVersion: apiserver.config.k8s.io/v1 | |
kind: EncryptionConfiguration | |
resources: | |
- providers: | |
- aescbc: | |
keys: | |
- name: k-abcde | |
secret: AbCDETRGRUMyMTExOUM4M0FBNjY4MTlDNTBDQTY4RkE= | |
- identity: {} | |
resources: | |
- secrets | |
``` | |
- But the secrets continue to be encrypted with **`k-jgxw9`** key | |
**Environment information** | |
- Rancher version (`rancher/rancher`/`rancher/server` image tag or shown bottom left in the UI): 2.3-head latest | |
- Installation option (single install/HA): single | |
<!-- | |
If the reported issue is regarding a created cluster, please provide requested info below | |
--> | |
**Cluster information** | |
- Cluster type (Hosted/Infrastructure Provider/Custom/Imported): rke DO | |
- Kubernetes version (use `kubectl version`): | |
``` | |
1.15.5 | |
``` | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment