subm_random_fail_20210719-1

logs.txt

2021-07-19T04:04:57.9559614Z Found online and idle hosted runner(s) in the current repository's organization account that matches the required labels: 'ubuntu-latest'
2021-07-19T04:04:57.9559671Z Waiting for a hosted runner in 'organization' to pick this job...
2021-07-19T04:05:07.4624404Z Current runner version: '2.278.0'
2021-07-19T04:05:07.4654625Z ##[group]Operating System
2021-07-19T04:05:07.4655474Z Ubuntu
2021-07-19T04:05:07.4656168Z 20.04.2
2021-07-19T04:05:07.4656584Z LTS
2021-07-19T04:05:07.4657042Z ##[endgroup]
2021-07-19T04:05:07.4657525Z ##[group]Virtual Environment
2021-07-19T04:05:07.4658156Z Environment: ubuntu-20.04
2021-07-19T04:05:07.4658647Z Version: 20210712.0
2021-07-19T04:05:07.4659608Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20210712.0/images/linux/Ubuntu2004-README.md
2021-07-19T04:05:07.4660887Z Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F20210712.0
2021-07-19T04:05:07.4661686Z ##[endgroup]
2021-07-19T04:05:07.4665070Z ##[group]GITHUB_TOKEN Permissions
2021-07-19T04:05:07.4667211Z Actions: write
2021-07-19T04:05:07.4668101Z Checks: write
2021-07-19T04:05:07.4668627Z Contents: write
2021-07-19T04:05:07.4669148Z Deployments: write
2021-07-19T04:05:07.4669801Z Discussions: write
2021-07-19T04:05:07.4670551Z Issues: write
2021-07-19T04:05:07.4671203Z Metadata: read
2021-07-19T04:05:07.4671648Z Packages: write
2021-07-19T04:05:07.4672164Z PullRequests: write
2021-07-19T04:05:07.4672713Z RepositoryProjects: write
2021-07-19T04:05:07.4673335Z SecurityEvents: write
2021-07-19T04:05:07.4673821Z Statuses: write
2021-07-19T04:05:07.4674440Z ##[endgroup]
2021-07-19T04:05:07.4677575Z Prepare workflow directory
2021-07-19T04:05:07.5350339Z Prepare all required actions
2021-07-19T04:05:07.5362445Z Getting action download info
2021-07-19T04:05:07.9765550Z Download action repository 'actions/checkout@v2'
2021-07-19T04:05:09.4316475Z Download action repository 'submariner-io/shipyard@devel'
2021-07-19T04:05:10.1223617Z ##[group]Run actions/checkout@v2
2021-07-19T04:05:10.1224854Z with:
2021-07-19T04:05:10.1225514Z   repository: submariner-io/submariner-operator
2021-07-19T04:05:10.1226958Z   token: ***
2021-07-19T04:05:10.1227923Z   ssh-strict: true
2021-07-19T04:05:10.1228467Z   persist-credentials: true
2021-07-19T04:05:10.1229174Z   clean: true
2021-07-19T04:05:10.1229641Z   fetch-depth: 1
2021-07-19T04:05:10.1230101Z   lfs: false
2021-07-19T04:05:10.1230517Z   submodules: false
2021-07-19T04:05:10.1231163Z ##[endgroup]
2021-07-19T04:05:11.0241828Z Syncing repository: submariner-io/submariner-operator
2021-07-19T04:05:11.0242864Z ##[group]Getting Git version info
2021-07-19T04:05:11.0244022Z Working directory is '/home/runner/work/submariner-operator/submariner-operator'
2021-07-19T04:05:11.0245232Z [command]/usr/bin/git version
2021-07-19T04:05:11.0245994Z git version 2.32.0
2021-07-19T04:05:11.0247034Z ##[endgroup]
2021-07-19T04:05:11.0248384Z Deleting the contents of '/home/runner/work/submariner-operator/submariner-operator'
2021-07-19T04:05:11.0250133Z ##[group]Initializing the repository
2021-07-19T04:05:11.0251206Z [command]/usr/bin/git init /home/runner/work/submariner-operator/submariner-operator
2021-07-19T04:05:11.0252412Z hint: Using 'master' as the name for the initial branch. This default branch name
2021-07-19T04:05:11.0253484Z hint: is subject to change. To configure the initial branch name to use in all
2021-07-19T04:05:11.0254539Z hint: of your new repositories, which will suppress this warning, call:
2021-07-19T04:05:11.0255202Z hint: 
2021-07-19T04:05:11.0256684Z hint: 	git config --global init.defaultBranch <name>
2021-07-19T04:05:11.0257353Z hint: 
2021-07-19T04:05:11.0258322Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2021-07-19T04:05:11.0259383Z hint: 'development'. The just-created branch can be renamed via this command:
2021-07-19T04:05:11.0260347Z hint: 
2021-07-19T04:05:11.0261066Z hint: 	git branch -m <name>
2021-07-19T04:05:11.0262286Z Initialized empty Git repository in /home/runner/work/submariner-operator/submariner-operator/.git/
2021-07-19T04:05:11.0263683Z [command]/usr/bin/git remote add origin https://github.com/submariner-io/submariner-operator
2021-07-19T04:05:11.0264547Z ##[endgroup]
2021-07-19T04:05:11.0265129Z ##[group]Disabling automatic garbage collection
2021-07-19T04:05:11.0266301Z [command]/usr/bin/git config --local gc.auto 0
2021-07-19T04:05:11.0266848Z ##[endgroup]
2021-07-19T04:05:11.0269684Z ##[group]Setting up auth
2021-07-19T04:05:11.0270773Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2021-07-19T04:05:11.0272698Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
2021-07-19T04:05:11.0274686Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2021-07-19T04:05:11.0277426Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :
2021-07-19T04:05:11.0279721Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2021-07-19T04:05:11.0281255Z ##[endgroup]
2021-07-19T04:05:11.0282221Z ##[group]Fetching the repository
2021-07-19T04:05:11.0284426Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +0105e4d08ef7236073547f1383566fc76c548f6d:refs/remotes/origin/devel
2021-07-19T04:05:11.0286771Z remote: Enumerating objects: 486, done.        
2021-07-19T04:05:11.0287972Z remote: Counting objects:   0% (1/486)        
2021-07-19T04:05:11.0288606Z remote: Counting objects:   1% (5/486)        
2021-07-19T04:05:11.0289389Z remote: Counting objects:   2% (10/486)        
2021-07-19T04:05:11.0290612Z remote: Counting objects:   3% (15/486)        
2021-07-19T04:05:11.0291755Z remote: Counting objects:   4% (20/486)        
2021-07-19T04:05:11.0292377Z remote: Counting objects:   5% (25/486)        
2021-07-19T04:05:11.0293321Z remote: Counting objects:   6% (30/486)        
2021-07-19T04:05:11.0294406Z remote: Counting objects:   7% (35/486)        
2021-07-19T04:05:11.0295238Z remote: Counting objects:   8% (39/486)        
2021-07-19T04:05:11.0296172Z remote: Counting objects:   9% (44/486)        
2021-07-19T04:05:11.0297331Z remote: Counting objects:  10% (49/486)        
2021-07-19T04:05:11.0298533Z remote: Counting objects:  11% (54/486)        
2021-07-19T04:05:11.0299596Z remote: Counting objects:  12% (59/486)        
2021-07-19T04:05:11.0300385Z remote: Counting objects:  13% (64/486)        
2021-07-19T04:05:11.0301077Z remote: Counting objects:  14% (69/486)        
2021-07-19T04:05:11.0301889Z remote: Counting objects:  15% (73/486)        
2021-07-19T04:05:11.0302642Z remote: Counting objects:  16% (78/486)        
2021-07-19T04:05:11.0304446Z remote: Counting objects:  17% (83/486)        
2021-07-19T04:05:11.0305013Z remote: Counting objects:  18% (88/486)        
2021-07-19T04:05:11.0305628Z remote: Counting objects:  19% (93/486)        
2021-07-19T04:05:11.0306199Z remote: Counting objects:  20% (98/486)        
2021-07-19T04:05:11.0307158Z remote: Counting objects:  21% (103/486)        
2021-07-19T04:05:11.0307929Z remote: Counting objects:  22% (107/486)        
2021-07-19T04:05:11.0308686Z remote: Counting objects:  23% (112/486)        
2021-07-19T04:05:11.0309666Z remote: Counting objects:  24% (117/486)        
2021-07-19T04:05:11.0310261Z remote: Counting objects:  25% (122/486)        
2021-07-19T04:05:11.0311395Z remote: Counting objects:  26% (127/486)        
2021-07-19T04:05:11.0311974Z remote: Counting objects:  27% (132/486)        
2021-07-19T04:05:11.0312815Z remote: Counting objects:  28% (137/486)        
2021-07-19T04:05:11.0313600Z remote: Counting objects:  29% (141/486)        
2021-07-19T04:05:11.0314293Z remote: Counting objects:  30% (146/486)        
2021-07-19T04:05:11.0315065Z remote: Counting objects:  31% (151/486)        
2021-07-19T04:05:11.0315693Z remote: Counting objects:  32% (156/486)        
2021-07-19T04:05:11.0316636Z remote: Counting objects:  33% (161/486)        
2021-07-19T04:05:11.0317759Z remote: Counting objects:  34% (166/486)        
2021-07-19T04:05:11.0320456Z remote: Counting objects:  35% (171/486)        
2021-07-19T04:05:11.0321187Z remote: Counting objects:  36% (175/486)        
2021-07-19T04:05:11.0321924Z remote: Counting objects:  37% (180/486)        
2021-07-19T04:05:11.0322606Z remote: Counting objects:  38% (185/486)        
2021-07-19T04:05:11.0323340Z remote: Counting objects:  39% (190/486)        
2021-07-19T04:05:11.0324098Z remote: Counting objects:  40% (195/486)        
2021-07-19T04:05:11.0324860Z remote: Counting objects:  41% (200/486)        
2021-07-19T04:05:11.0325658Z remote: Counting objects:  42% (205/486)        
2021-07-19T04:05:11.0326687Z remote: Counting objects:  43% (209/486)        
2021-07-19T04:05:11.0328008Z remote: Counting objects:  44% (214/486)        
2021-07-19T04:05:11.0328790Z remote: Counting objects:  45% (219/486)        
2021-07-19T04:05:11.0329446Z remote: Counting objects:  46% (224/486)        
2021-07-19T04:05:11.0330035Z remote: Counting objects:  47% (229/486)        
2021-07-19T04:05:11.0330681Z remote: Counting objects:  48% (234/486)        
2021-07-19T04:05:11.0331591Z remote: Counting objects:  49% (239/486)        
2021-07-19T04:05:11.0332203Z remote: Counting objects:  50% (243/486)        
2021-07-19T04:05:11.0332747Z remote: Counting objects:  51% (248/486)        
2021-07-19T04:05:11.0333537Z remote: Counting objects:  52% (253/486)        
2021-07-19T04:05:11.0334155Z remote: Counting objects:  53% (258/486)        
2021-07-19T04:05:11.0334731Z remote: Counting objects:  54% (263/486)        
2021-07-19T04:05:11.0335383Z remote: Counting objects:  55% (268/486)        
2021-07-19T04:05:11.0336857Z remote: Counting objects:  56% (273/486)        
2021-07-19T04:05:11.0337726Z remote: Counting objects:  57% (278/486)        
2021-07-19T04:05:11.0338325Z remote: Counting objects:  58% (282/486)        
2021-07-19T04:05:11.0338966Z remote: Counting objects:  59% (287/486)        
2021-07-19T04:05:11.0339594Z remote: Counting objects:  60% (292/486)        
2021-07-19T04:05:11.0340608Z remote: Counting objects:  61% (297/486)        
2021-07-19T04:05:11.0341207Z remote: Counting objects:  62% (302/486)        
2021-07-19T04:05:11.0341938Z remote: Counting objects:  63% (307/486)        
2021-07-19T04:05:11.0342563Z remote: Counting objects:  64% (312/486)        
2021-07-19T04:05:11.0343305Z remote: Counting objects:  65% (316/486)        
2021-07-19T04:05:11.0345119Z remote: Counting objects:  66% (321/486)        
2021-07-19T04:05:11.0345992Z remote: Counting objects:  67% (326/486)        
2021-07-19T04:05:11.0346943Z remote: Counting objects:  68% (331/486)        
2021-07-19T04:05:11.0347997Z remote: Counting objects:  69% (336/486)        
2021-07-19T04:05:11.0348806Z remote: Counting objects:  70% (341/486)        
2021-07-19T04:05:11.0349533Z remote: Counting objects:  71% (346/486)        
2021-07-19T04:05:11.0350157Z remote: Counting objects:  72% (350/486)        
2021-07-19T04:05:11.0351115Z remote: Counting objects:  73% (355/486)        
2021-07-19T04:05:11.0351884Z remote: Counting objects:  74% (360/486)        
2021-07-19T04:05:11.0352899Z remote: Counting objects:  75% (365/486)        
2021-07-19T04:05:11.0353996Z remote: Counting objects:  76% (370/486)        
2021-07-19T04:05:11.0354614Z remote: Counting objects:  77% (375/486)        
2021-07-19T04:05:11.0355286Z remote: Counting objects:  78% (380/486)        
2021-07-19T04:05:11.0355917Z remote: Counting objects:  79% (384/486)        
2021-07-19T04:05:11.0356941Z remote: Counting objects:  80% (389/486)        
2021-07-19T04:05:11.0357749Z remote: Counting objects:  81% (394/486)        
2021-07-19T04:05:11.0358384Z remote: Counting objects:  82% (399/486)        
2021-07-19T04:05:11.0358990Z remote: Counting objects:  83% (404/486)        
2021-07-19T04:05:11.0359649Z remote: Counting objects:  84% (409/486)        
2021-07-19T04:05:11.0360611Z remote: Counting objects:  85% (414/486)        
2021-07-19T04:05:11.0361341Z remote: Counting objects:  86% (418/486)        
2021-07-19T04:05:11.0362470Z remote: Counting objects:  87% (423/486)        
2021-07-19T04:05:11.0363067Z remote: Counting objects:  88% (428/486)        
2021-07-19T04:05:11.0363691Z remote: Counting objects:  89% (433/486)        
2021-07-19T04:05:11.0364271Z remote: Counting objects:  90% (438/486)        
2021-07-19T04:05:11.0365063Z remote: Counting objects:  91% (443/486)        
2021-07-19T04:05:11.0365628Z remote: Counting objects:  92% (448/486)        
2021-07-19T04:05:11.0563497Z remote: Counting objects:  93% (452/486)        
2021-07-19T04:05:11.0564680Z remote: Counting objects:  94% (457/486)        
2021-07-19T04:05:11.0565273Z remote: Counting objects:  95% (462/486)        
2021-07-19T04:05:11.0565872Z remote: Counting objects:  96% (467/486)        
2021-07-19T04:05:11.0566458Z remote: Counting objects:  97% (472/486)        
2021-07-19T04:05:11.0567037Z remote: Counting objects:  98% (477/486)        
2021-07-19T04:05:11.0567627Z remote: Counting objects:  99% (482/486)        
2021-07-19T04:05:11.0568201Z remote: Counting objects: 100% (486/486)        
2021-07-19T04:05:11.0568815Z remote: Counting objects: 100% (486/486), done.        
2021-07-19T04:05:11.0569815Z remote: Compressing objects:   0% (1/351)        
2021-07-19T04:05:11.0570434Z remote: Compressing objects:   1% (4/351)        
2021-07-19T04:05:11.0571065Z remote: Compressing objects:   2% (8/351)        
2021-07-19T04:05:11.0571680Z remote: Compressing objects:   3% (11/351)        
2021-07-19T04:05:11.0572305Z remote: Compressing objects:   4% (15/351)        
2021-07-19T04:05:11.0572925Z remote: Compressing objects:   5% (18/351)        
2021-07-19T04:05:11.0573547Z remote: Compressing objects:   6% (22/351)        
2021-07-19T04:05:11.0574383Z remote: Compressing objects:   7% (25/351)        
2021-07-19T04:05:11.0575063Z remote: Compressing objects:   8% (29/351)        
2021-07-19T04:05:11.0575902Z remote: Compressing objects:   9% (32/351)        
2021-07-19T04:05:11.0576536Z remote: Compressing objects:  10% (36/351)        
2021-07-19T04:05:11.0577175Z remote: Compressing objects:  11% (39/351)        
2021-07-19T04:05:11.0577785Z remote: Compressing objects:  12% (43/351)        
2021-07-19T04:05:11.0578402Z remote: Compressing objects:  13% (46/351)        
2021-07-19T04:05:11.0579011Z remote: Compressing objects:  14% (50/351)        
2021-07-19T04:05:11.0579632Z remote: Compressing objects:  15% (53/351)        
2021-07-19T04:05:11.0580247Z remote: Compressing objects:  16% (57/351)        
2021-07-19T04:05:11.0580860Z remote: Compressing objects:  17% (60/351)        
2021-07-19T04:05:11.0581474Z remote: Compressing objects:  18% (64/351)        
2021-07-19T04:05:11.0582103Z remote: Compressing objects:  19% (67/351)        
2021-07-19T04:05:11.0582727Z remote: Compressing objects:  20% (71/351)        
2021-07-19T04:05:11.0583350Z remote: Compressing objects:  21% (74/351)        
2021-07-19T04:05:11.0583977Z remote: Compressing objects:  22% (78/351)        
2021-07-19T04:05:11.0584591Z remote: Compressing objects:  23% (81/351)        
2021-07-19T04:05:11.0585225Z remote: Compressing objects:  24% (85/351)        
2021-07-19T04:05:11.0585837Z remote: Compressing objects:  25% (88/351)        
2021-07-19T04:05:11.0586455Z remote: Compressing objects:  26% (92/351)        
2021-07-19T04:05:11.0587073Z remote: Compressing objects:  27% (95/351)        
2021-07-19T04:05:11.0587693Z remote: Compressing objects:  28% (99/351)        
2021-07-19T04:05:11.0588306Z remote: Compressing objects:  29% (102/351)        
2021-07-19T04:05:11.0588918Z remote: Compressing objects:  30% (106/351)        
2021-07-19T04:05:11.0589545Z remote: Compressing objects:  31% (109/351)        
2021-07-19T04:05:11.0590177Z remote: Compressing objects:  32% (113/351)        
2021-07-19T04:05:11.0590808Z remote: Compressing objects:  33% (116/351)        
2021-07-19T04:05:11.0591421Z remote: Compressing objects:  34% (120/351)        
2021-07-19T04:05:11.0592044Z remote: Compressing objects:  35% (123/351)        
2021-07-19T04:05:11.0592667Z remote: Compressing objects:  36% (127/351)        
2021-07-19T04:05:11.0593442Z remote: Compressing objects:  37% (130/351)        
2021-07-19T04:05:11.0594062Z remote: Compressing objects:  38% (134/351)        
2021-07-19T04:05:11.0594851Z remote: Compressing objects:  39% (137/351)        
2021-07-19T04:05:11.0595448Z remote: Compressing objects:  40% (141/351)        
2021-07-19T04:05:11.0596041Z remote: Compressing objects:  41% (144/351)        
2021-07-19T04:05:11.0596986Z remote: Compressing objects:  42% (148/351)        
2021-07-19T04:05:11.0597578Z remote: Compressing objects:  43% (151/351)        
2021-07-19T04:05:11.0598172Z remote: Compressing objects:  44% (155/351)        
2021-07-19T04:05:11.0598782Z remote: Compressing objects:  45% (158/351)        
2021-07-19T04:05:11.0599570Z remote: Compressing objects:  46% (162/351)        
2021-07-19T04:05:11.0600903Z remote: Compressing objects:  47% (165/351)        
2021-07-19T04:05:11.0601698Z remote: Compressing objects:  48% (169/351)        
2021-07-19T04:05:11.0602479Z remote: Compressing objects:  49% (172/351)        
2021-07-19T04:05:11.0603259Z remote: Compressing objects:  50% (176/351)        
2021-07-19T04:05:11.0603960Z remote: Compressing objects:  51% (180/351)        
2021-07-19T04:05:11.0604581Z remote: Compressing objects:  52% (183/351)        
2021-07-19T04:05:11.0605202Z remote: Compressing objects:  53% (187/351)        
2021-07-19T04:05:11.0606025Z remote: Compressing objects:  54% (190/351)        
2021-07-19T04:05:11.0606815Z remote: Compressing objects:  55% (194/351)        
2021-07-19T04:05:11.0607451Z remote: Compressing objects:  56% (197/351)        
2021-07-19T04:05:11.0608065Z remote: Compressing objects:  57% (201/351)        
2021-07-19T04:05:11.0608971Z remote: Compressing objects:  58% (204/351)        
2021-07-19T04:05:11.0609780Z remote: Compressing objects:  59% (208/351)        
2021-07-19T04:05:11.0610393Z remote: Compressing objects:  60% (211/351)        
2021-07-19T04:05:11.0611005Z remote: Compressing objects:  61% (215/351)        
2021-07-19T04:05:11.0611633Z remote: Compressing objects:  62% (218/351)        
2021-07-19T04:05:11.0612248Z remote: Compressing objects:  63% (222/351)        
2021-07-19T04:05:11.0612858Z remote: Compressing objects:  64% (225/351)        
2021-07-19T04:05:11.0613476Z remote: Compressing objects:  65% (229/351)        
2021-07-19T04:05:11.0614221Z remote: Compressing objects:  66% (232/351)        
2021-07-19T04:05:11.0614849Z remote: Compressing objects:  67% (236/351)        
2021-07-19T04:05:11.0615464Z remote: Compressing objects:  68% (239/351)        
2021-07-19T04:05:11.0616216Z remote: Compressing objects:  69% (243/351)        
2021-07-19T04:05:11.0616834Z remote: Compressing objects:  70% (246/351)        
2021-07-19T04:05:11.0617532Z remote: Compressing objects:  71% (250/351)        
2021-07-19T04:05:11.0618354Z remote: Compressing objects:  72% (253/351)        
2021-07-19T04:05:11.0619318Z remote: Compressing objects:  73% (257/351)        
2021-07-19T04:05:11.0620721Z remote: Compressing objects:  74% (260/351)        
2021-07-19T04:05:11.0621505Z remote: Compressing objects:  75% (264/351)        
2021-07-19T04:05:11.0622124Z remote: Compressing objects:  76% (267/351)        
2021-07-19T04:05:11.0623682Z remote: Compressing objects:  77% (271/351)        
2021-07-19T04:05:11.0624408Z remote: Compressing objects:  78% (274/351)        
2021-07-19T04:05:11.0625098Z remote: Compressing objects:  79% (278/351)        
2021-07-19T04:05:11.0625828Z remote: Compressing objects:  80% (281/351)        
2021-07-19T04:05:11.0626654Z remote: Compressing objects:  81% (285/351)        
2021-07-19T04:05:11.0627711Z remote: Compressing objects:  82% (288/351)        
2021-07-19T04:05:11.0628369Z remote: Compressing objects:  83% (292/351)        
2021-07-19T04:05:11.0629002Z remote: Compressing objects:  84% (295/351)        
2021-07-19T04:05:11.0629642Z remote: Compressing objects:  85% (299/351)        
2021-07-19T04:05:11.0630276Z remote: Compressing objects:  86% (302/351)        
2021-07-19T04:05:11.0631108Z remote: Compressing objects:  87% (306/351)        
2021-07-19T04:05:11.0631784Z remote: Compressing objects:  88% (309/351)        
2021-07-19T04:05:11.0632420Z remote: Compressing objects:  89% (313/351)        
2021-07-19T04:05:11.0633078Z remote: Compressing objects:  90% (316/351)        
2021-07-19T04:05:11.0633777Z remote: Compressing objects:  91% (320/351)        
2021-07-19T04:05:11.0634412Z remote: Compressing objects:  92% (323/351)        
2021-07-19T04:05:11.0635044Z remote: Compressing objects:  93% (327/351)        
2021-07-19T04:05:11.0635674Z remote: Compressing objects:  94% (330/351)        
2021-07-19T04:05:11.0636330Z remote: Compressing objects:  95% (334/351)        
2021-07-19T04:05:11.0636970Z remote: Compressing objects:  96% (337/351)        
2021-07-19T04:05:11.0637630Z remote: Compressing objects:  97% (341/351)        
2021-07-19T04:05:11.0638269Z remote: Compressing objects:  98% (344/351)        
2021-07-19T04:05:11.0638927Z remote: Compressing objects:  99% (348/351)        
2021-07-19T04:05:11.0639591Z remote: Compressing objects: 100% (351/351)        
2021-07-19T04:05:11.0640254Z remote: Compressing objects: 100% (351/351), done.        
2021-07-19T04:05:11.0640871Z Receiving objects:   0% (1/486)
2021-07-19T04:05:11.0641407Z Receiving objects:   1% (5/486)
2021-07-19T04:05:11.0641948Z Receiving objects:   2% (10/486)
2021-07-19T04:05:11.0642483Z Receiving objects:   3% (15/486)
2021-07-19T04:05:11.0643028Z Receiving objects:   4% (20/486)
2021-07-19T04:05:11.0643561Z Receiving objects:   5% (25/486)
2021-07-19T04:05:11.0644102Z Receiving objects:   6% (30/486)
2021-07-19T04:05:11.0644630Z Receiving objects:   7% (35/486)
2021-07-19T04:05:11.0645257Z Receiving objects:   8% (39/486)
2021-07-19T04:05:11.0645834Z Receiving objects:   9% (44/486)
2021-07-19T04:05:11.0646368Z Receiving objects:  10% (49/486)
2021-07-19T04:05:11.0646909Z Receiving objects:  11% (54/486)
2021-07-19T04:05:11.0647444Z Receiving objects:  12% (59/486)
2021-07-19T04:05:11.0647986Z Receiving objects:  13% (64/486)
2021-07-19T04:05:11.0648530Z Receiving objects:  14% (69/486)
2021-07-19T04:05:11.0649073Z Receiving objects:  15% (73/486)
2021-07-19T04:05:11.0649615Z Receiving objects:  16% (78/486)
2021-07-19T04:05:11.0650152Z Receiving objects:  17% (83/486)
2021-07-19T04:05:11.0653208Z Receiving objects:  18% (88/486)
2021-07-19T04:05:11.0653763Z Receiving objects:  19% (93/486)
2021-07-19T04:05:11.0654296Z Receiving objects:  20% (98/486)
2021-07-19T04:05:11.0654819Z Receiving objects:  21% (103/486)
2021-07-19T04:05:11.0655345Z Receiving objects:  22% (107/486)
2021-07-19T04:05:11.0656114Z Receiving objects:  23% (112/486)
2021-07-19T04:05:11.0656673Z Receiving objects:  24% (117/486)
2021-07-19T04:05:11.0657250Z Receiving objects:  25% (122/486)
2021-07-19T04:05:11.0657799Z Receiving objects:  26% (127/486)
2021-07-19T04:05:11.0658323Z Receiving objects:  27% (132/486)
2021-07-19T04:05:11.0658862Z Receiving objects:  28% (137/486)
2021-07-19T04:05:11.0659385Z Receiving objects:  29% (141/486)
2021-07-19T04:05:11.0659934Z Receiving objects:  30% (146/486)
2021-07-19T04:05:11.0660461Z Receiving objects:  31% (151/486)
2021-07-19T04:05:11.0661012Z Receiving objects:  32% (156/486)
2021-07-19T04:05:11.0661536Z Receiving objects:  33% (161/486)
2021-07-19T04:05:11.0662056Z Receiving objects:  34% (166/486)
2021-07-19T04:05:11.0662648Z Receiving objects:  35% (171/486)
2021-07-19T04:05:11.0663161Z Receiving objects:  36% (175/486)
2021-07-19T04:05:11.0663690Z Receiving objects:  37% (180/486)
2021-07-19T04:05:11.0664207Z Receiving objects:  38% (185/486)
2021-07-19T04:05:11.0664737Z Receiving objects:  39% (190/486)
2021-07-19T04:05:11.0665248Z Receiving objects:  40% (195/486)
2021-07-19T04:05:11.0665788Z Receiving objects:  41% (200/486)
2021-07-19T04:05:11.0666312Z Receiving objects:  42% (205/486)
2021-07-19T04:05:11.0666830Z Receiving objects:  43% (209/486)
2021-07-19T04:05:11.0667358Z Receiving objects:  44% (214/486)
2021-07-19T04:05:11.0667874Z Receiving objects:  45% (219/486)
2021-07-19T04:05:11.0668598Z Receiving objects:  46% (224/486)
2021-07-19T04:05:11.0669126Z Receiving objects:  47% (229/486)
2021-07-19T04:05:11.0669649Z Receiving objects:  48% (234/486)
2021-07-19T04:05:11.0670165Z Receiving objects:  49% (239/486)
2021-07-19T04:05:11.0670685Z Receiving objects:  50% (243/486)
2021-07-19T04:05:11.0671200Z Receiving objects:  51% (248/486)
2021-07-19T04:05:11.0671725Z Receiving objects:  52% (253/486)
2021-07-19T04:05:11.0672242Z Receiving objects:  53% (258/486)
2021-07-19T04:05:11.0672773Z Receiving objects:  54% (263/486)
2021-07-19T04:05:11.0673294Z Receiving objects:  55% (268/486)
2021-07-19T04:05:11.0673831Z Receiving objects:  56% (273/486)
2021-07-19T04:05:11.0674379Z Receiving objects:  57% (278/486)
2021-07-19T04:05:11.0674908Z Receiving objects:  58% (282/486)
2021-07-19T04:05:11.0675599Z Receiving objects:  59% (287/486)
2021-07-19T04:05:11.0676104Z Receiving objects:  60% (292/486)
2021-07-19T04:05:11.0676629Z Receiving objects:  61% (297/486)
2021-07-19T04:05:11.0677169Z Receiving objects:  62% (302/486)
2021-07-19T04:05:11.0677701Z Receiving objects:  63% (307/486)
2021-07-19T04:05:11.0678205Z Receiving objects:  64% (312/486)
2021-07-19T04:05:11.0678725Z Receiving objects:  65% (316/486)
2021-07-19T04:05:11.0679232Z Receiving objects:  66% (321/486)
2021-07-19T04:05:11.0679756Z Receiving objects:  67% (326/486)
2021-07-19T04:05:11.0680266Z Receiving objects:  68% (331/486)
2021-07-19T04:05:11.0680790Z Receiving objects:  69% (336/486)
2021-07-19T04:05:11.0681312Z Receiving objects:  70% (341/486)
2021-07-19T04:05:11.0681818Z Receiving objects:  71% (346/486)
2021-07-19T04:05:11.0682342Z Receiving objects:  72% (350/486)
2021-07-19T04:05:11.0682941Z Receiving objects:  73% (355/486)
2021-07-19T04:05:11.0683491Z Receiving objects:  74% (360/486)
2021-07-19T04:05:11.0683997Z Receiving objects:  75% (365/486)
2021-07-19T04:05:11.0684517Z Receiving objects:  76% (370/486)
2021-07-19T04:05:11.0685022Z Receiving objects:  77% (375/486)
2021-07-19T04:05:11.0685548Z Receiving objects:  78% (380/486)
2021-07-19T04:05:11.0686064Z Receiving objects:  79% (384/486)
2021-07-19T04:05:11.0686785Z Receiving objects:  80% (389/486)
2021-07-19T04:05:11.0687473Z Receiving objects:  81% (394/486)
2021-07-19T04:05:11.0687977Z Receiving objects:  82% (399/486)
2021-07-19T04:05:11.0688501Z Receiving objects:  83% (404/486)
2021-07-19T04:05:11.0689008Z Receiving objects:  84% (409/486)
2021-07-19T04:05:11.0689702Z Receiving objects:  85% (414/486)
2021-07-19T04:05:11.0690381Z Receiving objects:  86% (418/486)
2021-07-19T04:05:11.0690901Z Receiving objects:  87% (423/486)
2021-07-19T04:05:11.0692356Z remote: Total 486 (delta 156), reused 303 (delta 89), pack-reused 0        
2021-07-19T04:05:11.0693175Z Receiving objects:  88% (428/486)
2021-07-19T04:05:11.0693894Z Receiving objects:  89% (433/486)
2021-07-19T04:05:11.0694402Z Receiving objects:  90% (438/486)
2021-07-19T04:05:11.0694908Z Receiving objects:  91% (443/486)
2021-07-19T04:05:11.0695399Z Receiving objects:  92% (448/486)
2021-07-19T04:05:11.0696072Z Receiving objects:  93% (452/486)
2021-07-19T04:05:11.0696778Z Receiving objects:  94% (457/486)
2021-07-19T04:05:11.0697663Z Receiving objects:  95% (462/486)
2021-07-19T04:05:11.0698185Z Receiving objects:  96% (467/486)
2021-07-19T04:05:11.0698726Z Receiving objects:  97% (472/486)
2021-07-19T04:05:11.0699398Z Receiving objects:  98% (477/486)
2021-07-19T04:05:11.0699920Z Receiving objects:  99% (482/486)
2021-07-19T04:05:11.0700576Z Receiving objects: 100% (486/486)
2021-07-19T04:05:11.0701145Z Receiving objects: 100% (486/486), 348.32 KiB | 3.14 MiB/s, done.
2021-07-19T04:05:11.0701694Z Resolving deltas:   0% (0/156)
2021-07-19T04:05:11.0702178Z Resolving deltas:   1% (2/156)
2021-07-19T04:05:11.0702678Z Resolving deltas:   2% (4/156)
2021-07-19T04:05:11.0703157Z Resolving deltas:   3% (5/156)
2021-07-19T04:05:11.0703649Z Resolving deltas:   4% (7/156)
2021-07-19T04:05:11.0704124Z Resolving deltas:   5% (8/156)
2021-07-19T04:05:11.0704619Z Resolving deltas:   6% (10/156)
2021-07-19T04:05:11.0705097Z Resolving deltas:   7% (11/156)
2021-07-19T04:05:11.0705740Z Resolving deltas:   8% (13/156)
2021-07-19T04:05:11.0706218Z Resolving deltas:   9% (15/156)
2021-07-19T04:05:11.0706894Z Resolving deltas:  10% (16/156)
2021-07-19T04:05:11.0707389Z Resolving deltas:  11% (18/156)
2021-07-19T04:05:11.0707905Z Resolving deltas:  12% (19/156)
2021-07-19T04:05:11.0708413Z Resolving deltas:  13% (21/156)
2021-07-19T04:05:11.0708904Z Resolving deltas:  14% (22/156)
2021-07-19T04:05:11.0709408Z Resolving deltas:  15% (24/156)
2021-07-19T04:05:11.0709900Z Resolving deltas:  16% (25/156)
2021-07-19T04:05:11.0710409Z Resolving deltas:  17% (27/156)
2021-07-19T04:05:11.0710902Z Resolving deltas:  18% (29/156)
2021-07-19T04:05:11.0711417Z Resolving deltas:  19% (30/156)
2021-07-19T04:05:11.0711912Z Resolving deltas:  20% (32/156)
2021-07-19T04:05:11.0712424Z Resolving deltas:  21% (33/156)
2021-07-19T04:05:11.0712917Z Resolving deltas:  22% (35/156)
2021-07-19T04:05:11.0713425Z Resolving deltas:  23% (36/156)
2021-07-19T04:05:11.0714108Z Resolving deltas:  24% (38/156)
2021-07-19T04:05:11.0714586Z Resolving deltas:  25% (39/156)
2021-07-19T04:05:11.0715081Z Resolving deltas:  26% (41/156)
2021-07-19T04:05:11.0715561Z Resolving deltas:  27% (43/156)
2021-07-19T04:05:11.0716052Z Resolving deltas:  28% (44/156)
2021-07-19T04:05:11.0716530Z Resolving deltas:  29% (46/156)
2021-07-19T04:05:11.0717021Z Resolving deltas:  30% (47/156)
2021-07-19T04:05:11.0717887Z Resolving deltas:  31% (49/156)
2021-07-19T04:05:11.0718380Z Resolving deltas:  32% (50/156)
2021-07-19T04:05:11.0718857Z Resolving deltas:  33% (52/156)
2021-07-19T04:05:11.0719349Z Resolving deltas:  34% (54/156)
2021-07-19T04:05:11.0719908Z Resolving deltas:  35% (55/156)
2021-07-19T04:05:11.0720429Z Resolving deltas:  36% (57/156)
2021-07-19T04:05:11.0720923Z Resolving deltas:  37% (58/156)
2021-07-19T04:05:11.0721400Z Resolving deltas:  38% (60/156)
2021-07-19T04:05:11.0721891Z Resolving deltas:  39% (61/156)
2021-07-19T04:05:11.0722369Z Resolving deltas:  40% (63/156)
2021-07-19T04:05:11.0722869Z Resolving deltas:  41% (64/156)
2021-07-19T04:05:11.0726391Z Resolving deltas:  42% (66/156)
2021-07-19T04:05:11.0727056Z Resolving deltas:  43% (68/156)
2021-07-19T04:05:11.0727622Z Resolving deltas:  44% (69/156)
2021-07-19T04:05:11.0728206Z Resolving deltas:  45% (71/156)
2021-07-19T04:05:11.0728820Z Resolving deltas:  46% (72/156)
2021-07-19T04:05:11.0729408Z Resolving deltas:  47% (74/156)
2021-07-19T04:05:11.0730233Z Resolving deltas:  48% (75/156)
2021-07-19T04:05:11.0730727Z Resolving deltas:  49% (77/156)
2021-07-19T04:05:11.0731225Z Resolving deltas:  50% (78/156)
2021-07-19T04:05:11.0731702Z Resolving deltas:  51% (80/156)
2021-07-19T04:05:11.0732211Z Resolving deltas:  52% (82/156)
2021-07-19T04:05:11.0732698Z Resolving deltas:  53% (83/156)
2021-07-19T04:05:11.0733190Z Resolving deltas:  54% (85/156)
2021-07-19T04:05:11.0733668Z Resolving deltas:  55% (86/156)
2021-07-19T04:05:11.0734161Z Resolving deltas:  56% (88/156)
2021-07-19T04:05:11.0734639Z Resolving deltas:  57% (89/156)
2021-07-19T04:05:11.0735147Z Resolving deltas:  58% (91/156)
2021-07-19T04:05:11.0735834Z Resolving deltas:  59% (93/156)
2021-07-19T04:05:11.0736349Z Resolving deltas:  60% (94/156)
2021-07-19T04:05:11.0737026Z Resolving deltas:  61% (96/156)
2021-07-19T04:05:11.0737519Z Resolving deltas:  62% (97/156)
2021-07-19T04:05:11.0738030Z Resolving deltas:  63% (99/156)
2021-07-19T04:05:11.0738528Z Resolving deltas:  64% (100/156)
2021-07-19T04:05:11.0739198Z Resolving deltas:  65% (102/156)
2021-07-19T04:05:11.0739677Z Resolving deltas:  66% (103/156)
2021-07-19T04:05:11.0740173Z Resolving deltas:  67% (105/156)
2021-07-19T04:05:11.0740653Z Resolving deltas:  68% (107/156)
2021-07-19T04:05:11.0741163Z Resolving deltas:  69% (108/156)
2021-07-19T04:05:11.0741648Z Resolving deltas:  70% (110/156)
2021-07-19T04:05:11.0742148Z Resolving deltas:  71% (111/156)
2021-07-19T04:05:11.0742627Z Resolving deltas:  72% (113/156)
2021-07-19T04:05:11.0743127Z Resolving deltas:  73% (114/156)
2021-07-19T04:05:11.0743631Z Resolving deltas:  74% (116/156)
2021-07-19T04:05:11.0744325Z Resolving deltas:  75% (117/156)
2021-07-19T04:05:11.0744826Z Resolving deltas:  76% (119/156)
2021-07-19T04:05:11.0745306Z Resolving deltas:  77% (121/156)
2021-07-19T04:05:11.0745802Z Resolving deltas:  78% (122/156)
2021-07-19T04:05:11.0746288Z Resolving deltas:  79% (124/156)
2021-07-19T04:05:11.0746787Z Resolving deltas:  80% (125/156)
2021-07-19T04:05:11.0747460Z Resolving deltas:  81% (127/156)
2021-07-19T04:05:11.0747971Z Resolving deltas:  82% (128/156)
2021-07-19T04:05:11.0748468Z Resolving deltas:  83% (130/156)
2021-07-19T04:05:11.0749137Z Resolving deltas:  84% (132/156)
2021-07-19T04:05:11.0749629Z Resolving deltas:  85% (133/156)
2021-07-19T04:05:11.0750132Z Resolving deltas:  86% (135/156)
2021-07-19T04:05:11.0750631Z Resolving deltas:  87% (136/156)
2021-07-19T04:05:11.0751111Z Resolving deltas:  88% (138/156)
2021-07-19T04:05:11.0751607Z Resolving deltas:  89% (139/156)
2021-07-19T04:05:11.0752085Z Resolving deltas:  90% (141/156)
2021-07-19T04:05:11.0752591Z Resolving deltas:  91% (142/156)
2021-07-19T04:05:11.0753073Z Resolving deltas:  92% (144/156)
2021-07-19T04:05:11.0753725Z Resolving deltas:  93% (146/156)
2021-07-19T04:05:11.0754190Z Resolving deltas:  94% (147/156)
2021-07-19T04:05:11.0754678Z Resolving deltas:  95% (149/156)
2021-07-19T04:05:11.0755142Z Resolving deltas:  96% (150/156)
2021-07-19T04:05:11.0755806Z Resolving deltas:  97% (152/156)
2021-07-19T04:05:11.0756302Z Resolving deltas:  98% (153/156)
2021-07-19T04:05:11.0757153Z Resolving deltas:  99% (155/156)
2021-07-19T04:05:11.0757837Z Resolving deltas: 100% (156/156)
2021-07-19T04:05:11.0758355Z Resolving deltas: 100% (156/156), done.
2021-07-19T04:05:11.0759715Z From https://github.com/submariner-io/submariner-operator
2021-07-19T04:05:11.0760811Z  * [new ref]         0105e4d08ef7236073547f1383566fc76c548f6d -> origin/devel
2021-07-19T04:05:11.0761529Z ##[endgroup]
2021-07-19T04:05:11.0762052Z ##[group]Determining the checkout info
2021-07-19T04:05:11.0762560Z ##[endgroup]
2021-07-19T04:05:11.0763049Z ##[group]Checking out the ref
2021-07-19T04:05:11.0763935Z [command]/usr/bin/git checkout --progress --force -B devel refs/remotes/origin/devel
2021-07-19T04:05:11.0834612Z Switched to a new branch 'devel'
2021-07-19T04:05:11.0844138Z Branch 'devel' set up to track remote branch 'devel' from 'origin'.
2021-07-19T04:05:11.0850688Z ##[endgroup]
2021-07-19T04:05:11.0852167Z [command]/usr/bin/git log -1 --format='%H'
2021-07-19T04:05:11.0853571Z '0105e4d08ef7236073547f1383566fc76c548f6d'
2021-07-19T04:05:11.1078646Z ##[group]Run submariner-io/shipyard/gh-actions/e2e@devel
2021-07-19T04:05:11.1079249Z with:
2021-07-19T04:05:11.1079628Z   k8s_version: 1.19
2021-07-19T04:05:11.1080054Z   using:  globalnet 
2021-07-19T04:05:11.1080521Z   working-directory: .
2021-07-19T04:05:11.1080979Z   target: e2e
2021-07-19T04:05:11.1081369Z   focus: \[.*\]
2021-07-19T04:05:11.1081843Z   skip: \[external-dataplane\]
2021-07-19T04:05:11.1082345Z   testdir: test/e2e
2021-07-19T04:05:11.1082771Z ##[endgroup]
2021-07-19T04:05:12.0130752Z ##[group]Reclaiming free space
2021-07-19T04:05:12.7973843Z Filesystem      Size  Used Avail Use% Mounted on
2021-07-19T04:05:12.8002180Z /dev/root        84G   42G   43G  50% /
2021-07-19T04:05:12.8054621Z devtmpfs        3.4G     0  3.4G   0% /dev
2021-07-19T04:05:12.8056193Z tmpfs           3.4G   12K  3.4G   1% /dev/shm
2021-07-19T04:05:12.8057337Z tmpfs           696M  1.1M  695M   1% /run
2021-07-19T04:05:12.8058263Z tmpfs           5.0M     0  5.0M   0% /run/lock
2021-07-19T04:05:12.8059336Z tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
2021-07-19T04:05:12.8060224Z /dev/loop0       56M   56M     0 100% /snap/core18/2066
2021-07-19T04:05:12.8061110Z /dev/loop1       68M   68M     0 100% /snap/lxd/20326
2021-07-19T04:05:12.8062045Z /dev/loop2       33M   33M     0 100% /snap/snapd/12398
2021-07-19T04:05:12.8063160Z /dev/sda15      105M  7.9M   97M   8% /boot/efi
2021-07-19T04:05:12.8071530Z /dev/loop3       56M   56M     0 100% /snap/core18/2074
2021-07-19T04:05:12.8072564Z /dev/loop4       33M   33M     0 100% /snap/snapd/12159
2021-07-19T04:05:12.8073224Z /dev/sdb1        14G  4.1G  9.0G  32% /mnt
2021-07-19T04:05:12.8074014Z ##[endgroup]
2021-07-19T04:05:12.8231501Z ##[group]Disable swap
2021-07-19T04:05:12.8513835Z ##[endgroup]
2021-07-19T04:05:12.8655305Z ##[group]Report available RAM
2021-07-19T04:05:12.8679577Z               total        used        free      shared  buff/cache   available
2021-07-19T04:05:12.8705676Z Mem:          6.8Gi       486Mi       5.2Gi       7.0Mi       1.1Gi       6.0Gi
2021-07-19T04:05:12.8706097Z Swap:            0B          0B          0B
2021-07-19T04:05:12.8706942Z ##[endgroup]
2021-07-19T04:05:12.9237831Z Downloading Makefile.dapper
2021-07-19T04:05:12.9976472Z Downloading dapper
2021-07-19T04:05:13.5960988Z .dapper.tmp version v0.5.6
2021-07-19T04:05:13.6021928Z Downloading Dockerfile.dapper
2021-07-19T04:05:13.6445650Z docker network create -d bridge kind
2021-07-19T04:05:14.0096689Z 1d444dc5f238470d8f1ac99647dd24694a1c4ec7b22fb3ae5830feb23f6d5476
2021-07-19T04:05:14.0115106Z ./.dapper -m bind  make --debug=b e2e
2021-07-19T04:05:14.2725690Z Sending build context to Docker daemon  7.516MB
2021-07-19T04:05:14.2741421Z 
2021-07-19T04:05:14.2828577Z Step 1/8 : ARG BASE_BRANCH
2021-07-19T04:05:14.2833830Z Step 2/8 : FROM quay.io/submariner/shipyard-dapper-base:${BASE_BRANCH}
2021-07-19T04:05:15.0964166Z devel: Pulling from submariner/shipyard-dapper-base
2021-07-19T04:05:15.0977205Z b1495d80d526: Pulling fs layer
2021-07-19T04:05:15.0980618Z fb3037ce215d: Pulling fs layer
2021-07-19T04:05:15.0981108Z 95344349df1d: Pulling fs layer
2021-07-19T04:05:15.0981565Z a3e11b0fdbf6: Pulling fs layer
2021-07-19T04:05:15.0982235Z b2d0ef6cbe4f: Pulling fs layer
2021-07-19T04:05:15.1019144Z fa188fe30729: Pulling fs layer
2021-07-19T04:05:15.1021908Z 05ca4b0545ef: Pulling fs layer
2021-07-19T04:05:15.1022447Z b17ce80b21da: Pulling fs layer
2021-07-19T04:05:15.1022861Z a3e11b0fdbf6: Waiting
2021-07-19T04:05:15.1023372Z b2d0ef6cbe4f: Waiting
2021-07-19T04:05:15.1023805Z fa188fe30729: Waiting
2021-07-19T04:05:15.1024190Z 05ca4b0545ef: Waiting
2021-07-19T04:05:15.1024561Z b17ce80b21da: Waiting
2021-07-19T04:05:16.9336379Z 95344349df1d: Verifying Checksum
2021-07-19T04:05:16.9336857Z 95344349df1d: Download complete
2021-07-19T04:05:17.0591671Z b1495d80d526: Verifying Checksum
2021-07-19T04:05:17.0592155Z b1495d80d526: Download complete
2021-07-19T04:05:17.2197217Z a3e11b0fdbf6: Verifying Checksum
2021-07-19T04:05:17.2197732Z a3e11b0fdbf6: Download complete
2021-07-19T04:05:17.3742869Z b2d0ef6cbe4f: Verifying Checksum
2021-07-19T04:05:17.3743429Z b2d0ef6cbe4f: Download complete
2021-07-19T04:05:17.6377607Z 05ca4b0545ef: Verifying Checksum
2021-07-19T04:05:17.6378159Z 05ca4b0545ef: Download complete
2021-07-19T04:05:17.6489333Z fa188fe30729: Verifying Checksum
2021-07-19T04:05:17.6489847Z fa188fe30729: Download complete
2021-07-19T04:05:17.9826732Z b17ce80b21da: Verifying Checksum
2021-07-19T04:05:17.9830586Z b17ce80b21da: Download complete
2021-07-19T04:05:20.2150057Z b1495d80d526: Pull complete
2021-07-19T04:05:23.5137177Z fb3037ce215d: Verifying Checksum
2021-07-19T04:05:23.5138043Z fb3037ce215d: Download complete
2021-07-19T04:05:34.5195279Z fb3037ce215d: Pull complete
2021-07-19T04:05:36.1229984Z 95344349df1d: Pull complete
2021-07-19T04:05:36.1989880Z a3e11b0fdbf6: Pull complete
2021-07-19T04:05:36.2852561Z b2d0ef6cbe4f: Pull complete
2021-07-19T04:05:36.4704326Z fa188fe30729: Pull complete
2021-07-19T04:05:36.5528582Z 05ca4b0545ef: Pull complete
2021-07-19T04:05:36.6429410Z b17ce80b21da: Pull complete
2021-07-19T04:05:36.6485188Z Digest: sha256:29762595cdd1efd22ed179cd45188c3fef4adc232f27415f2ad4f15e621053b8
2021-07-19T04:05:36.6509342Z Status: Downloaded newer image for quay.io/submariner/shipyard-dapper-base:devel
2021-07-19T04:05:36.6525679Z  ---> 92b726655a41
2021-07-19T04:05:36.6526134Z Step 3/8 : ARG PROJECT
2021-07-19T04:05:36.6764322Z  ---> Running in eb4a9f73520d
2021-07-19T04:05:37.6989569Z Removing intermediate container eb4a9f73520d
2021-07-19T04:05:37.6990937Z  ---> c19150ff2386
2021-07-19T04:05:37.6992373Z Step 4/8 : ENV DAPPER_ENV="QUAY_USERNAME QUAY_PASSWORD CLUSTERS_ARGS DEPLOY_ARGS CLEANUP_ARGS E2E_ARGS RELEASE_ARGS MAKEFLAGS FOCUS SKIP PLUGIN E2E_TESTDIR"     DAPPER_SOURCE=/go/src/github.com/submariner-io/${PROJECT} DAPPER_DOCKER_SOCKET=true
2021-07-19T04:05:37.7307149Z  ---> Running in 0cd7ae7d27f8
2021-07-19T04:05:38.7450287Z Removing intermediate container 0cd7ae7d27f8
2021-07-19T04:05:38.7451394Z  ---> b262597a6354
2021-07-19T04:05:38.7451899Z Step 5/8 : ENV DAPPER_OUTPUT=${DAPPER_SOURCE}/output
2021-07-19T04:05:38.7769931Z  ---> Running in 0ca1d698f419
2021-07-19T04:05:39.7941475Z Removing intermediate container 0ca1d698f419
2021-07-19T04:05:39.7946050Z  ---> be9f13ce0b82
2021-07-19T04:05:39.7951572Z Step 6/8 : WORKDIR ${DAPPER_SOURCE}
2021-07-19T04:05:39.8301267Z  ---> Running in e94f5bfdad28
2021-07-19T04:05:40.9176715Z Removing intermediate container e94f5bfdad28
2021-07-19T04:05:40.9185726Z  ---> 92032a324cba
2021-07-19T04:05:40.9186732Z Step 7/8 : ENTRYPOINT ["/opt/shipyard/scripts/entry"]
2021-07-19T04:05:40.9380959Z  ---> Running in c07f8d4d85f9
2021-07-19T04:05:42.0081475Z Removing intermediate container c07f8d4d85f9
2021-07-19T04:05:42.0082679Z  ---> 5c801f189956
2021-07-19T04:05:42.0083064Z Step 8/8 : CMD ["sh"]
2021-07-19T04:05:42.0370575Z  ---> Running in e90f067dd284
2021-07-19T04:05:43.0698853Z Removing intermediate container e90f067dd284
2021-07-19T04:05:43.0700091Z  ---> 8d9e828e7e9c
2021-07-19T04:05:43.0864284Z Successfully built 8d9e828e7e9c
2021-07-19T04:05:43.0903253Z Successfully tagged submariner-operator:devel
2021-07-19T04:05:43.6121037Z [submariner-operator]$ trap chown -R 1001:121 . exit
2021-07-19T04:05:43.6133835Z [submariner-operator]$ mkdir -p bin dist output
2021-07-19T04:05:43.6164441Z [submariner-operator]$ make --debug=b e2e
2021-07-19T04:05:43.6165042Z GNU Make 4.3
2021-07-19T04:05:43.6173263Z Built for x86_64-redhat-linux-gnu
2021-07-19T04:05:43.6258005Z Copyright (C) 1988-2020 Free Software Foundation, Inc.
2021-07-19T04:05:43.6259543Z License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
2021-07-19T04:05:43.6265923Z This is free software: you are free to change and redistribute it.
2021-07-19T04:05:43.6266664Z There is NO WARRANTY, to the extent permitted by law.
2021-07-19T04:05:43.6267250Z Reading makefiles...
2021-07-19T04:05:43.7231208Z Makefile:94: warning: overriding recipe for target 'e2e'
2021-07-19T04:05:43.7232352Z /opt/shipyard/Makefile.inc:106: warning: ignoring old recipe for target 'e2e'
2021-07-19T04:05:43.9154234Z Updating makefiles....
2021-07-19T04:05:43.9158767Z Updating goal targets....
2021-07-19T04:05:43.9160034Z  File 'e2e' does not exist.
2021-07-19T04:05:43.9161199Z      File 'clusters' does not exist.
2021-07-19T04:05:43.9166995Z     Must remake target 'clusters'.
2021-07-19T04:05:43.9168738Z /opt/shipyard/scripts/clusters.sh --k8s_version="1.19" --globalnet --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-19T04:05:43.9187015Z Downloading shflags 1.0.3
2021-07-19T04:05:43.9272326Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2021-07-19T04:05:43.9272979Z                                  Dload  Upload   Total   Spent    Left  Speed
2021-07-19T04:05:43.9273306Z 
2021-07-19T04:05:43.9726207Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2021-07-19T04:05:43.9726953Z 100 31091  100 31091    0     0   646k      0 --:--:-- --:--:-- --:--:--  646k
2021-07-19T04:05:44.0522190Z Running with: k8s_version=1.19.11, olm_version=0.14.1, olm=false, globalnet=true, registry_inmemory=true, cluster_settings=/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings, timeout=5m
2021-07-19T04:05:44.0554790Z [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
2021-07-19T04:05:44.0569408Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:05:44.0595709Z [submariner-operator]$ script_name=utils
2021-07-19T04:05:44.0610804Z [submariner-operator]$ exec_name=clusters.sh
2021-07-19T04:05:44.0633213Z [submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings
2021-07-19T04:05:44.0646805Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:05:44.0661395Z [submariner-operator]$ script_name=cluster_settings
2021-07-19T04:05:44.0672545Z [submariner-operator]$ exec_name=clusters.sh
2021-07-19T04:05:44.0697961Z [submariner-operator]$ broker=cluster1
2021-07-19T04:05:44.0710978Z [submariner-operator]$ declare -gA cluster_nodes
2021-07-19T04:05:44.0724624Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker
2021-07-19T04:05:44.0741339Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker
2021-07-19T04:05:44.0751486Z [submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker
2021-07-19T04:05:44.0765399Z [submariner-operator]$ declare -gA cluster_subm
2021-07-19T04:05:44.0779248Z [submariner-operator]$ cluster_subm[cluster1]=true
2021-07-19T04:05:44.0791833Z [submariner-operator]$ cluster_subm[cluster2]=true
2021-07-19T04:05:44.0804069Z [submariner-operator]$ cluster_subm[cluster3]=true
2021-07-19T04:05:44.0815872Z [submariner-operator]$ declare -gA cluster_cni
2021-07-19T04:05:44.0826069Z [submariner-operator]$ cluster_cni[cluster2]=weave
2021-07-19T04:05:44.0842511Z [submariner-operator]$ cluster_cni[cluster3]=weave
2021-07-19T04:05:44.0855269Z [submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-19T04:05:44.0866323Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:05:44.0880575Z [submariner-operator]$ script_name=cluster_settings
2021-07-19T04:05:44.0891194Z [submariner-operator]$ exec_name=clusters.sh
2021-07-19T04:05:44.0911914Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker
2021-07-19T04:05:44.0922778Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker
2021-07-19T04:05:44.0951244Z [submariner-operator]$ cat <<EOM
2021-07-19T04:05:44.0952412Z Cluster settings::
2021-07-19T04:05:44.0953039Z   broker - ${broker@Q}
2021-07-19T04:05:44.0953579Z   clusters - ${clusters[*]@Q}
2021-07-19T04:05:44.0954445Z   cni - $(typeset -p cluster_cni | cut -f 2- -d=)
2021-07-19T04:05:44.0955298Z   nodes per cluster - $(typeset -p cluster_nodes | cut -f 2- -d=)
2021-07-19T04:05:44.0956107Z   install submariner - $(typeset -p cluster_subm | cut -f 2- -d=)
2021-07-19T04:05:44.0956709Z EOM
2021-07-19T04:05:44.0976582Z [submariner-operator]$ typeset -p cluster_cni
2021-07-19T04:05:44.0989408Z [submariner-operator]$ cut -f 2- -d=
2021-07-19T04:05:44.1018750Z [submariner-operator]$ typeset -p cluster_nodes
2021-07-19T04:05:44.1034388Z [submariner-operator]$ cut -f 2- -d=
2021-07-19T04:05:44.1065548Z [submariner-operator]$ typeset -p cluster_subm
2021-07-19T04:05:44.1080844Z [submariner-operator]$ cut -f 2- -d=
2021-07-19T04:05:44.1102428Z Cluster settings::
2021-07-19T04:05:44.1103599Z   broker - 'cluster1'
2021-07-19T04:05:44.1104405Z   clusters - 'cluster1' 'cluster2'
2021-07-19T04:05:44.1105241Z   cni - ([cluster2]="weave" [cluster1]="weave" )
2021-07-19T04:05:44.1106448Z   nodes per cluster - ([cluster3]="control-plane worker worker" [cluster2]="control-plane worker worker" [cluster1]="control-plane worker worker" )
2021-07-19T04:05:44.1108105Z   install submariner - ([cluster2]="true" [cluster1]="true" )
2021-07-19T04:05:44.1118544Z [submariner-operator]$ rm -rf /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs
2021-07-19T04:05:44.1140837Z [submariner-operator]$ mkdir -p /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs
2021-07-19T04:05:44.1164698Z [submariner-operator]$ run_local_registry
2021-07-19T04:05:44.1176145Z [submariner-operator]$ run_local_registry
2021-07-19T04:05:44.1189361Z [submariner-operator]$ registry_running
2021-07-19T04:05:44.1201090Z [submariner-operator]$ registry_running
2021-07-19T04:05:44.1212489Z [submariner-operator]$ docker ps --filter name=^/?kind-registry$
2021-07-19T04:05:44.1228220Z [submariner-operator]$ grep kind-registry
2021-07-19T04:05:44.3757789Z [submariner-operator]$ return 0
2021-07-19T04:05:44.3763846Z Deploying local registry kind-registry to serve images centrally.
2021-07-19T04:05:44.3776776Z [submariner-operator]$ volume_flag=-v /dev/shm/kind-registry:/var/lib/registry
2021-07-19T04:05:44.3790500Z [submariner-operator]$ selinuxenabled
2021-07-19T04:05:44.3815401Z [submariner-operator]$ docker run -d -v /dev/shm/kind-registry:/var/lib/registry -p 127.0.0.1:5000:5000 --restart=always --name kind-registry registry:2
2021-07-19T04:05:44.6478622Z Unable to find image 'registry:2' locally
2021-07-19T04:05:45.3513402Z 2: Pulling from library/registry
2021-07-19T04:05:45.5481038Z ddad3d7c1e96: Already exists
2021-07-19T04:05:45.5501519Z 6eda6749503f: Pulling fs layer
2021-07-19T04:05:45.5502259Z 363ab70c2143: Pulling fs layer
2021-07-19T04:05:45.5510985Z 5b94580856e6: Pulling fs layer
2021-07-19T04:05:45.5511490Z 12008541203a: Pulling fs layer
2021-07-19T04:05:45.5512044Z 12008541203a: Waiting
2021-07-19T04:05:45.7812433Z 5b94580856e6: Verifying Checksum
2021-07-19T04:05:45.7813990Z 5b94580856e6: Download complete
2021-07-19T04:05:45.8042833Z 6eda6749503f: Verifying Checksum
2021-07-19T04:05:45.8043761Z 6eda6749503f: Download complete
2021-07-19T04:05:45.8912215Z 363ab70c2143: Verifying Checksum
2021-07-19T04:05:45.8913218Z 363ab70c2143: Download complete
2021-07-19T04:05:45.9640712Z 6eda6749503f: Pull complete
2021-07-19T04:05:46.0218125Z 12008541203a: Verifying Checksum
2021-07-19T04:05:46.0218680Z 12008541203a: Download complete
2021-07-19T04:05:46.3241375Z 363ab70c2143: Pull complete
2021-07-19T04:05:46.4007317Z 5b94580856e6: Pull complete
2021-07-19T04:05:46.4723669Z 12008541203a: Pull complete
2021-07-19T04:05:46.4785939Z Digest: sha256:aba2bfe9f0cff1ac0618ec4a54bfefb2e685bbac67c8ebaf3b6405929b3e616f
2021-07-19T04:05:46.4807493Z Status: Downloaded newer image for registry:2
2021-07-19T04:05:46.5060392Z 1b9f07d49f1a7b1b093ff87b968c60291457a7c4114a608a7054b19ab9d9e71b
2021-07-19T04:05:46.8635924Z [submariner-operator]$ docker network connect kind kind-registry
2021-07-19T04:05:47.2559729Z [submariner-operator]$ declare_cidrs
2021-07-19T04:05:47.2572736Z [submariner-operator]$ declare_cidrs
2021-07-19T04:05:47.2586148Z [submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
2021-07-19T04:05:47.2594140Z [submariner-operator]$ i=1
2021-07-19T04:05:47.2604477Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-19T04:05:47.2614500Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-19T04:05:47.2623430Z [submariner-operator]$ [cluster1] val=0
2021-07-19T04:05:47.2633442Z [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.0.0.0/16
2021-07-19T04:05:47.2642800Z [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.0.0.0/16
2021-07-19T04:05:47.2654583Z [submariner-operator]$ [cluster1] global_CIDRs[cluster1]=242.254.1.0/24
2021-07-19T04:05:47.2656113Z [submariner-operator]$ [cluster1] i=$(("$i"+1))
2021-07-19T04:05:47.2662228Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-19T04:05:47.2677965Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-19T04:05:47.2698315Z [submariner-operator]$ [cluster2] val=0
2021-07-19T04:05:47.2711384Z [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.0.0.0/16
2021-07-19T04:05:47.2727005Z [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.0.0.0/16
2021-07-19T04:05:47.2739920Z [submariner-operator]$ [cluster2] global_CIDRs[cluster2]=242.254.2.0/24
2021-07-19T04:05:47.2751546Z [submariner-operator]$ [cluster2] i=$(("$i"+1))
2021-07-19T04:05:47.2770128Z [submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster
2021-07-19T04:05:47.2774350Z [submariner-operator]$ [cluster2] wait 237
2021-07-19T04:05:47.2781778Z [submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster
2021-07-19T04:05:47.2794630Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 with_retries 3 create_kind_cluster
2021-07-19T04:05:47.2809238Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 with_retries 3 create_kind_cluster
2021-07-19T04:05:47.2827202Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-19T04:05:47.2844758Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-19T04:05:47.2867102Z [submariner-operator]$ [cluster1] pids[cluster1]=246
2021-07-19T04:05:47.2871951Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-19T04:05:47.2885734Z [submariner-operator]$ [cluster2] pids[cluster2]=249
2021-07-19T04:05:47.2895033Z [submariner-operator]$ [cluster1] with_retries 3 create_kind_cluster
2021-07-19T04:05:47.2902292Z [submariner-operator]$ [cluster2] wait 249
2021-07-19T04:05:47.2914754Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-19T04:05:47.2929694Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-19T04:05:47.2938214Z [submariner-operator]$ [cluster2] with_retries 3 create_kind_cluster
2021-07-19T04:05:47.2955552Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-19T04:07:54.8716871Z [submariner-operator]$ [cluster2] with_retries
2021-07-19T04:07:54.8717917Z [submariner-operator]$ [cluster2] iteration=1
2021-07-19T04:07:54.8718851Z [submariner-operator]$ [cluster2] create_kind_cluster
2021-07-19T04:07:54.8719743Z [submariner-operator]$ [cluster2] wait 274
2021-07-19T04:07:54.8720603Z [submariner-operator]$ [cluster2] create_kind_cluster
2021-07-19T04:07:54.8722608Z [submariner-operator]$ [cluster2] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:07:54.8725034Z [submariner-operator]$ [cluster2] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:07:54.8726786Z [submariner-operator]$ [cluster2] kind get clusters
2021-07-19T04:07:54.8727661Z [submariner-operator]$ [cluster2] grep -q ^cluster2$
2021-07-19T04:07:54.8728250Z [cluster2] No kind clusters found.
2021-07-19T04:07:54.8728747Z [cluster2] Creating KIND cluster...
2021-07-19T04:07:54.8729528Z [submariner-operator]$ [cluster2] generate_cluster_yaml
2021-07-19T04:07:54.8730454Z [submariner-operator]$ [cluster2] generate_cluster_yaml
2021-07-19T04:07:54.8731325Z [submariner-operator]$ [cluster2] disable_cni=true
2021-07-19T04:07:54.8732141Z [submariner-operator]$ [cluster2] nodes=
2021-07-19T04:07:54.8733472Z [cluster2] - role: control-plane
2021-07-19T04:07:54.8734206Z [submariner-operator]$ [cluster2] nodes=
2021-07-19T04:07:54.8734923Z [cluster2] - role: control-plane
2021-07-19T04:07:54.8735959Z [cluster2] - role: worker
2021-07-19T04:07:54.8736701Z [submariner-operator]$ [cluster2] nodes=
2021-07-19T04:07:54.8737768Z [cluster2] - role: control-plane
2021-07-19T04:07:54.8738355Z [cluster2] - role: worker
2021-07-19T04:07:54.8738891Z [cluster2] - role: worker
2021-07-19T04:07:54.8739407Z [cluster2] 
2021-07-19T04:07:54.8739866Z [cluster2] 
2021-07-19T04:07:54.8740565Z [submariner-operator]$ [cluster2] eval "echo \"$(cat $1)\""
2021-07-19T04:07:54.8741910Z [submariner-operator]$ [cluster2] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml
2021-07-19T04:07:54.8743639Z [submariner-operator]$ [cluster2] image_flag=--image=kindest/node:v1.19.11
2021-07-19T04:07:54.8744598Z [submariner-operator]$ [cluster2] kind version
2021-07-19T04:07:54.8745189Z [cluster2] kind v0.11.1 go1.16.4 linux/amd64
2021-07-19T04:07:54.8746356Z [submariner-operator]$ [cluster2] cat /opt/shipyard/scripts/resources/cluster2-config.yaml
2021-07-19T04:07:54.8747184Z [cluster2] ---
2021-07-19T04:07:54.8747575Z [cluster2] kind: Cluster
2021-07-19T04:07:54.8748213Z [cluster2] apiVersion: kind.x-k8s.io/v1alpha4
2021-07-19T04:07:54.8748729Z [cluster2] networking:
2021-07-19T04:07:54.8749235Z [cluster2]   disableDefaultCNI: true
2021-07-19T04:07:54.8749737Z [cluster2]   podSubnet: 10.0.0.0/16
2021-07-19T04:07:54.8750215Z [cluster2]   serviceSubnet: 100.0.0.0/16
2021-07-19T04:07:54.8750783Z [cluster2] containerdConfigPatches:
2021-07-19T04:07:54.8751438Z [cluster2]   - |-
2021-07-19T04:07:54.8752129Z [cluster2]     [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
2021-07-19T04:07:54.8753726Z [cluster2]       endpoint = ["http://kind-registry:5000"]
2021-07-19T04:07:54.8754414Z [cluster2] kubeadmConfigPatches:
2021-07-19T04:07:54.8755550Z [cluster2]   - |
2021-07-19T04:07:54.8756195Z [cluster2]     apiVersion: kubeadm.k8s.io/v1beta2
2021-07-19T04:07:54.8756795Z [cluster2]     kind: ClusterConfiguration
2021-07-19T04:07:54.8757279Z [cluster2]     metadata:
2021-07-19T04:07:54.8757684Z [cluster2]       name: config
2021-07-19T04:07:54.8758562Z [cluster2]     networking:
2021-07-19T04:07:54.8759162Z [cluster2]       podSubnet: 10.0.0.0/16
2021-07-19T04:07:54.8759644Z [cluster2]       serviceSubnet: 100.0.0.0/16
2021-07-19T04:07:54.8760310Z [cluster2]       dnsDomain: cluster2.local
2021-07-19T04:07:54.8760760Z [cluster2] nodes: 
2021-07-19T04:07:54.8761336Z [cluster2] - role: control-plane
2021-07-19T04:07:54.8761912Z [cluster2] - role: worker
2021-07-19T04:07:54.8762421Z [cluster2] - role: worker
2021-07-19T04:07:54.8764089Z [submariner-operator]$ [cluster2] kind create cluster --image=kindest/node:v1.19.11 --name=cluster2 --config=/opt/shipyard/scripts/resources/cluster2-config.yaml
2021-07-19T04:07:54.8765153Z [cluster2] Creating cluster "cluster2" ...
2021-07-19T04:07:54.8767310Z [cluster2]  • Ensuring node image (kindest/node:v1.19.11) 🖼  ...
2021-07-19T04:07:54.8768439Z [cluster2]  ✓ Ensuring node image (kindest/node:v1.19.11) 🖼
2021-07-19T04:07:54.8769207Z [cluster2]  • Preparing nodes 📦 📦 📦   ...
2021-07-19T04:07:54.8769845Z [cluster2]  ✓ Preparing nodes 📦 📦 📦 
2021-07-19T04:07:54.8770483Z [cluster2]  • Writing configuration 📜  ...
2021-07-19T04:07:54.8771153Z [cluster2]  ✓ Writing configuration 📜
2021-07-19T04:07:54.8771837Z [cluster2]  • Starting control-plane 🕹️  ...
2021-07-19T04:07:54.8772753Z [cluster2]  ✓ Starting control-plane 🕹️
2021-07-19T04:07:54.8773480Z [cluster2]  • Installing StorageClass 💾  ...
2021-07-19T04:07:54.8774417Z [cluster2]  ✓ Installing StorageClass 💾
2021-07-19T04:07:54.8775315Z [cluster2]  • Joining worker nodes 🚜  ...
2021-07-19T04:07:54.8776498Z [cluster2]  ✓ Joining worker nodes 🚜
2021-07-19T04:07:54.8777355Z [cluster2] Set kubectl context to "kind-cluster2"
2021-07-19T04:07:54.8777919Z [cluster2] You can now use your cluster with:
2021-07-19T04:07:54.8778349Z [cluster2] 
2021-07-19T04:07:54.8780048Z [cluster2] kubectl cluster-info --context kind-cluster2
2021-07-19T04:07:54.8780722Z [cluster2] 
2021-07-19T04:07:54.8781919Z [cluster2] Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
2021-07-19T04:07:54.8783382Z [submariner-operator]$ [cluster2] kind_fixup_config
2021-07-19T04:07:54.8784474Z [submariner-operator]$ [cluster2] kind_fixup_config
2021-07-19T04:07:54.8786539Z [submariner-operator]$ [cluster2] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster2-control-plane
2021-07-19T04:07:56.5635995Z [submariner-operator]$ [cluster2] head -n 1[submariner-operator]$ [cluster1] with_retries
2021-07-19T04:07:56.5637296Z [submariner-operator]$ [cluster1] iteration=1
2021-07-19T04:07:56.5638778Z [submariner-operator]$ [cluster1] wait 263
2021-07-19T04:07:56.5639674Z [submariner-operator]$ [cluster1] create_kind_cluster
2021-07-19T04:07:56.5640612Z [submariner-operator]$ [cluster1] create_kind_cluster
2021-07-19T04:07:56.5642172Z [submariner-operator]$ [cluster1] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:07:56.5644082Z [submariner-operator]$ [cluster1] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:07:56.5645416Z [submariner-operator]$ [cluster1] kind get clusters
2021-07-19T04:07:56.5646328Z [submariner-operator]$ [cluster1] grep -q ^cluster1$
2021-07-19T04:07:56.5647099Z [cluster1] No kind clusters found.
2021-07-19T04:07:56.5647616Z [cluster1] Creating KIND cluster...
2021-07-19T04:07:56.5648488Z [submariner-operator]$ [cluster1] generate_cluster_yaml
2021-07-19T04:07:56.5649444Z [submariner-operator]$ [cluster1] generate_cluster_yaml
2021-07-19T04:07:56.5650377Z [submariner-operator]$ [cluster1] disable_cni=true
2021-07-19T04:07:56.5651237Z [submariner-operator]$ [cluster1] nodes=
2021-07-19T04:07:56.5651998Z [cluster1] - role: control-plane
2021-07-19T04:07:56.5652766Z [submariner-operator]$ [cluster1] nodes=
2021-07-19T04:07:56.5653495Z [cluster1] - role: control-plane
2021-07-19T04:07:56.5654129Z [cluster1] - role: worker
2021-07-19T04:07:56.5654833Z [submariner-operator]$ [cluster1] nodes=
2021-07-19T04:07:56.5656786Z [cluster1] - role: control-plane
2021-07-19T04:07:56.5657493Z [cluster1] - role: worker
2021-07-19T04:07:56.5658304Z [cluster1] - role: worker
2021-07-19T04:07:56.5658850Z [cluster1] 
2021-07-19T04:07:56.5659525Z [cluster1] 
2021-07-19T04:07:56.5660474Z [submariner-operator]$ [cluster1] eval "echo \"$(cat $1)\""
2021-07-19T04:07:56.5661963Z [submariner-operator]$ [cluster1] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml
2021-07-19T04:07:56.5663185Z [submariner-operator]$ [cluster1] image_flag=--image=kindest/node:v1.19.11
2021-07-19T04:07:56.5664110Z [submariner-operator]$ [cluster1] kind version
2021-07-19T04:07:56.5665316Z [cluster1] kind v0.11.1 go1.16.4 linux/amd64
2021-07-19T04:07:56.5666439Z [submariner-operator]$ [cluster1] cat /opt/shipyard/scripts/resources/cluster1-config.yaml
2021-07-19T04:07:56.5667489Z [cluster1] ---
2021-07-19T04:07:56.5667886Z [cluster1] kind: Cluster
2021-07-19T04:07:56.5672286Z [cluster1] apiVersion: kind.x-k8s.io/v1alpha4
2021-07-19T04:07:56.5672903Z [cluster1] networking:
2021-07-19T04:07:56.5673502Z [cluster1]   disableDefaultCNI: true
2021-07-19T04:07:56.5674272Z [cluster1]   podSubnet: 10.0.0.0/16
2021-07-19T04:07:56.5674990Z [cluster1]   serviceSubnet: 100.0.0.0/16
2021-07-19T04:07:56.5675682Z [cluster1] containerdConfigPatches:
2021-07-19T04:07:56.5676439Z [cluster1]   - |-
2021-07-19T04:07:56.5678057Z [cluster1]     [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
2021-07-19T04:07:56.5679643Z [cluster1]       endpoint = ["http://kind-registry:5000"]
2021-07-19T04:07:56.5680438Z [cluster1] kubeadmConfigPatches:
2021-07-19T04:07:56.5681293Z [cluster1]   - |
2021-07-19T04:07:56.5681817Z [cluster1]     apiVersion: kubeadm.k8s.io/v1beta2
2021-07-19T04:07:56.5682454Z [cluster1]     kind: ClusterConfiguration
2021-07-19T04:07:56.5682994Z [cluster1]     metadata:
2021-07-19T04:07:56.5683419Z [cluster1]       name: config
2021-07-19T04:07:56.5683872Z [cluster1]     networking:
2021-07-19T04:07:56.5684344Z [cluster1]       podSubnet: 10.0.0.0/16
2021-07-19T04:07:56.5684840Z [cluster1]       serviceSubnet: 100.0.0.0/16
2021-07-19T04:07:56.5685781Z [cluster1]       dnsDomain: cluster1.local
2021-07-19T04:07:56.5686274Z [cluster1] nodes: 
2021-07-19T04:07:56.5706185Z [cluster1] - role: control-plane
2021-07-19T04:07:56.5706878Z [cluster1] - role: worker
2021-07-19T04:07:56.5707674Z [cluster1] - role: worker
2021-07-19T04:07:56.5709264Z [submariner-operator]$ [cluster1] kind create cluster --image=kindest/node:v1.19.11 --name=cluster1 --config=/opt/shipyard/scripts/resources/cluster1-config.yaml
2021-07-19T04:07:56.5710810Z [cluster1] Creating cluster "cluster1" ...
2021-07-19T04:07:56.5711785Z [cluster1]  • Ensuring node image (kindest/node:v1.19.11) 🖼  ...
2021-07-19T04:07:56.5712735Z [cluster1]  ✓ Ensuring node image (kindest/node:v1.19.11) 🖼
2021-07-19T04:07:56.5713631Z [cluster1]  • Preparing nodes 📦 📦 📦   ...
2021-07-19T04:07:56.5731584Z [cluster1]  ✓ Preparing nodes 📦 📦 📦 
2021-07-19T04:07:56.5732615Z [cluster1]  • Writing configuration 📜  ...
2021-07-19T04:07:56.5733410Z [cluster1]  ✓ Writing configuration 📜
2021-07-19T04:07:56.5734302Z [cluster1]  • Starting control-plane 🕹️  ...
2021-07-19T04:07:56.5735111Z [cluster1]  ✓ Starting control-plane 🕹️
2021-07-19T04:07:56.5736267Z [cluster1]  • Installing StorageClass 💾  ...
2021-07-19T04:07:56.5737241Z [cluster1]  ✓ Installing StorageClass 💾
2021-07-19T04:07:56.5738003Z [cluster1]  • Joining worker nodes 🚜  ...
2021-07-19T04:07:56.5738845Z [cluster1]  ✓ Joining worker nodes 🚜
2021-07-19T04:07:56.5739667Z [cluster1] Set kubectl context to "kind-cluster1"
2021-07-19T04:07:56.5740370Z [cluster1] You can now use your cluster with:
2021-07-19T04:07:56.5740855Z [cluster1] 
2021-07-19T04:07:56.5741631Z [cluster1] kubectl cluster-info --context kind-cluster1
2021-07-19T04:07:56.5742213Z [cluster1] 
2021-07-19T04:07:56.5743270Z [cluster1] Have a question, bug, or feature request? Let us know! https://kind.sigs.k8s.io/#community 🙂
2021-07-19T04:07:56.5744463Z [submariner-operator]$ [cluster1] kind_fixup_config
2021-07-19T04:07:56.5745445Z [submariner-operator]$ [cluster1] kind_fixup_config
2021-07-19T04:07:56.5746985Z [submariner-operator]$ [cluster1] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster1-control-plane
2021-07-19T04:08:32.0757968Z [submariner-operator]$ [cluster2] wait 246
2021-07-19T04:08:32.0758882Z [submariner-operator]$ [cluster1] head -n 1
2021-07-19T04:08:32.0761597Z [submariner-operator]$ [cluster2] sed -i -- s/server: .*/server: https:\/\/172.18.0.5:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:08:32.0764214Z [submariner-operator]$ [cluster2] sed -i -- s/user: kind-.*/user: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:08:32.0766710Z [submariner-operator]$ [cluster2] sed -i -- s/name: kind-.*/name: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:08:32.0769280Z [submariner-operator]$ [cluster2] sed -i -- s/cluster: kind-.*/cluster: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:08:32.0771764Z [submariner-operator]$ [cluster2] sed -i -- s/current-context: .*/current-context: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:08:32.0773939Z [submariner-operator]$ [cluster2] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-19T04:08:32.0775840Z [submariner-operator]$ [cluster2] wait 1520
2021-07-19T04:08:32.0777304Z [submariner-operator]$ [cluster2] deploy_cluster_capabilities
2021-07-19T04:08:32.0778602Z [submariner-operator]$ [cluster2] deploy_cluster_capabilities
2021-07-19T04:08:32.0780041Z [submariner-operator]$ [cluster2] deploy_cni
2021-07-19T04:08:32.0781893Z [submariner-operator]$ [cluster2] deploy_cni
2021-07-19T04:08:32.0783191Z [submariner-operator]$ [cluster2] eval deploy_weave_cni
2021-07-19T04:08:32.0784270Z [submariner-operator]$ [cluster2] deploy_weave_cni
2021-07-19T04:08:32.0785700Z [submariner-operator]$ [cluster2] deploy_weave_cni
2021-07-19T04:08:32.0786570Z [cluster2] Applying weave network...
2021-07-19T04:08:32.0790199Z [submariner-operator]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.19.11&env.IPALLOC_RANGE=10.0.0.0/16
2021-07-19T04:08:32.0792329Z [submariner-operator]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.19.11&env.IPALLOC_RANGE=10.0.0.0/16
2021-07-19T04:08:32.0795053Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.19.11&env.IPALLOC_RANGE=10.0.0.0/16
2021-07-19T04:08:32.0796768Z [cluster2] serviceaccount/weave-net created
2021-07-19T04:08:32.0798027Z [cluster2] clusterrole.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:32.0800157Z [cluster2] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:32.0801607Z [cluster2] role.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:32.0802747Z [cluster2] rolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:32.0803721Z [cluster2] daemonset.apps/weave-net created
2021-07-19T04:08:32.0804462Z [cluster2] Waiting for weave-net pods to be ready...
2021-07-19T04:08:32.0805518Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-19T04:08:32.0807260Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-19T04:08:32.0808846Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-19T04:08:32.0810721Z [cluster2] pod/weave-net-hdqg2 condition met
2021-07-19T04:08:32.0811515Z [cluster2] pod/weave-net-scdnz condition met
2021-07-19T04:08:32.0812232Z [cluster2] pod/weave-net-sv66l condition met
2021-07-19T04:08:32.0812994Z [cluster2] Waiting for core-dns deployment to be ready...
2021-07-19T04:08:32.0813988Z [submariner-operator]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-19T04:08:32.0815328Z [submariner-operator]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-19T04:08:32.0817643Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-19T04:08:32.0819168Z [cluster2] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
2021-07-19T04:08:32.0820562Z [cluster2] deployment "coredns" successfully rolled out
2021-07-19T04:08:32.0821640Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:08:34.0304318Z 
2021-07-19T04:08:34.0307168Z [submariner-operator]$ [cluster1] sed -i -- s/server: .*/server: https:\/\/172.18.0.7:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:08:34.0311408Z [submariner-operator]$ [cluster1] sed -i -- s/user: kind-.*/user: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:08:34.0313670Z [submariner-operator]$ [cluster1] sed -i -- s/name: kind-.*/name: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:08:34.0315716Z [submariner-operator]$ [cluster1] sed -i -- s/cluster: kind-.*/cluster: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:08:34.0318392Z [submariner-operator]$ [cluster1] sed -i -- s/current-context: .*/current-context: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:08:34.0322932Z [submariner-operator]$ [cluster1] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-19T04:08:34.0324552Z [submariner-operator]$ [cluster1] wait 1604
2021-07-19T04:08:34.0326111Z [submariner-operator]$ [cluster1] deploy_cluster_capabilities
2021-07-19T04:08:34.0327395Z [submariner-operator]$ [cluster1] deploy_cluster_capabilities
2021-07-19T04:08:34.0328417Z [submariner-operator]$ [cluster1] deploy_cni
2021-07-19T04:08:34.0329350Z [submariner-operator]$ [cluster1] deploy_cni
2021-07-19T04:08:34.0330669Z [submariner-operator]$ [cluster1] eval deploy_weave_cni
2021-07-19T04:08:34.0331672Z [submariner-operator]$ [cluster1] deploy_weave_cni
2021-07-19T04:08:34.0332633Z [submariner-operator]$ [cluster1] deploy_weave_cni
2021-07-19T04:08:34.0333786Z [cluster1] Applying weave network...
2021-07-19T04:08:34.0335110Z [submariner-operator]$ [cluster1] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.19.11&env.IPALLOC_RANGE=10.0.0.0/16
2021-07-19T04:08:34.0338193Z [submariner-operator]$ [cluster1] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.19.11&env.IPALLOC_RANGE=10.0.0.0/16
2021-07-19T04:08:34.0340261Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.19.11&env.IPALLOC_RANGE=10.0.0.0/16
2021-07-19T04:08:34.0341871Z [cluster1] serviceaccount/weave-net created
2021-07-19T04:08:34.0343247Z [cluster1] clusterrole.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:34.0346006Z [cluster1] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:34.0348148Z [cluster1] role.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:34.0350418Z [cluster1] rolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-19T04:08:34.0351931Z [cluster1] daemonset.apps/weave-net created
2021-07-19T04:08:34.0352976Z [cluster1] Waiting for weave-net pods to be ready...
2021-07-19T04:08:34.0354298Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-19T04:08:34.0355816Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-19T04:08:34.0357780Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-19T04:08:34.0359349Z [cluster1] pod/weave-net-frtsm condition met
2021-07-19T04:08:34.0360179Z [cluster1] pod/weave-net-qj58s condition met
2021-07-19T04:08:34.0360968Z [cluster1] pod/weave-net-rwj56 condition met
2021-07-19T04:08:34.0361803Z [cluster1] Waiting for core-dns deployment to be ready...
2021-07-19T04:08:34.0362902Z [submariner-operator]$ [cluster1] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-19T04:08:34.0364230Z [submariner-operator]$ [cluster1] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-19T04:08:34.0365651Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-19T04:08:34.0366825Z [cluster1] Waiting for deployment "coredns" rollout to finish: 0 of 2 updated replicas are available...
2021-07-19T04:08:34.0367839Z [cluster1] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
2021-07-19T04:08:34.0368684Z [cluster1] deployment "coredns" successfully rolled out
2021-07-19T04:08:34.0369569Z [submariner-operator]$ [cluster1] return 0
2021-07-19T04:08:34.0392911Z [submariner-operator]$ [cluster2] print_clusters_message
2021-07-19T04:08:34.0408969Z [submariner-operator]$ [cluster2] print_clusters_message
2021-07-19T04:08:34.0427884Z [submariner-operator]$ [cluster2] cat <<EOM
2021-07-19T04:08:34.0429169Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-19T04:08:34.0429725Z 
2021-07-19T04:08:34.0431074Z export KUBECONFIG=\$(find \$(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-19T04:08:34.0431653Z 
2021-07-19T04:08:34.0432352Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-19T04:08:34.0432793Z 
2021-07-19T04:08:34.0433425Z To clean everthing up, just run: make clean-clusters
2021-07-19T04:08:34.0434041Z EOM
2021-07-19T04:08:34.0465157Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-19T04:08:34.0465701Z 
2021-07-19T04:08:34.0466898Z export KUBECONFIG=$(find $(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-19T04:08:34.0467687Z 
2021-07-19T04:08:34.0468593Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-19T04:08:34.0469064Z 
2021-07-19T04:08:34.0469767Z To clean everthing up, just run: make clean-clusters
2021-07-19T04:08:34.0489356Z     Successfully remade target file 'clusters'.
2021-07-19T04:08:34.0492958Z      File 'preload-images' does not exist.
2021-07-19T04:08:34.0493690Z        File 'images' does not exist.
2021-07-19T04:08:34.0494604Z          File 'package/.image.submariner-operator' does not exist.
2021-07-19T04:08:34.0757869Z find: 'bin/submariner-operator': No such file or directory
2021-07-19T04:08:34.3830581Z              File 'bin/submariner-operator' does not exist.
2021-07-19T04:08:34.3831912Z                File 'vendor/modules.txt' does not exist.
2021-07-19T04:08:34.3832906Z               Must remake target 'vendor/modules.txt'.
2021-07-19T04:08:34.3833499Z go mod download
2021-07-19T04:09:38.0692165Z go mod vendor
2021-07-19T04:09:41.0975748Z go mod tidy
2021-07-19T04:09:41.5607651Z               Successfully remade target file 'vendor/modules.txt'.
2021-07-19T04:09:41.5627825Z                File 'generate-embeddedyamls' does not exist.
2021-07-19T04:09:41.5629009Z                  File 'generate' does not exist.
2021-07-19T04:09:41.5630960Z                    File '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen' does not exist.
2021-07-19T04:09:41.5633157Z                   Must remake target '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen'.
2021-07-19T04:09:41.5634624Z mkdir -p /go/src/github.com/submariner-io/submariner-operator/bin
2021-07-19T04:09:41.5690028Z go build -o /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen sigs.k8s.io/controller-tools/cmd/controller-gen
2021-07-19T04:10:03.9465787Z                   Successfully remade target file '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen'.
2021-07-19T04:10:03.9468733Z                 Must remake target 'generate'.
2021-07-19T04:10:03.9542171Z /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt,year=2021" paths="./..."
2021-07-19T04:10:16.2206633Z                 Successfully remade target file 'generate'.
2021-07-19T04:10:16.2208156Z                  File 'pkg/subctl/operator/common/embeddedyamls/yamls.go' does not exist.
2021-07-19T04:10:16.2216267Z                    File 'deploy/crds/submariner.io_servicediscoveries.yaml' does not exist.
2021-07-19T04:10:16.2225989Z                   Must remake target 'deploy/crds/submariner.io_servicediscoveries.yaml'.
2021-07-19T04:10:16.2227672Z /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen "crd:crdVersions=v1,trivialVersions=false" paths="./..." output:crd:artifacts:config=deploy/crds
2021-07-19T04:10:18.9608525Z                   Successfully remade target file 'deploy/crds/submariner.io_servicediscoveries.yaml'.
2021-07-19T04:10:18.9610025Z                    File 'deploy/submariner/crds/submariner.io_clusters.yaml' does not exist.
2021-07-19T04:10:18.9611155Z                   Must remake target 'deploy/submariner/crds/submariner.io_clusters.yaml'.
2021-07-19T04:10:18.9621098Z cd vendor/github.com/submariner-io/submariner && /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen "crd:crdVersions=v1,trivialVersions=false" paths="./..." output:crd:artifacts:config=../../../../deploy/submariner/crds
2021-07-19T04:10:20.4458194Z                   Successfully remade target file 'deploy/submariner/crds/submariner.io_clusters.yaml'.
2021-07-19T04:10:20.4565777Z                 Must remake target 'pkg/subctl/operator/common/embeddedyamls/yamls.go'.
2021-07-19T04:10:20.4566633Z go generate pkg/subctl/operator/common/embeddedyamls/generate.go
2021-07-19T04:10:21.0788857Z Generating yamls.go
2021-07-19T04:10:21.0795147Z deploy/crds/submariner.io_brokers.yaml
2021-07-19T04:10:21.0799088Z deploy/crds/submariner.io_submariners.yaml
2021-07-19T04:10:21.0829668Z deploy/crds/submariner.io_servicediscoveries.yaml
2021-07-19T04:10:21.0832921Z deploy/submariner/crds/submariner.io_clusters.yaml
2021-07-19T04:10:21.0837439Z deploy/submariner/crds/submariner.io_endpoints.yaml
2021-07-19T04:10:21.0841265Z deploy/submariner/crds/submariner.io_gateways.yaml
2021-07-19T04:10:21.0847459Z deploy/submariner/crds/submariner.io_clusterglobalegressips.yaml
2021-07-19T04:10:21.0862611Z deploy/submariner/crds/submariner.io_globalegressips.yaml
2021-07-19T04:10:21.0864158Z deploy/submariner/crds/submariner.io_globalingressips.yaml
2021-07-19T04:10:21.0873360Z deploy/mcsapi/crds/multicluster.x_k8s.io_serviceexports.yaml
2021-07-19T04:10:21.0880371Z deploy/mcsapi/crds/multicluster.x_k8s.io_serviceimports.yaml
2021-07-19T04:10:21.0890191Z config/broker/broker-admin/service_account.yaml
2021-07-19T04:10:21.0897741Z config/broker/broker-admin/role.yaml
2021-07-19T04:10:21.0901581Z config/broker/broker-admin/role_binding.yaml
2021-07-19T04:10:21.0907666Z config/broker/broker-client/service_account.yaml
2021-07-19T04:10:21.0911688Z config/broker/broker-client/role.yaml
2021-07-19T04:10:21.0917314Z config/broker/broker-client/role_binding.yaml
2021-07-19T04:10:21.0921145Z config/rbac/submariner-operator/service_account.yaml
2021-07-19T04:10:21.0927883Z config/rbac/submariner-operator/role.yaml
2021-07-19T04:10:21.0940877Z config/rbac/submariner-operator/role_binding.yaml
2021-07-19T04:10:21.0942269Z config/rbac/submariner-operator/cluster_role.yaml
2021-07-19T04:10:21.0950055Z config/rbac/submariner-operator/cluster_role_binding.yaml
2021-07-19T04:10:21.0953403Z config/rbac/submariner-gateway/service_account.yaml
2021-07-19T04:10:21.0962086Z config/rbac/submariner-gateway/role.yaml
2021-07-19T04:10:21.0969288Z config/rbac/submariner-gateway/role_binding.yaml
2021-07-19T04:10:21.0973354Z config/rbac/submariner-gateway/cluster_role.yaml
2021-07-19T04:10:21.0980482Z config/rbac/submariner-gateway/cluster_role_binding.yaml
2021-07-19T04:10:21.0986311Z config/rbac/submariner-route-agent/service_account.yaml
2021-07-19T04:10:21.0990652Z config/rbac/submariner-route-agent/role.yaml
2021-07-19T04:10:21.0997492Z config/rbac/submariner-route-agent/role_binding.yaml
2021-07-19T04:10:21.1001856Z config/rbac/submariner-route-agent/cluster_role.yaml
2021-07-19T04:10:21.1006132Z config/rbac/submariner-route-agent/cluster_role_binding.yaml
2021-07-19T04:10:21.1011163Z config/rbac/submariner-globalnet/service_account.yaml
2021-07-19T04:10:21.1034301Z config/rbac/submariner-globalnet/role.yaml
2021-07-19T04:10:21.1035362Z config/rbac/submariner-globalnet/role_binding.yaml
2021-07-19T04:10:21.1037923Z config/rbac/submariner-globalnet/cluster_role.yaml
2021-07-19T04:10:21.1038910Z config/rbac/submariner-globalnet/cluster_role_binding.yaml
2021-07-19T04:10:21.1040851Z config/rbac/lighthouse-agent/service_account.yaml
2021-07-19T04:10:21.1041955Z config/rbac/lighthouse-agent/cluster_role.yaml
2021-07-19T04:10:21.1042942Z config/rbac/lighthouse-agent/cluster_role_binding.yaml
2021-07-19T04:10:21.1045683Z config/rbac/lighthouse-coredns/service_account.yaml
2021-07-19T04:10:21.1052502Z config/rbac/lighthouse-coredns/cluster_role.yaml
2021-07-19T04:10:21.1059088Z config/rbac/lighthouse-coredns/cluster_role_binding.yaml
2021-07-19T04:10:21.1062489Z config/rbac/networkplugin_syncer/service_account.yaml
2021-07-19T04:10:21.1067846Z config/rbac/networkplugin_syncer/cluster_role.yaml
2021-07-19T04:10:21.1071881Z config/rbac/networkplugin_syncer/cluster_role_binding.yaml
2021-07-19T04:10:21.1079385Z config/rbac/submariner-metrics-reader/role.yaml
2021-07-19T04:10:21.1080733Z config/rbac/submariner-metrics-reader/role_binding.yaml
2021-07-19T04:10:21.1116996Z                 Successfully remade target file 'pkg/subctl/operator/common/embeddedyamls/yamls.go'.
2021-07-19T04:10:21.1118276Z               Must remake target 'generate-embeddedyamls'.
2021-07-19T04:10:21.1119533Z               Successfully remade target file 'generate-embeddedyamls'.
2021-07-19T04:10:21.1120643Z             Must remake target 'bin/submariner-operator'.
2021-07-19T04:10:21.1121461Z /opt/shipyard/scripts/compile.sh \
2021-07-19T04:10:21.1123083Z --ldflags "-X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d" \
2021-07-19T04:10:21.1124167Z bin/submariner-operator main.go 
2021-07-19T04:10:21.1776160Z [submariner-operator]$ mkdir -p bin
2021-07-19T04:10:21.1796996Z Building 'bin/submariner-operator' (ldflags: '-X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d')
2021-07-19T04:10:21.1808585Z [submariner-operator]$ ldflags=-s -w -X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d
2021-07-19T04:10:21.1823260Z [submariner-operator]$ CGO_ENABLED=0 go build -trimpath -ldflags -s -w -X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d -o bin/submariner-operator main.go
2021-07-19T04:12:03.8640155Z [submariner-operator]$ upx bin/submariner-operator
2021-07-19T04:12:11.4427695Z                        Ultimate Packer for eXecutables
2021-07-19T04:12:11.4429064Z                           Copyright (C) 1996 - 2020
2021-07-19T04:12:11.4429630Z UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020
2021-07-19T04:12:11.4430066Z 
2021-07-19T04:12:11.4430621Z         File size         Ratio      Format      Name
2021-07-19T04:12:11.4431334Z    --------------------   ------   -----------   -----------
2021-07-19T04:12:11.4432576Z   34459648 ->   9933112   28.83%   linux/amd64   submariner-operator
2021-07-19T04:12:11.4433134Z 
2021-07-19T04:12:11.4433858Z Packed 1 file.
2021-07-19T04:12:11.4442260Z             Successfully remade target file 'bin/submariner-operator'.
2021-07-19T04:12:11.4448034Z            Prerequisite 'bin/submariner-operator' is newer than target 'package/Dockerfile.submariner-operator'.
2021-07-19T04:12:11.4452491Z           Must remake target 'package/Dockerfile.submariner-operator'.
2021-07-19T04:12:11.4464404Z           Successfully remade target file 'package/Dockerfile.submariner-operator'.
2021-07-19T04:12:11.4467162Z            File 'FORCE_IMAGE' does not exist.
2021-07-19T04:12:11.4467863Z           Must remake target 'FORCE_IMAGE'.
2021-07-19T04:12:11.4468580Z           Successfully remade target file 'FORCE_IMAGE'.
2021-07-19T04:12:11.4469527Z         Must remake target 'package/.image.submariner-operator'.
2021-07-19T04:12:11.4471133Z /opt/shipyard/scripts/build_image.sh -i submariner-operator -f package/Dockerfile.submariner-operator -h package/.image.submariner-operator 
2021-07-19T04:12:11.4924476Z flags:WARN flag short name (h) already defined
2021-07-19T04:12:12.1320990Z [submariner-operator]$ set -e
2021-07-19T04:12:12.1322742Z [submariner-operator]$ cache_image=quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:12.1324084Z [submariner-operator]$ cache_flag=
2021-07-19T04:12:12.1325531Z [submariner-operator]$ cache_flag=--cache-from quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:12.1326849Z [submariner-operator]$ docker image ls -q quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:12.1328102Z [submariner-operator]$ docker pull quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:12.9698494Z devel: Pulling from submariner/submariner-operator
2021-07-19T04:12:12.9699318Z 9aabc6bb514c: Pulling fs layer
2021-07-19T04:12:13.4949611Z 9aabc6bb514c: Verifying Checksum
2021-07-19T04:12:13.4950316Z 9aabc6bb514c: Download complete
2021-07-19T04:12:13.8634081Z 9aabc6bb514c: Pull complete
2021-07-19T04:12:13.8711900Z Digest: sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b
2021-07-19T04:12:13.8752967Z Status: Downloaded newer image for quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:13.8793640Z quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:13.8818632Z [submariner-operator]$ grep FROM package/Dockerfile.submariner-operator
2021-07-19T04:12:13.8824317Z [submariner-operator]$ cut -f2 -d 
2021-07-19T04:12:13.8830781Z [submariner-operator]$ grep -v scratch
2021-07-19T04:12:13.8870815Z [submariner-operator]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel
2021-07-19T04:12:13.8883440Z 
2021-07-19T04:12:14.7014403Z [submariner-operator]$ docker buildx use buildx_builder
2021-07-19T04:12:15.4744714Z error: failed to find instance "buildx_builder": open /root/.docker/buildx/instances/buildx_builder: no such file or directory
2021-07-19T04:12:15.4812264Z [submariner-operator]$ docker buildx create --name buildx_builder --use
2021-07-19T04:12:16.2918216Z buildx_builder
2021-07-19T04:12:16.2988337Z [submariner-operator]$ docker buildx build --load -t quay.io/submariner/submariner-operator:dev --cache-from quay.io/submariner/submariner-operator:devel -f package/Dockerfile.submariner-operator --iidfile package/.image.submariner-operator --platform linux/amd64 --build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel .
2021-07-19T04:12:17.0790163Z #1 [internal] booting buildkit
2021-07-19T04:12:17.0791147Z #1 sha256:d4b3334d4f622686f338807d38d93de832f11dc82cb31c1f0f94c48c408e2786
2021-07-19T04:12:17.2292626Z #1 pulling image moby/buildkit:buildx-stable-1
2021-07-19T04:12:21.8485964Z #1 pulling image moby/buildkit:buildx-stable-1 4.8s done
2021-07-19T04:12:21.9987865Z #1 creating container buildx_buildkit_buildx_builder0
2021-07-19T04:12:22.4489577Z #1 creating container buildx_buildkit_buildx_builder0 0.5s done
2021-07-19T04:12:22.4490255Z #1 DONE 5.3s
2021-07-19T04:12:22.7189917Z 
2021-07-19T04:12:22.7191689Z #2 [internal] load build definition from Dockerfile.submariner-operator
2021-07-19T04:12:22.7200251Z #2 sha256:3ab1f154d3de7847382ebf0dcce5e41ef79fb97b81b6dbfc15e10a70ee95d257
2021-07-19T04:12:22.7201789Z #2 transferring dockerfile: 230B 0.0s done
2021-07-19T04:12:22.7202738Z #2 DONE 0.0s
2021-07-19T04:12:22.7203190Z 
2021-07-19T04:12:22.7203701Z #3 [internal] load .dockerignore
2021-07-19T04:12:22.7204734Z #3 sha256:10c6a886396d183da0bd7aeaae37ff172ad3159c41ab6fbb050765148e635ee4
2021-07-19T04:12:22.7206323Z #3 transferring context: 2B done
2021-07-19T04:12:22.7209708Z #3 DONE 0.0s
2021-07-19T04:12:22.7209957Z 
2021-07-19T04:12:22.7210962Z #6 importing cache manifest from quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:22.7212190Z #6 sha256:db6e513456d7d63fd9e86cca8d8a83f1eb0b2eaefc7f50d06048094c1b3ec182
2021-07-19T04:12:22.8714629Z #6 ...
2021-07-19T04:12:22.8715466Z 
2021-07-19T04:12:22.8716194Z #4 [internal] load build context
2021-07-19T04:12:22.8717310Z #4 sha256:bf86ed50e37a90f0aad99da00d0ad72dc298c7a0162f19ab87d2b562d99384ec
2021-07-19T04:12:22.8718623Z #4 DONE 0.0s
2021-07-19T04:12:23.0215134Z 
2021-07-19T04:12:23.0217785Z #6 importing cache manifest from quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:23.0218944Z #6 sha256:db6e513456d7d63fd9e86cca8d8a83f1eb0b2eaefc7f50d06048094c1b3ec182
2021-07-19T04:12:23.6156593Z #6 DONE 0.8s
2021-07-19T04:12:23.6156872Z 
2021-07-19T04:12:23.6157252Z #4 [internal] load build context
2021-07-19T04:12:23.6158447Z #4 sha256:bf86ed50e37a90f0aad99da00d0ad72dc298c7a0162f19ab87d2b562d99384ec
2021-07-19T04:12:23.9160494Z #4 transferring context: 9.94MB 0.3s done
2021-07-19T04:12:23.9161560Z #4 DONE 0.3s
2021-07-19T04:12:23.9161825Z 
2021-07-19T04:12:23.9163006Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-19T04:12:23.9164244Z #5 sha256:82ddde8d14a5b3a1b5d1793384b650cf4ee70c3a7988a22abd67bca506262b05
2021-07-19T04:12:23.9165066Z #5 CACHED
2021-07-19T04:12:23.9165284Z 
2021-07-19T04:12:23.9165638Z #7 exporting to oci image format
2021-07-19T04:12:23.9166509Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-19T04:12:23.9167577Z #7 exporting layers done
2021-07-19T04:12:23.9168635Z #7 exporting manifest sha256:de9bbdc497e579259a39507af2f25bbb63c3b0a7ac43576fa382f07e1dd99e1f done
2021-07-19T04:12:23.9170561Z #7 exporting config sha256:2d7e862e7c9a7cf6d0fff86d715882397631fcc6b1ba617b2d8a8d78322e83b7 done
2021-07-19T04:12:24.0663682Z #7 ...
2021-07-19T04:12:24.0664248Z 
2021-07-19T04:12:24.0665418Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-19T04:12:24.0666622Z #5 sha256:82ddde8d14a5b3a1b5d1793384b650cf4ee70c3a7988a22abd67bca506262b05
2021-07-19T04:12:24.0668513Z #5 sha256:9aabc6bb514c8021810110e5fb3bafd3f119cb23f253d128b1b53e2bf65b8a9a 0B / 9.62MB 0.2s
2021-07-19T04:12:24.0670061Z #5 CACHED
2021-07-19T04:12:24.2166836Z 
2021-07-19T04:12:24.2169967Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-19T04:12:24.2171654Z #5 sha256:82ddde8d14a5b3a1b5d1793384b650cf4ee70c3a7988a22abd67bca506262b05
2021-07-19T04:12:24.2172990Z #5 CACHED
2021-07-19T04:12:24.2173220Z 
2021-07-19T04:12:24.2173615Z #7 exporting to oci image format
2021-07-19T04:12:24.2174679Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-19T04:12:24.3676037Z #7 ...
2021-07-19T04:12:24.3676345Z 
2021-07-19T04:12:24.3677620Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-19T04:12:24.3679332Z #5 sha256:82ddde8d14a5b3a1b5d1793384b650cf4ee70c3a7988a22abd67bca506262b05
2021-07-19T04:12:24.3680782Z #5 sha256:9aabc6bb514c8021810110e5fb3bafd3f119cb23f253d128b1b53e2bf65b8a9a 9.62MB / 9.62MB 0.4s done
2021-07-19T04:12:24.3682159Z #5 CACHED
2021-07-19T04:12:24.3682372Z 
2021-07-19T04:12:24.3682720Z #7 exporting to oci image format
2021-07-19T04:12:24.3683748Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-19T04:12:24.3688793Z #7 sending tarball
2021-07-19T04:12:24.5797345Z #7 sending tarball 0.3s done
2021-07-19T04:12:24.5797847Z #7 DONE 0.8s
2021-07-19T04:12:24.5798057Z 
2021-07-19T04:12:24.5798418Z #9 exporting cache
2021-07-19T04:12:24.5801961Z #9 sha256:2700d4ef94dee473593c5c614b55b2dedcca7893909811a8f2b48291a1f581e4
2021-07-19T04:12:24.5802941Z #9 preparing build cache for export done
2021-07-19T04:12:24.5803442Z #9 DONE 0.0s
2021-07-19T04:12:24.5803655Z 
2021-07-19T04:12:24.5804020Z #8 importing to docker
2021-07-19T04:12:24.5805057Z #8 sha256:c9f0c3443ea43a504fee001dda67bb28e8d0358f569982e8ea938c87c3bb214e
2021-07-19T04:12:24.5805895Z #8 DONE 0.0s
2021-07-19T04:12:24.5929999Z [submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev quay.io/submariner/submariner-operator:devel
2021-07-19T04:12:24.8556393Z         Successfully remade target file 'package/.image.submariner-operator'.
2021-07-19T04:12:24.8558558Z          File 'build' does not exist.
2021-07-19T04:12:24.8561123Z            File 'bin/subctl' does not exist.
2021-07-19T04:12:24.8562115Z              File 'bin/subctl-devel-0105e4d-linux-amd64' does not exist.
2021-07-19T04:12:24.8825059Z             Must remake target 'bin/subctl-devel-0105e4d-linux-amd64'.
2021-07-19T04:12:24.8826369Z mkdir -p bin
2021-07-19T04:12:24.8851906Z target=bin/subctl-devel-0105e4d-linux-amd64; \
2021-07-19T04:12:24.8852602Z target=${target%.exe}; \
2021-07-19T04:12:24.8853645Z components=($(echo ${target//-/ })); \
2021-07-19T04:12:24.8854252Z GOOS=${components[-2]}; \
2021-07-19T04:12:24.8854823Z GOARCH=${components[-1]}; \
2021-07-19T04:12:24.8855239Z export GOARCH GOOS; \
2021-07-19T04:12:24.8856197Z /opt/shipyard/scripts/compile.sh \
2021-07-19T04:12:24.8857254Z 	--ldflags "-X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d \
2021-07-19T04:12:24.8859374Z 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=${DEFAULT_IMAGE_VERSION#v}" \
2021-07-19T04:12:24.8860972Z 	--noupx bin/subctl-devel-0105e4d-linux-amd64 ./pkg/subctl/main.go 
2021-07-19T04:12:24.9386776Z [submariner-operator]$ mkdir -p bin
2021-07-19T04:12:24.9404446Z Building 'bin/subctl-devel-0105e4d-linux-amd64' (ldflags: $'-X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d \t\t   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel')
2021-07-19T04:12:24.9424803Z [submariner-operator]$ ldflags=-s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel
2021-07-19T04:12:24.9441012Z [submariner-operator]$ CGO_ENABLED=0 go build -trimpath -ldflags -s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0105e4d 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel -o bin/subctl-devel-0105e4d-linux-amd64 ./pkg/subctl/main.go
2021-07-19T04:13:03.9757491Z             Successfully remade target file 'bin/subctl-devel-0105e4d-linux-amd64'.
2021-07-19T04:13:03.9761090Z           Must remake target 'bin/subctl'.
2021-07-19T04:13:03.9761958Z ln -sf subctl-devel-0105e4d-linux-amd64 bin/subctl
2021-07-19T04:13:03.9788315Z           Successfully remade target file 'bin/subctl'.
2021-07-19T04:13:03.9789612Z         Must remake target 'build'.
2021-07-19T04:13:03.9791887Z         Successfully remade target file 'build'.
2021-07-19T04:13:03.9792949Z       Must remake target 'images'.
2021-07-19T04:13:03.9793886Z       Successfully remade target file 'images'.
2021-07-19T04:13:03.9795007Z     Must remake target 'preload-images'.
2021-07-19T04:13:03.9795720Z set -e;\
2021-07-19T04:13:03.9796175Z . /opt/shipyard/scripts/lib/deploy_funcs;\
2021-07-19T04:13:03.9798143Z . /opt/shipyard/scripts/lib/debug_functions;\
2021-07-19T04:13:03.9800167Z for image in submariner-operator submariner-gateway submariner-route-agent lighthouse-agent lighthouse-coredns submariner-globalnet; do\
2021-07-19T04:13:03.9801382Z 	import_image quay.io/submariner/${image};\
2021-07-19T04:13:03.9801829Z done
2021-07-19T04:13:03.9852615Z [submariner-operator]$ import_image quay.io/submariner/submariner-operator
2021-07-19T04:13:03.9862806Z [submariner-operator]$ import_image quay.io/submariner/submariner-operator
2021-07-19T04:13:03.9876707Z [submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev localhost:5000/submariner-operator:local
2021-07-19T04:13:04.2675740Z [submariner-operator]$ docker push localhost:5000/submariner-operator:local
2021-07-19T04:13:04.5347276Z The push refers to repository [localhost:5000/submariner-operator]
2021-07-19T04:13:04.5781817Z f9269f9cee12: Preparing
2021-07-19T04:13:05.1568932Z f9269f9cee12: Pushed
2021-07-19T04:13:05.1795428Z local: digest: sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b size: 528
2021-07-19T04:13:05.1854276Z [submariner-operator]$ import_image quay.io/submariner/submariner-gateway
2021-07-19T04:13:05.1870795Z [submariner-operator]$ import_image quay.io/submariner/submariner-gateway
2021-07-19T04:13:05.1888074Z [submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:dev localhost:5000/submariner-gateway:local
2021-07-19T04:13:05.4476575Z Error response from daemon: No such image: quay.io/submariner/submariner-gateway:dev
2021-07-19T04:13:05.4508074Z [submariner-operator]$ docker pull quay.io/submariner/submariner-gateway:devel
2021-07-19T04:13:06.4937009Z devel: Pulling from submariner/submariner-gateway
2021-07-19T04:13:06.4937999Z 8c741e919b67: Pulling fs layer
2021-07-19T04:13:06.4939077Z 13495efbbac4: Pulling fs layer
2021-07-19T04:13:06.4939826Z 6df6df193f2d: Pulling fs layer
2021-07-19T04:13:06.4940335Z 3afbe9ee7387: Pulling fs layer
2021-07-19T04:13:06.4940889Z 3afbe9ee7387: Waiting
2021-07-19T04:13:06.7673886Z 13495efbbac4: Verifying Checksum
2021-07-19T04:13:06.7674397Z 13495efbbac4: Download complete
2021-07-19T04:13:07.1210417Z 6df6df193f2d: Verifying Checksum
2021-07-19T04:13:07.1211270Z 6df6df193f2d: Download complete
2021-07-19T04:13:07.2930147Z 3afbe9ee7387: Verifying Checksum
2021-07-19T04:13:07.2930817Z 3afbe9ee7387: Download complete
2021-07-19T04:13:07.4844820Z 8c741e919b67: Verifying Checksum
2021-07-19T04:13:07.4846046Z 8c741e919b67: Download complete
2021-07-19T04:13:10.5592145Z 8c741e919b67: Pull complete
2021-07-19T04:13:11.7364639Z 13495efbbac4: Pull complete
2021-07-19T04:13:12.9947045Z 6df6df193f2d: Pull complete
2021-07-19T04:13:13.1928580Z 3afbe9ee7387: Pull complete
2021-07-19T04:13:13.1981249Z Digest: sha256:16252ca036cdc08183e3368d617ba5b424b6b2c09a9103dce35a0044453bf895
2021-07-19T04:13:13.2006899Z Status: Downloaded newer image for quay.io/submariner/submariner-gateway:devel
2021-07-19T04:13:13.2047823Z quay.io/submariner/submariner-gateway:devel
2021-07-19T04:13:13.2099441Z [submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:devel quay.io/submariner/submariner-gateway:dev
2021-07-19T04:13:13.4691700Z [submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:dev localhost:5000/submariner-gateway:local
2021-07-19T04:13:13.7244140Z [submariner-operator]$ docker push localhost:5000/submariner-gateway:local
2021-07-19T04:13:13.9783792Z The push refers to repository [localhost:5000/submariner-gateway]
2021-07-19T04:13:13.9853343Z 71972a942ed4: Preparing
2021-07-19T04:13:13.9853796Z 3c7f2d60eacb: Preparing
2021-07-19T04:13:13.9854597Z 00c5f6897675: Preparing
2021-07-19T04:13:13.9855248Z b3b62c837788: Preparing
2021-07-19T04:13:14.5456976Z 00c5f6897675: Pushed
2021-07-19T04:13:14.9135847Z 71972a942ed4: Pushed
2021-07-19T04:13:18.6151695Z 3c7f2d60eacb: Pushed
2021-07-19T04:13:26.3805484Z b3b62c837788: Pushed
2021-07-19T04:13:26.4037516Z local: digest: sha256:16252ca036cdc08183e3368d617ba5b424b6b2c09a9103dce35a0044453bf895 size: 1159
2021-07-19T04:13:26.4101674Z [submariner-operator]$ import_image quay.io/submariner/submariner-route-agent
2021-07-19T04:13:26.4134605Z [submariner-operator]$ import_image quay.io/submariner/submariner-route-agent
2021-07-19T04:13:26.4154093Z [submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local
2021-07-19T04:13:26.6741508Z Error response from daemon: No such image: quay.io/submariner/submariner-route-agent:dev
2021-07-19T04:13:26.6837199Z [submariner-operator]$ docker pull quay.io/submariner/submariner-route-agent:devel
2021-07-19T04:13:27.6957156Z devel: Pulling from submariner/submariner-route-agent
2021-07-19T04:13:27.6958750Z 8c741e919b67: Already exists
2021-07-19T04:13:27.6966709Z 13495efbbac4: Already exists
2021-07-19T04:13:27.6995771Z 70cc6422093d: Pulling fs layer
2021-07-19T04:13:27.6996771Z 7ba74d821748: Pulling fs layer
2021-07-19T04:13:27.6998724Z 62b0802bcf90: Pulling fs layer
2021-07-19T04:13:27.6999297Z 3e9c83283ee4: Pulling fs layer
2021-07-19T04:13:27.6999742Z 3e9c83283ee4: Waiting
2021-07-19T04:13:27.9814784Z 62b0802bcf90: Verifying Checksum
2021-07-19T04:13:27.9815396Z 62b0802bcf90: Download complete
2021-07-19T04:13:28.2017108Z 7ba74d821748: Verifying Checksum
2021-07-19T04:13:28.2017708Z 7ba74d821748: Download complete
2021-07-19T04:13:28.2433942Z 3e9c83283ee4: Verifying Checksum
2021-07-19T04:13:28.2435856Z 3e9c83283ee4: Download complete
2021-07-19T04:13:28.3762867Z 70cc6422093d: Verifying Checksum
2021-07-19T04:13:28.3763416Z 70cc6422093d: Download complete
2021-07-19T04:13:29.8921589Z 70cc6422093d: Pull complete
2021-07-19T04:13:30.1051390Z 7ba74d821748: Pull complete
2021-07-19T04:13:30.1841104Z 62b0802bcf90: Pull complete
2021-07-19T04:13:30.2802458Z 3e9c83283ee4: Pull complete
2021-07-19T04:13:30.2865838Z Digest: sha256:1b9fb51c2e4e74ff6e1f02bef98d8998d772ad01e73ece5e247462590d8c8c4c
2021-07-19T04:13:30.2893617Z Status: Downloaded newer image for quay.io/submariner/submariner-route-agent:devel
2021-07-19T04:13:30.2925929Z quay.io/submariner/submariner-route-agent:devel
2021-07-19T04:13:30.2977710Z [submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:devel quay.io/submariner/submariner-route-agent:dev
2021-07-19T04:13:30.5620117Z [submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local
2021-07-19T04:13:30.8476193Z [submariner-operator]$ docker push localhost:5000/submariner-route-agent:local
2021-07-19T04:13:31.1092316Z The push refers to repository [localhost:5000/submariner-route-agent]
2021-07-19T04:13:31.1153971Z 1d810a4c0610: Preparing
2021-07-19T04:13:31.1154447Z 7a86b74623fc: Preparing
2021-07-19T04:13:31.1154873Z 26de773c4486: Preparing
2021-07-19T04:13:31.1155269Z 6931e591f8e2: Preparing
2021-07-19T04:13:31.1155703Z 00c5f6897675: Preparing
2021-07-19T04:13:31.1156099Z b3b62c837788: Preparing
2021-07-19T04:13:31.1171720Z b3b62c837788: Waiting
2021-07-19T04:13:31.5358024Z 00c5f6897675: Mounted from submariner-gateway
2021-07-19T04:13:31.5359111Z 1d810a4c0610: Pushed
2021-07-19T04:13:31.8197922Z 7a86b74623fc: Pushed
2021-07-19T04:13:31.8273942Z b3b62c837788: Mounted from submariner-gateway
2021-07-19T04:13:31.8275758Z 26de773c4486: Pushed
2021-07-19T04:13:35.7868374Z 6931e591f8e2: Pushed
2021-07-19T04:13:35.8087224Z local: digest: sha256:1b9fb51c2e4e74ff6e1f02bef98d8998d772ad01e73ece5e247462590d8c8c4c size: 1575
2021-07-19T04:13:35.8141496Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-agent
2021-07-19T04:13:35.8142842Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-agent
2021-07-19T04:13:35.8204903Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:dev localhost:5000/lighthouse-agent:local
2021-07-19T04:13:36.0901040Z Error response from daemon: No such image: quay.io/submariner/lighthouse-agent:dev
2021-07-19T04:13:36.0922649Z [submariner-operator]$ docker pull quay.io/submariner/lighthouse-agent:devel
2021-07-19T04:13:37.0991639Z devel: Pulling from submariner/lighthouse-agent
2021-07-19T04:13:37.0992324Z ecd69139dc76: Pulling fs layer
2021-07-19T04:13:37.0992874Z 8261c1767de3: Pulling fs layer
2021-07-19T04:13:37.3734532Z ecd69139dc76: Verifying Checksum
2021-07-19T04:13:37.3739741Z ecd69139dc76: Download complete
2021-07-19T04:13:37.4512961Z ecd69139dc76: Pull complete
2021-07-19T04:13:37.5984900Z 8261c1767de3: Verifying Checksum
2021-07-19T04:13:37.5985688Z 8261c1767de3: Download complete
2021-07-19T04:13:37.7805823Z 8261c1767de3: Pull complete
2021-07-19T04:13:37.7868817Z Digest: sha256:93957d5e6a2d72de099dcf9f589a4321618b426900e681d3837a17c64c38d69f
2021-07-19T04:13:37.7891553Z Status: Downloaded newer image for quay.io/submariner/lighthouse-agent:devel
2021-07-19T04:13:37.7980962Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:devel quay.io/submariner/lighthouse-agent:dev
2021-07-19T04:13:37.7982249Z quay.io/submariner/lighthouse-agent:devel
2021-07-19T04:13:38.0823790Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:dev localhost:5000/lighthouse-agent:local
2021-07-19T04:13:38.3442570Z [submariner-operator]$ docker push localhost:5000/lighthouse-agent:local
2021-07-19T04:13:38.6312361Z The push refers to repository [localhost:5000/lighthouse-agent]
2021-07-19T04:13:38.6358055Z 83c79e393e11: Preparing
2021-07-19T04:13:38.6358538Z 0026d2f62bbf: Preparing
2021-07-19T04:13:38.6781238Z 0026d2f62bbf: Pushed
2021-07-19T04:13:39.1701356Z 83c79e393e11: Pushed
2021-07-19T04:13:39.1901996Z local: digest: sha256:93957d5e6a2d72de099dcf9f589a4321618b426900e681d3837a17c64c38d69f size: 735
2021-07-19T04:13:39.1947588Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns
2021-07-19T04:13:39.1963254Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns
2021-07-19T04:13:39.1970407Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:dev localhost:5000/lighthouse-coredns:local
2021-07-19T04:13:39.4520745Z Error response from daemon: No such image: quay.io/submariner/lighthouse-coredns:dev
2021-07-19T04:13:39.4549053Z [submariner-operator]$ docker pull quay.io/submariner/lighthouse-coredns:devel
2021-07-19T04:13:40.5134130Z devel: Pulling from submariner/lighthouse-coredns
2021-07-19T04:13:40.5134894Z 088e56360f76: Pulling fs layer
2021-07-19T04:13:40.5135401Z 18aed70882d2: Pulling fs layer
2021-07-19T04:13:40.8396142Z 088e56360f76: Download complete
2021-07-19T04:13:40.9297772Z 088e56360f76: Pull complete
2021-07-19T04:13:41.0314511Z 18aed70882d2: Verifying Checksum
2021-07-19T04:13:41.0315327Z 18aed70882d2: Download complete
2021-07-19T04:13:41.2792126Z 18aed70882d2: Pull complete
2021-07-19T04:13:41.2846435Z Digest: sha256:d7e32908d5d6c680b813e0444db977c5b7578f7480fc650826c6dc993bbce595
2021-07-19T04:13:41.2870709Z Status: Downloaded newer image for quay.io/submariner/lighthouse-coredns:devel
2021-07-19T04:13:41.2939217Z quay.io/submariner/lighthouse-coredns:devel
2021-07-19T04:13:41.2978784Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:devel quay.io/submariner/lighthouse-coredns:dev
2021-07-19T04:13:41.5805008Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:dev localhost:5000/lighthouse-coredns:local
2021-07-19T04:13:41.8703748Z [submariner-operator]$ docker push localhost:5000/lighthouse-coredns:local
2021-07-19T04:13:42.1436270Z The push refers to repository [localhost:5000/lighthouse-coredns]
2021-07-19T04:13:42.1487864Z 7fca351e51be: Preparing
2021-07-19T04:13:42.1488393Z 4fa0ece06e25: Preparing
2021-07-19T04:13:42.2232998Z 4fa0ece06e25: Pushed
2021-07-19T04:13:42.9309511Z 7fca351e51be: Pushed
2021-07-19T04:13:42.9675088Z local: digest: sha256:d7e32908d5d6c680b813e0444db977c5b7578f7480fc650826c6dc993bbce595 size: 739
2021-07-19T04:13:42.9714229Z [submariner-operator]$ import_image quay.io/submariner/submariner-globalnet
2021-07-19T04:13:42.9730784Z [submariner-operator]$ import_image quay.io/submariner/submariner-globalnet
2021-07-19T04:13:42.9742501Z [submariner-operator]$ docker tag quay.io/submariner/submariner-globalnet:dev localhost:5000/submariner-globalnet:local
2021-07-19T04:13:43.2337599Z Error response from daemon: No such image: quay.io/submariner/submariner-globalnet:dev
2021-07-19T04:13:43.2361922Z [submariner-operator]$ docker pull quay.io/submariner/submariner-globalnet:devel
2021-07-19T04:13:44.2462651Z devel: Pulling from submariner/submariner-globalnet
2021-07-19T04:13:44.2463820Z 8c741e919b67: Already exists
2021-07-19T04:13:44.2477448Z 13495efbbac4: Already exists
2021-07-19T04:13:44.2506890Z bb77c1f3f2df: Pulling fs layer
2021-07-19T04:13:44.2507533Z 8d0ffdb5918e: Pulling fs layer
2021-07-19T04:13:44.2508077Z bcf8fc53d486: Pulling fs layer
2021-07-19T04:13:44.2508643Z 218c178ba1a9: Pulling fs layer
2021-07-19T04:13:44.2509108Z 218c178ba1a9: Waiting
2021-07-19T04:13:44.5155983Z bcf8fc53d486: Verifying Checksum
2021-07-19T04:13:44.5156614Z bcf8fc53d486: Download complete
2021-07-19T04:13:44.7533997Z bb77c1f3f2df: Verifying Checksum
2021-07-19T04:13:44.7534568Z bb77c1f3f2df: Download complete
2021-07-19T04:13:44.7832186Z 8d0ffdb5918e: Verifying Checksum
2021-07-19T04:13:44.7832798Z 8d0ffdb5918e: Download complete
2021-07-19T04:13:44.7836490Z 218c178ba1a9: Verifying Checksum
2021-07-19T04:13:44.7840005Z 218c178ba1a9: Download complete
2021-07-19T04:13:45.3396746Z bb77c1f3f2df: Pull complete
2021-07-19T04:13:45.5513775Z 8d0ffdb5918e: Pull complete
2021-07-19T04:13:45.6306386Z bcf8fc53d486: Pull complete
2021-07-19T04:13:45.7260017Z 218c178ba1a9: Pull complete
2021-07-19T04:13:45.7326483Z Digest: sha256:623e38998e0624d9a74f0c3638833e968ba472ef2ae235b724a67de11f113a69
2021-07-19T04:13:45.7352297Z Status: Downloaded newer image for quay.io/submariner/submariner-globalnet:devel
2021-07-19T04:13:45.7418984Z quay.io/submariner/submariner-globalnet:devel
2021-07-19T04:13:45.7457989Z [submariner-operator]$ docker tag quay.io/submariner/submariner-globalnet:devel quay.io/submariner/submariner-globalnet:dev
2021-07-19T04:13:46.0153141Z [submariner-operator]$ docker tag quay.io/submariner/submariner-globalnet:dev localhost:5000/submariner-globalnet:local
2021-07-19T04:13:46.2898936Z [submariner-operator]$ docker push localhost:5000/submariner-globalnet:local
2021-07-19T04:13:46.5681469Z The push refers to repository [localhost:5000/submariner-globalnet]
2021-07-19T04:13:46.5685000Z a395ed568a2f: Preparing
2021-07-19T04:13:46.5686540Z 39ac285976a0: Preparing
2021-07-19T04:13:46.5687910Z de48a84fa0cb: Preparing
2021-07-19T04:13:46.5688422Z 70c3bf07b5e8: Preparing
2021-07-19T04:13:46.5692603Z 00c5f6897675: Preparing
2021-07-19T04:13:46.5697941Z b3b62c837788: Preparing
2021-07-19T04:13:46.5704141Z b3b62c837788: Waiting
2021-07-19T04:13:46.9553252Z 00c5f6897675: Mounted from submariner-route-agent
2021-07-19T04:13:47.2913849Z a395ed568a2f: Pushed
2021-07-19T04:13:47.2923664Z 39ac285976a0: Pushed
2021-07-19T04:13:47.2933630Z b3b62c837788: Mounted from submariner-route-agent
2021-07-19T04:13:47.3024057Z de48a84fa0cb: Pushed
2021-07-19T04:13:48.2106808Z 70c3bf07b5e8: Pushed
2021-07-19T04:13:48.2326772Z local: digest: sha256:623e38998e0624d9a74f0c3638833e968ba472ef2ae235b724a67de11f113a69 size: 1574
2021-07-19T04:13:48.2361449Z     Successfully remade target file 'preload-images'.
2021-07-19T04:13:48.2367033Z    Prerequisite 'clusters' of target 'deploy' does not exist.
2021-07-19T04:13:48.2368000Z    Prerequisite 'preload-images' of target 'deploy' does not exist.
2021-07-19T04:13:48.2368876Z    Prerequisite 'bin/subctl' is newer than target 'deploy'.
2021-07-19T04:13:48.2370399Z   Must remake target 'deploy'.
2021-07-19T04:13:48.2372721Z /opt/shipyard/scripts/deploy.sh --globalnet --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-19T04:13:48.3333546Z Running with: globalnet='true', deploytool='operator', deploytool_broker_args='', deploytool_submariner_args='', cluster_settings='/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings', timeout=5m, image_tag=local, cable_driver=libreswan, service_discovery=false
2021-07-19T04:13:48.3360734Z [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
2021-07-19T04:13:48.3383864Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:13:48.3395879Z [submariner-operator]$ script_name=utils
2021-07-19T04:13:48.3409496Z [submariner-operator]$ exec_name=deploy.sh
2021-07-19T04:13:48.3435589Z [submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs
2021-07-19T04:13:48.3451101Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:13:48.3466617Z [submariner-operator]$ script_name=deploy_funcs
2021-07-19T04:13:48.3479746Z [submariner-operator]$ exec_name=deploy.sh
2021-07-19T04:13:48.3498060Z [submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings
2021-07-19T04:13:48.3514056Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:13:48.3527323Z [submariner-operator]$ script_name=cluster_settings
2021-07-19T04:13:48.3539475Z [submariner-operator]$ exec_name=deploy.sh
2021-07-19T04:13:48.3576074Z [submariner-operator]$ broker=cluster1
2021-07-19T04:13:48.3592486Z [submariner-operator]$ declare -gA cluster_nodes
2021-07-19T04:13:48.3608969Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker
2021-07-19T04:13:48.3622176Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker
2021-07-19T04:13:48.3636077Z [submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker
2021-07-19T04:13:48.3649365Z [submariner-operator]$ declare -gA cluster_subm
2021-07-19T04:13:48.3662507Z [submariner-operator]$ cluster_subm[cluster1]=true
2021-07-19T04:13:48.3674608Z [submariner-operator]$ cluster_subm[cluster2]=true
2021-07-19T04:13:48.3688474Z [submariner-operator]$ cluster_subm[cluster3]=true
2021-07-19T04:13:48.3702040Z [submariner-operator]$ declare -gA cluster_cni
2021-07-19T04:13:48.3715918Z [submariner-operator]$ cluster_cni[cluster2]=weave
2021-07-19T04:13:48.3733698Z [submariner-operator]$ cluster_cni[cluster3]=weave
2021-07-19T04:13:48.3750944Z [submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-19T04:13:48.3763878Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:13:48.3779069Z [submariner-operator]$ script_name=cluster_settings
2021-07-19T04:13:48.3789066Z [submariner-operator]$ exec_name=deploy.sh
2021-07-19T04:13:48.3809657Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker
2021-07-19T04:13:48.3820261Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker
2021-07-19T04:13:48.3849012Z [submariner-operator]$ declare_cidrs
2021-07-19T04:13:48.3862302Z [submariner-operator]$ declare_cidrs
2021-07-19T04:13:48.3873393Z [submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
2021-07-19T04:13:48.3884454Z [submariner-operator]$ i=1
2021-07-19T04:13:48.3899023Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-19T04:13:48.3910049Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-19T04:13:48.3920983Z [submariner-operator]$ [cluster1] val=0
2021-07-19T04:13:48.3933581Z [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.0.0.0/16
2021-07-19T04:13:48.3942355Z [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.0.0.0/16
2021-07-19T04:13:48.3963285Z [submariner-operator]$ [cluster1] global_CIDRs[cluster1]=242.254.1.0/24
2021-07-19T04:13:48.3980636Z [submariner-operator]$ [cluster1] i=$(("$i"+1))
2021-07-19T04:13:48.3997482Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-19T04:13:48.4011415Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-19T04:13:48.4023743Z [submariner-operator]$ [cluster2] val=0
2021-07-19T04:13:48.4034991Z [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.0.0.0/16
2021-07-19T04:13:48.4052534Z [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.0.0.0/16
2021-07-19T04:13:48.4064405Z [submariner-operator]$ [cluster2] global_CIDRs[cluster2]=242.254.2.0/24
2021-07-19T04:13:48.4079251Z [submariner-operator]$ [cluster2] i=$(("$i"+1))
2021-07-19T04:13:48.4092916Z [submariner-operator]$ [cluster2] declare_kubeconfig
2021-07-19T04:13:48.4122681Z [submariner-operator]$ [cluster2] declare_kubeconfig
2021-07-19T04:13:48.4137575Z [submariner-operator]$ [cluster2] source /opt/shipyard/scripts/lib/kubecfg
2021-07-19T04:13:48.4155845Z [submariner-operator]$ [cluster2] export KUBECONFIG
2021-07-19T04:13:48.4174493Z [submariner-operator]$ [cluster2] KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)
2021-07-19T04:13:48.4195096Z [submariner-operator]$ [cluster2] find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:
2021-07-19T04:13:48.4229830Z [submariner-operator]$ [cluster2] bash -c curl -Ls https://get.submariner.io | VERSION=devel DESTDIR=/go/bin bash
2021-07-19T04:13:48.6048281Z Installing subctl version devel
2021-07-19T04:13:48.6049016Z   OS detected:           linux
2021-07-19T04:13:48.6053601Z   Architecture detected: amd64
2021-07-19T04:13:48.6058804Z   Download URL:          https://github.com/submariner-io/submariner-operator/releases/download/subctl-devel/subctl-devel-linux-amd64.tar.xz
2021-07-19T04:13:48.6060256Z 
2021-07-19T04:13:48.6064375Z Downloading...
2021-07-19T04:13:49.7570756Z subctl-devel-linux-amd64 has been installed as /go/bin/subctl
2021-07-19T04:13:49.7807931Z This provides subctl version: devel
2021-07-19T04:13:49.7875951Z 
2021-07-19T04:13:49.7883963Z please update your path (and consider adding it to ~/.profile):
2021-07-19T04:13:49.7884592Z     export PATH=$PATH:/go/bin
2021-07-19T04:13:49.7891945Z     
2021-07-19T04:13:49.8016527Z [submariner-operator]$ [cluster2] load_deploytool operator
2021-07-19T04:13:49.8034964Z [submariner-operator]$ [cluster2] load_deploytool operator
2021-07-19T04:13:49.8044518Z Will deploy submariner using operator
2021-07-19T04:13:49.8060312Z [submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/deploy_operator
2021-07-19T04:13:49.8079682Z [submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:13:49.8094074Z [submariner-operator]$ [cluster2] script_name=deploy_operator
2021-07-19T04:13:49.8111698Z [submariner-operator]$ [cluster2] exec_name=deploy.sh
2021-07-19T04:13:49.8134019Z [submariner-operator]$ [cluster2] deploytool_prereqs
2021-07-19T04:13:49.8147940Z [submariner-operator]$ [cluster2] deploytool_prereqs
2021-07-19T04:13:49.8160647Z 
2021-07-19T04:13:49.8174004Z [submariner-operator]$ [cluster2] subctl version
2021-07-19T04:13:49.8374907Z subctl version: devel-0105e4d
2021-07-19T04:13:49.8397277Z [submariner-operator]$ [cluster2] run_if_defined pre_deploy
2021-07-19T04:13:49.8412077Z [submariner-operator]$ [cluster2] run_if_defined pre_deploy
2021-07-19T04:13:49.8426163Z [submariner-operator]$ [cluster2] type -t pre_deploy
2021-07-19T04:13:49.8440456Z [submariner-operator]$ [cluster2] grep -q function
2021-07-19T04:13:49.8468982Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:13:49.8479644Z [submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster
2021-07-19T04:13:49.8493933Z [submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster
2021-07-19T04:13:49.8506176Z [submariner-operator]$ [cluster2] declare -a subm_clusters
2021-07-19T04:13:49.8542110Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 prepare_cluster
2021-07-19T04:13:49.8557468Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 prepare_cluster
2021-07-19T04:13:49.8574023Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-19T04:13:49.8594013Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-19T04:13:49.8620978Z [submariner-operator]$ [cluster1] pids[cluster1]=7247
2021-07-19T04:13:49.8637898Z [submariner-operator]$ [cluster2] pids[cluster2]=7249
2021-07-19T04:13:49.8671118Z [submariner-operator]$ [cluster2] wait 7249
2021-07-19T04:13:49.8674438Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-19T04:13:49.8686017Z [submariner-operator]$ [cluster2] prepare_cluster
2021-07-19T04:13:49.8700470Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-19T04:13:49.8709399Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-19T04:13:49.8740733Z [submariner-operator]$ [cluster1] prepare_cluster
2021-07-19T04:13:49.8767391Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-19T04:13:50.2958241Z [submariner-operator]$ [cluster1] prepare_cluster
2021-07-19T04:13:50.2960643Z [submariner-operator]$ [cluster1] add_subm_gateway_label
2021-07-19T04:13:50.2961913Z [submariner-operator]$ [cluster1] add_subm_gateway_label
2021-07-19T04:13:50.2963516Z [submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
2021-07-19T04:13:50.2965083Z [submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
2021-07-19T04:13:50.2967068Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 label node cluster1-worker submariner.io/gateway=true --overwrite
2021-07-19T04:13:50.2968513Z [cluster1] node/cluster1-worker labeled
2021-07-19T04:13:50.3062607Z [submariner-operator]$ [cluster2] prepare_cluster
2021-07-19T04:13:50.3063723Z [submariner-operator]$ [cluster2] add_subm_gateway_label
2021-07-19T04:13:50.3064914Z [submariner-operator]$ [cluster2] add_subm_gateway_label
2021-07-19T04:13:50.3066249Z [submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
2021-07-19T04:13:50.3067730Z [submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
2021-07-19T04:13:50.3069524Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 label node cluster2-worker submariner.io/gateway=true --overwrite
2021-07-19T04:13:50.3070724Z [cluster2] node/cluster2-worker labeled
2021-07-19T04:13:50.3104038Z [submariner-operator]$ [cluster2] wait 7247
2021-07-19T04:13:50.3115695Z [submariner-operator]$ [cluster2] with_context cluster1 setup_broker
2021-07-19T04:13:50.3126587Z [submariner-operator]$ [cluster2] with_context cluster1 setup_broker
2021-07-19T04:13:50.3140652Z [submariner-operator]$ [cluster1] setup_broker
2021-07-19T04:13:50.3154337Z [submariner-operator]$ [cluster1] setup_broker
2021-07-19T04:13:50.3173061Z [submariner-operator]$ [cluster1] gn=--globalnet
2021-07-19T04:13:50.3180832Z Installing broker...
2021-07-19T04:13:50.3193697Z [submariner-operator]$ [cluster1] subctlrepver=
2021-07-19T04:13:50.3208458Z [submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local
2021-07-19T04:13:50.3227297Z [submariner-operator]$ [cluster1] cd /go/src/github.com/submariner-io/submariner-operator/output
2021-07-19T04:13:50.3240087Z [output]$ [cluster1] subctl deploy-broker --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --repository localhost:5000 --version local --globalnet
2021-07-19T04:13:50.3487013Z  • Setting up broker RBAC  ...
2021-07-19T04:13:50.4495165Z  ✓ Setting up broker RBAC
2021-07-19T04:13:50.4497121Z  • Deploying the Submariner operator  ...
2021-07-19T04:14:01.7592005Z  ✓ Deploying the Submariner operator
2021-07-19T04:14:01.7593050Z  ✓ Created operator CRDs
2021-07-19T04:14:01.7596483Z  ✓ Created operator namespace: submariner-operator
2021-07-19T04:14:01.7597792Z  ✓ Created operator service account and role
2021-07-19T04:14:01.7598719Z  ✓ Created lighthouse service account and role
2021-07-19T04:14:01.7599493Z  ✓ Created Lighthouse service accounts and roles
2021-07-19T04:14:01.7600320Z  ✓ Deployed the operator successfully
2021-07-19T04:14:01.7604412Z  • Deploying the broker  ...
2021-07-19T04:14:01.8137575Z  ✓ Deploying the broker
2021-07-19T04:14:01.8138600Z  ✓ The broker has been deployed
2021-07-19T04:14:01.8139313Z  • Creating broker-info.subm file  ...
2021-07-19T04:14:01.8311162Z  ✓ Creating broker-info.subm file
2021-07-19T04:14:01.8312123Z  ✓ A new IPsec PSK will be generated for broker-info.subm
2021-07-19T04:14:01.8362792Z [submariner-operator]$ [cluster2] install_subm_all_clusters
2021-07-19T04:14:01.8376713Z [submariner-operator]$ [cluster2] install_subm_all_clusters
2021-07-19T04:14:01.8389115Z [submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm
2021-07-19T04:14:01.8400269Z [submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm
2021-07-19T04:14:01.8415773Z [submariner-operator]$ [cluster2] declare -a subm_clusters
2021-07-19T04:14:01.8447405Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 subctl_install_subm
2021-07-19T04:14:01.8459634Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 subctl_install_subm
2021-07-19T04:14:01.8472572Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-19T04:14:01.8489069Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-19T04:14:01.8511913Z [submariner-operator]$ [cluster1] pids[cluster1]=7312
2021-07-19T04:14:01.8529678Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-19T04:14:01.8535491Z [submariner-operator]$ [cluster2] pids[cluster2]=7315
2021-07-19T04:14:01.8553122Z [submariner-operator]$ [cluster1] subctl_install_subm
2021-07-19T04:14:01.8563224Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-19T04:14:01.8582828Z [submariner-operator]$ [cluster2] wait 7315
2021-07-19T04:14:01.8584971Z [submariner-operator]$ [cluster2] subctl_install_subm
2021-07-19T04:14:01.8596623Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-19T04:14:01.8661962Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-19T04:14:09.0991107Z [submariner-operator]$ [cluster1] subctl_install_subm
2021-07-19T04:14:09.0992947Z [submariner-operator]$ [cluster1] subctlrepver=
2021-07-19T04:14:09.0994766Z [submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local
2021-07-19T04:14:09.0996197Z [submariner-operator]$ [cluster1] PASS_CIDR_ARGS=false
2021-07-19T04:14:09.0998994Z [submariner-operator]$ [cluster1] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --clusterid cluster1 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr 242.254.1.0/24 --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm
2021-07-19T04:14:09.1002389Z [cluster1] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.7:6443
2021-07-19T04:14:09.1003702Z [cluster1] * There are 1 labeled nodes in the cluster:
2021-07-19T04:14:09.1004555Z [cluster1]   - cluster1-worker
2021-07-19T04:14:09.1005415Z [cluster1]  • Discovering network details  ...
2021-07-19T04:14:09.1006315Z [cluster1]         Network plugin:  weave-net
2021-07-19T04:14:09.1007192Z [cluster1]         Service CIDRs:   [100.0.0.0/16]
2021-07-19T04:14:09.1007800Z [cluster1]         Cluster CIDRs:   [10.0.0.0/16]
2021-07-19T04:14:09.1009135Z [cluster1]  ✓ Discovering network details
2021-07-19T04:14:09.1010940Z [cluster1]  • Discovering multi cluster details  ...
2021-07-19T04:14:09.1012331Z [cluster1]  • Validating Globalnet configurations  ...
2021-07-19T04:14:09.1014055Z [cluster1]  ✓ Validating Globalnet configurations
2021-07-19T04:14:09.1015047Z [cluster1]  • Assigning Globalnet IPs  ...
2021-07-19T04:14:09.1016585Z [cluster1]  ✓ Assigning Globalnet IPs
2021-07-19T04:14:09.1017653Z [cluster1]  ✓ GlobalCIDR is: 242.254.1.0/24
2021-07-19T04:14:09.1018788Z [cluster1]  ✓ Discovering multi cluster details
2021-07-19T04:14:09.1020591Z [cluster1]  • Deploying the Submariner operator  ...
2021-07-19T04:14:09.1024533Z [cluster1]  ✓ Deploying the Submariner operator
2021-07-19T04:14:09.1025624Z [cluster1]  ✓ Created Lighthouse service accounts and roles
2021-07-19T04:14:09.1026482Z [cluster1]  • Creating SA for cluster  ...
2021-07-19T04:14:09.1027582Z [cluster1]  ✓ Creating SA for cluster
2021-07-19T04:14:09.1028629Z [cluster1]  • Deploying Submariner  ...
2021-07-19T04:14:09.1029501Z [cluster1]  ✓ Deploying Submariner
2021-07-19T04:14:09.1030207Z [cluster1]  ✓ Submariner is up and running
2021-07-19T04:14:17.9218161Z [submariner-operator]$ [cluster2] subctl_install_subm
2021-07-19T04:14:17.9219785Z [submariner-operator]$ [cluster2] subctlrepver=
2021-07-19T04:14:17.9221011Z [submariner-operator]$ [cluster2] subctlrepver=--repository localhost:5000 --version local
2021-07-19T04:14:17.9222386Z [submariner-operator]$ [cluster2] PASS_CIDR_ARGS=false
2021-07-19T04:14:17.9226042Z [submariner-operator]$ [cluster2] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2 --clusterid cluster2 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr 242.254.2.0/24 --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm
2021-07-19T04:14:17.9229686Z [cluster2] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.7:6443
2021-07-19T04:14:17.9230970Z [cluster2] * There are 1 labeled nodes in the cluster:
2021-07-19T04:14:17.9231886Z [cluster2]   - cluster2-worker
2021-07-19T04:14:17.9232629Z [cluster2]  • Discovering network details  ...
2021-07-19T04:14:17.9233535Z [cluster2]         Network plugin:  weave-net
2021-07-19T04:14:17.9234098Z [cluster2]         Service CIDRs:   [100.0.0.0/16]
2021-07-19T04:14:17.9234808Z [cluster2]         Cluster CIDRs:   [10.0.0.0/16]
2021-07-19T04:14:17.9235565Z [cluster2]  ✓ Discovering network details
2021-07-19T04:14:17.9236366Z [cluster2]  • Discovering multi cluster details  ...
2021-07-19T04:14:17.9237702Z [cluster2]  • Validating Globalnet configurations  ...
2021-07-19T04:14:17.9238764Z [cluster2]  ✓ Validating Globalnet configurations
2021-07-19T04:14:17.9239569Z [cluster2]  • Assigning Globalnet IPs  ...
2021-07-19T04:14:17.9240545Z [cluster2]  ✓ Assigning Globalnet IPs
2021-07-19T04:14:17.9241593Z [cluster2]  ✓ GlobalCIDR is: 242.254.2.0/24
2021-07-19T04:14:17.9242491Z [cluster2]  ✓ Discovering multi cluster details
2021-07-19T04:14:17.9243578Z [cluster2]  • Deploying the Submariner operator  ...
2021-07-19T04:14:17.9244812Z [cluster2]  ✓ Deploying the Submariner operator
2021-07-19T04:14:17.9245595Z [cluster2]  ✓ Created operator CRDs
2021-07-19T04:14:17.9246688Z [cluster2]  ✓ Created operator namespace: submariner-operator
2021-07-19T04:14:17.9247833Z [cluster2]  ✓ Created operator service account and role
2021-07-19T04:14:17.9248721Z [cluster2]  ✓ Created lighthouse service account and role
2021-07-19T04:14:17.9249621Z [cluster2]  ✓ Created Lighthouse service accounts and roles
2021-07-19T04:14:17.9250472Z [cluster2]  ✓ Deployed the operator successfully
2021-07-19T04:14:17.9251991Z [cluster2]  • Creating SA for cluster  ...
2021-07-19T04:14:17.9252887Z [cluster2]  ✓ Creating SA for cluster
2021-07-19T04:14:17.9253931Z [cluster2]  • Deploying Submariner  ...
2021-07-19T04:14:17.9254774Z [cluster2]  ✓ Deploying Submariner
2021-07-19T04:14:17.9256505Z [cluster2]  ✓ Submariner is up and running
2021-07-19T04:14:17.9275225Z [submariner-operator]$ [cluster2] wait 7312
2021-07-19T04:14:17.9309936Z [submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status
2021-07-19T04:14:17.9355273Z [submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status
2021-07-19T04:14:17.9428750Z [submariner-operator]$ [cluster2] with_retries 30 verify_gw_status
2021-07-19T04:14:17.9468562Z [submariner-operator]$ [cluster2] with_retries
2021-07-19T04:14:17.9485200Z [submariner-operator]$ [cluster2] iteration=1
2021-07-19T04:14:17.9586243Z [submariner-operator]$ [cluster2] wait 7357
2021-07-19T04:14:17.9587517Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:17.9619647Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:17.9633280Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-19T04:14:17.9663163Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:17.9819057Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-19T04:14:20.0565055Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:20.0582280Z [submariner-operator]$ [cluster2] grep connected
2021-07-19T04:14:20.2542559Z  • Showing Connections  ...
2021-07-19T04:14:20.2920551Z  ✗ Showing Connections
2021-07-19T04:14:20.2922014Z  ✗ There are no gateways detected
2021-07-19T04:14:20.5732750Z  • Showing Connections  ...
2021-07-19T04:14:20.6026563Z  ✗ Showing Connections
2021-07-19T04:14:20.6027306Z  ✗ There are no gateways detected
2021-07-19T04:14:20.6048457Z iter: 1. Clusters not yet connected. sleeping for 6 secs
2021-07-19T04:14:20.6060884Z [submariner-operator]$ [cluster2] sleep 6
2021-07-19T04:14:26.6102806Z [submariner-operator]$ [cluster2] return 1
2021-07-19T04:14:26.6178486Z [submariner-operator]$ [cluster2] wait 7384
2021-07-19T04:14:26.6179702Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:26.6210513Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:26.6233120Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-19T04:14:26.6259115Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:26.6320374Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-19T04:14:26.9056515Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:26.9124892Z [submariner-operator]$ [cluster2] grep connected
2021-07-19T04:14:27.0875488Z  • Showing Connections  ...
2021-07-19T04:14:27.0926990Z  ✗ Showing Connections
2021-07-19T04:14:27.0927736Z  ✗ There are no gateways detected
2021-07-19T04:14:27.1182081Z  • Showing Connections  ...
2021-07-19T04:14:27.1182665Z  ✗ Showing Connections
2021-07-19T04:14:27.1183412Z  ✗ There are no gateways detected
2021-07-19T04:14:27.1241393Z iter: 2. Clusters not yet connected. sleeping for 6 secs
2021-07-19T04:14:27.1258318Z [submariner-operator]$ [cluster2] sleep 6
2021-07-19T04:14:33.1305224Z [submariner-operator]$ [cluster2] return 1
2021-07-19T04:14:33.1373507Z [submariner-operator]$ [cluster2] wait 7410
2021-07-19T04:14:33.1417485Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:33.1474989Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:33.1496651Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-19T04:14:33.1581083Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:33.1611232Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-19T04:14:34.0527349Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:34.0580622Z [submariner-operator]$ [cluster2] grep connected
2021-07-19T04:14:34.5706734Z  • Showing Connections  ...
2021-07-19T04:14:34.7153682Z  ✗ Showing Connections
2021-07-19T04:14:34.7154322Z  ✗ No connections found
2021-07-19T04:14:35.0417240Z  • Showing Connections  ...
2021-07-19T04:14:35.4795564Z  ✗ Showing Connections
2021-07-19T04:14:35.4796244Z  ✗ There are no gateways detected
2021-07-19T04:14:35.4836090Z iter: 3. Clusters not yet connected. sleeping for 6 secs
2021-07-19T04:14:35.4849251Z [submariner-operator]$ [cluster2] sleep 6
2021-07-19T04:14:41.4882088Z [submariner-operator]$ [cluster2] return 1
2021-07-19T04:14:41.4953623Z [submariner-operator]$ [cluster2] wait 7437
2021-07-19T04:14:41.4959648Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:41.4978680Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:41.4989988Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-19T04:14:41.5001828Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:41.5045045Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-19T04:14:42.4033539Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:42.4051601Z [submariner-operator]$ [cluster2] grep connected
2021-07-19T04:14:42.6869594Z  • Showing Connections  ...
2021-07-19T04:14:42.7041533Z  ✗ Showing Connections
2021-07-19T04:14:42.7042168Z  ✗ No connections found
2021-07-19T04:14:42.7640697Z  • Showing Connections  ...
2021-07-19T04:14:42.8162043Z  ✗ Showing Connections
2021-07-19T04:14:42.8167463Z  ✗ There are no gateways detected
2021-07-19T04:14:42.8196237Z iter: 4. Clusters not yet connected. sleeping for 6 secs
2021-07-19T04:14:42.8212295Z [submariner-operator]$ [cluster2] sleep 6
2021-07-19T04:14:48.8244522Z [submariner-operator]$ [cluster2] return 1
2021-07-19T04:14:48.8286392Z [submariner-operator]$ [cluster2] wait 7463
2021-07-19T04:14:48.8299853Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:48.8323137Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:48.8349049Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-19T04:14:48.8368424Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:48.8421580Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-19T04:14:49.1457178Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:49.1473663Z [submariner-operator]$ [cluster2] grep connected
2021-07-19T04:14:49.3460746Z  • Showing Connections  ...
2021-07-19T04:14:49.3566024Z  ✗ Showing Connections
2021-07-19T04:14:49.3566724Z  ✗ No connections found
2021-07-19T04:14:49.4155110Z  • Showing Connections  ...
2021-07-19T04:14:49.5156663Z  ✗ Showing Connections
2021-07-19T04:14:49.5157685Z  ✗ There are no gateways detected
2021-07-19T04:14:49.5185602Z iter: 5. Clusters not yet connected. sleeping for 6 secs
2021-07-19T04:14:49.5204924Z [submariner-operator]$ [cluster2] sleep 6
2021-07-19T04:14:55.5238029Z [submariner-operator]$ [cluster2] return 1
2021-07-19T04:14:55.5277163Z [submariner-operator]$ [cluster2] wait 7490
2021-07-19T04:14:55.5289066Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:55.5317398Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:14:55.5332927Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-19T04:14:55.5366837Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:55.5403408Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-19T04:14:55.7702360Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:14:55.7721616Z [submariner-operator]$ [cluster2] grep connected
2021-07-19T04:14:55.9187715Z  • Showing Connections  ...
2021-07-19T04:14:55.9354988Z  ✗ Showing Connections
2021-07-19T04:14:55.9360277Z  ✗ No connections found
2021-07-19T04:14:56.0308667Z  • Showing Connections  ...
2021-07-19T04:14:56.0448590Z  ✗ Showing Connections
2021-07-19T04:14:56.0454290Z  ✗ There are no gateways detected
2021-07-19T04:14:56.0494391Z iter: 6. Clusters not yet connected. sleeping for 6 secs
2021-07-19T04:14:56.0523015Z [submariner-operator]$ [cluster2] sleep 6
2021-07-19T04:15:02.0559942Z [submariner-operator]$ [cluster2] return 1
2021-07-19T04:15:02.0611276Z [submariner-operator]$ [cluster2] wait 7516
2021-07-19T04:15:02.0636220Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:15:02.0650516Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-19T04:15:02.0665151Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-19T04:15:02.0686104Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:15:02.0715090Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-19T04:15:02.3499651Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-19T04:15:02.3551854Z [submariner-operator]$ [cluster2] grep connected
2021-07-19T04:15:02.4387148Z  • Showing Connections  ...
2021-07-19T04:15:02.4449103Z  ✗ Showing Connections
2021-07-19T04:15:02.4450095Z  ✗ No connections found
2021-07-19T04:15:02.5106770Z  • Showing Connections  ...
2021-07-19T04:15:02.5137282Z  ✓ Showing Connections
2021-07-19T04:15:02.5163525Z cluster1-worker  cluster1  172.18.0.8  no   libreswan     242.254.1.0/24  connected  0s        
2021-07-19T04:15:02.5181985Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:02.5243433Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:02.5258808Z [submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1
2021-07-19T04:15:02.5270470Z [submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1
2021-07-19T04:15:02.5288756Z [submariner-operator]$ [cluster2] connectivity_tests cluster1
2021-07-19T04:15:02.5312661Z [submariner-operator]$ [cluster2] connectivity_tests
2021-07-19T04:15:02.5326582Z [submariner-operator]$ [cluster2] target_cluster=cluster1
2021-07-19T04:15:02.5358230Z [submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest
2021-07-19T04:15:02.5370943Z [submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest
2021-07-19T04:15:02.5388925Z [submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local
2021-07-19T04:15:03.1230445Z Error response from daemon: No such image: quay.io/submariner/nettest:dev
2021-07-19T04:15:03.1461812Z [submariner-operator]$ [cluster2] docker pull quay.io/submariner/nettest:devel
2021-07-19T04:15:04.5046213Z devel: Pulling from submariner/nettest
2021-07-19T04:15:04.5055185Z 5843afab3874: Already exists
2021-07-19T04:15:04.5067629Z 9565bb92f43f: Pulling fs layer
2021-07-19T04:15:04.5068108Z a31c0d5b4ad6: Pulling fs layer
2021-07-19T04:15:04.5071523Z dc3d13650205: Pulling fs layer
2021-07-19T04:15:04.7840746Z 9565bb92f43f: Verifying Checksum
2021-07-19T04:15:04.7841658Z 9565bb92f43f: Download complete
2021-07-19T04:15:04.8561716Z dc3d13650205: Verifying Checksum
2021-07-19T04:15:04.9883741Z a31c0d5b4ad6: Verifying Checksum
2021-07-19T04:15:04.9884363Z a31c0d5b4ad6: Download complete
2021-07-19T04:15:05.0523331Z 9565bb92f43f: Pull complete
2021-07-19T04:15:05.4477109Z a31c0d5b4ad6: Pull complete
2021-07-19T04:15:05.5412511Z dc3d13650205: Pull complete
2021-07-19T04:15:05.5472686Z Digest: sha256:ef019e5e56af44d25e621cd556d8f40edcd641d0acd17c307486b4f4047b848f
2021-07-19T04:15:05.5501730Z Status: Downloaded newer image for quay.io/submariner/nettest:devel
2021-07-19T04:15:05.5556404Z quay.io/submariner/nettest:devel
2021-07-19T04:15:05.5615411Z [submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:devel quay.io/submariner/nettest:dev
2021-07-19T04:15:05.8459004Z [submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local
2021-07-19T04:15:06.1390095Z [submariner-operator]$ [cluster2] docker push localhost:5000/nettest:local
2021-07-19T04:15:06.4360833Z The push refers to repository [localhost:5000/nettest]
2021-07-19T04:15:06.4433475Z 4f2cdf68c81d: Preparing
2021-07-19T04:15:06.4434042Z 1f803d62c8b3: Preparing
2021-07-19T04:15:06.4434509Z 80c9eaab4c9e: Preparing
2021-07-19T04:15:06.4434955Z 72e830a4dff5: Preparing
2021-07-19T04:15:07.1833058Z 80c9eaab4c9e: Pushed
2021-07-19T04:15:07.1834304Z 4f2cdf68c81d: Pushed
2021-07-19T04:15:07.2140817Z 72e830a4dff5: Pushed
2021-07-19T04:15:07.8464307Z 1f803d62c8b3: Pushed
2021-07-19T04:15:07.8783243Z local: digest: sha256:ef019e5e56af44d25e621cd556d8f40edcd641d0acd17c307486b4f4047b848f size: 1155
2021-07-19T04:15:07.8862606Z [submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:07.8880328Z [submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:07.8897456Z [submariner-operator]$ [cluster2] resource_name=$(basename "$resource_file" ".yaml")
2021-07-19T04:15:07.8918146Z [submariner-operator]$ [cluster2] basename /opt/shipyard/scripts/resources/netshoot.yaml .yaml
2021-07-19T04:15:07.8948500Z [submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:07.8961512Z [submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:07.8975283Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:08.4431628Z deployment.apps/netshoot created
2021-07-19T04:15:08.4524239Z Waiting for netshoot pods to be ready.
2021-07-19T04:15:08.4556592Z [submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m
2021-07-19T04:15:08.4586525Z [submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m
2021-07-19T04:15:08.4597484Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 rollout status deploy/netshoot --timeout=5m
2021-07-19T04:15:08.8693752Z Waiting for deployment "netshoot" rollout to finish: 0 of 1 updated replicas are available...
2021-07-19T04:15:10.6310942Z deployment "netshoot" successfully rolled out
2021-07-19T04:15:10.6384872Z [submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:10.6410687Z [submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:10.6439372Z [submariner-operator]$ [cluster1] deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:10.6467002Z [submariner-operator]$ [cluster1] deploy_resource
2021-07-19T04:15:10.6487735Z [submariner-operator]$ [cluster1] resource_name=$(basename "$resource_file" ".yaml")
2021-07-19T04:15:10.6522787Z [submariner-operator]$ [cluster1] basename /opt/shipyard/scripts/resources/nginx-demo.yaml .yaml
2021-07-19T04:15:10.6563206Z [submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:10.6591332Z [submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:10.6613377Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:11.3739760Z deployment.apps/nginx-demo created
2021-07-19T04:15:11.3840355Z service/nginx-demo created
2021-07-19T04:15:11.4577050Z serviceexport.multicluster.x-k8s.io/nginx-demo created
2021-07-19T04:15:11.4659017Z Waiting for nginx-demo pods to be ready.
2021-07-19T04:15:11.4687008Z [submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m
2021-07-19T04:15:11.4711786Z [submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m
2021-07-19T04:15:11.4766791Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 rollout status deploy/nginx-demo --timeout=5m
2021-07-19T04:15:11.9262101Z Waiting for deployment "nginx-demo" rollout to finish: 0 of 2 updated replicas are available...
2021-07-19T04:15:18.9297136Z Waiting for deployment "nginx-demo" rollout to finish: 1 of 2 updated replicas are available...
2021-07-19T04:15:19.3683787Z deployment "nginx-demo" successfully rolled out
2021-07-19T04:15:19.3741851Z [submariner-operator]$ [cluster2] netshoot_pod=$(kubectl get pods -l app=netshoot | awk 'FNR == 2 {print $1}')
2021-07-19T04:15:19.3764747Z [submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot
2021-07-19T04:15:19.3789525Z [submariner-operator]$ [cluster2] awk FNR == 2 {print $1}
2021-07-19T04:15:19.3803437Z [submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot
2021-07-19T04:15:19.3885055Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods -l app=netshoot
2021-07-19T04:15:19.8275678Z [submariner-operator]$ [cluster2] nginx_svc_ip=$(with_context "$target_cluster" get_svc_ip nginx-demo)
2021-07-19T04:15:19.8291828Z [submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo
2021-07-19T04:15:19.8309135Z [submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo
2021-07-19T04:15:19.8508930Z [submariner-operator]$ [cluster1] get_svc_ip nginx-demo
2021-07-19T04:15:19.8525073Z [submariner-operator]$ [cluster1] get_svc_ip
2021-07-19T04:15:19.8550298Z [submariner-operator]$ [cluster1] svc_ip=$(with_retries 30 get_globalip "${svc_name}")
2021-07-19T04:15:19.8572536Z [submariner-operator]$ [cluster1] with_retries 30 get_globalip nginx-demo
2021-07-19T04:15:19.8588358Z [submariner-operator]$ [cluster1] with_retries 30 get_globalip nginx-demo
2021-07-19T04:15:19.8648857Z [submariner-operator]$ [cluster1] iteration=1
2021-07-19T04:15:19.8680613Z [submariner-operator]$ [cluster1] get_globalip nginx-demo
2021-07-19T04:15:19.8684865Z [submariner-operator]$ [cluster1] wait 7657
2021-07-19T04:15:19.8708126Z [submariner-operator]$ [cluster1] get_globalip
2021-07-19T04:15:19.8726794Z [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
2021-07-19T04:15:19.8746999Z [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:19.8760457Z [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:19.8778601Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:20.1786117Z [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
2021-07-19T04:15:20.1802626Z [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:20.1813676Z [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:20.1829466Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:20.5116668Z Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
2021-07-19T04:15:20.5173430Z [submariner-operator]$ [cluster1] sleep 1
2021-07-19T04:15:21.5213096Z [submariner-operator]$ [cluster1] return 1
2021-07-19T04:15:21.5253448Z [submariner-operator]$ [cluster1] get_globalip nginx-demo
2021-07-19T04:15:21.5256603Z [submariner-operator]$ [cluster1] wait 7686
2021-07-19T04:15:21.5269187Z [submariner-operator]$ [cluster1] get_globalip
2021-07-19T04:15:21.5288649Z [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
2021-07-19T04:15:21.5308665Z [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:21.5321571Z [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:21.5334894Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:21.8059347Z [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
2021-07-19T04:15:21.8079652Z [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:21.8100661Z [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:21.8128482Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:22.1052955Z Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
2021-07-19T04:15:22.1113086Z [submariner-operator]$ [cluster1] sleep 1
2021-07-19T04:15:23.1141955Z [submariner-operator]$ [cluster1] return 1
2021-07-19T04:15:23.1178707Z [submariner-operator]$ [cluster1] get_globalip nginx-demo
2021-07-19T04:15:23.1191778Z [submariner-operator]$ [cluster1] wait 7715
2021-07-19T04:15:23.1201260Z [submariner-operator]$ [cluster1] get_globalip
2021-07-19T04:15:23.1217538Z [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
2021-07-19T04:15:23.1234703Z [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:23.1256058Z [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:23.1271882Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
2021-07-19T04:15:23.4386721Z [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
2021-07-19T04:15:23.4395527Z [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:23.4405592Z [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:23.4417375Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
2021-07-19T04:15:23.7563374Z [submariner-operator]$ [cluster1] return 0
2021-07-19T04:15:23.7594972Z [submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-pjdw4 242.254.1.252
2021-07-19T04:15:23.7610247Z [submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-pjdw4 242.254.1.252
2021-07-19T04:15:23.7626398Z [submariner-operator]$ [cluster2] iteration=1
2021-07-19T04:15:23.7716888Z [submariner-operator]$ [cluster2] wait 7744
2021-07-19T04:15:23.7727464Z [submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-pjdw4 242.254.1.252
2021-07-19T04:15:23.7746007Z [submariner-operator]$ [cluster2] test_connection
2021-07-19T04:15:23.7753115Z Attempting connectivity between clusters - netshoot-57677568b8-pjdw4 --> 242.254.1.252
2021-07-19T04:15:23.7766497Z [submariner-operator]$ [cluster2] wait_time=30
2021-07-19T04:15:23.7779646Z [submariner-operator]$ [cluster2] start_time=$(date +%s)
2021-07-19T04:15:23.7873915Z [submariner-operator]$ [cluster2] date +%s
2021-07-19T04:15:23.7913481Z [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-pjdw4 -- curl --output /dev/null -m 30 --silent --head --fail 242.254.1.252
2021-07-19T04:15:23.7925372Z [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-pjdw4 -- curl --output /dev/null -m 30 --silent --head --fail 242.254.1.252
2021-07-19T04:15:23.7937335Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-pjdw4 -- curl --output /dev/null -m 30 --silent --head --fail 242.254.1.252
2021-07-19T04:15:24.2265671Z Connection test was successful!
2021-07-19T04:15:24.2287124Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:24.2307867Z [submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:24.2333524Z [submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:24.2353883Z [submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:24.2374590Z [submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:24.2397263Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 delete -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-19T04:15:24.5689023Z deployment.apps "netshoot" deleted
2021-07-19T04:15:24.5888034Z [submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:24.5909254Z [submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:24.5927983Z [submariner-operator]$ [cluster1] remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:24.5941399Z [submariner-operator]$ [cluster1] remove_resource
2021-07-19T04:15:24.5957209Z [submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:24.5978125Z [submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:24.5997428Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-19T04:15:24.9034426Z deployment.apps "nginx-demo" deleted
2021-07-19T04:15:25.0821794Z service "nginx-demo" deleted
2021-07-19T04:15:25.1711146Z serviceexport.multicluster.x-k8s.io "nginx-demo" deleted
2021-07-19T04:15:25.1852851Z [submariner-operator]$ [cluster2] run_if_defined post_deploy
2021-07-19T04:15:25.1872009Z [submariner-operator]$ [cluster2] run_if_defined post_deploy
2021-07-19T04:15:25.1888120Z [submariner-operator]$ [cluster2] type -t post_deploy
2021-07-19T04:15:25.1912401Z [submariner-operator]$ [cluster2] grep -q function
2021-07-19T04:15:25.1954384Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:25.1969152Z   Successfully remade target file 'deploy'.
2021-07-19T04:15:25.1969989Z Must remake target 'e2e'.
2021-07-19T04:15:25.1970562Z scripts/kind-e2e/e2e.sh
2021-07-19T04:15:25.2069581Z [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
2021-07-19T04:15:25.2083724Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:15:25.2097183Z [submariner-operator]$ script_name=utils
2021-07-19T04:15:25.2116850Z [submariner-operator]$ exec_name=e2e.sh
2021-07-19T04:15:25.2138479Z [submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs
2021-07-19T04:15:25.2155655Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:15:25.2167622Z [submariner-operator]$ script_name=deploy_funcs
2021-07-19T04:15:25.2180387Z [submariner-operator]$ exec_name=e2e.sh
2021-07-19T04:15:25.2205186Z [submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_operator
2021-07-19T04:15:25.2224573Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:15:25.2237392Z [submariner-operator]$ script_name=deploy_operator
2021-07-19T04:15:25.2250718Z [submariner-operator]$ exec_name=e2e.sh
2021-07-19T04:15:25.2271029Z [submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings
2021-07-19T04:15:25.2286080Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:15:25.2300585Z [submariner-operator]$ script_name=cluster_settings
2021-07-19T04:15:25.2312130Z [submariner-operator]$ exec_name=e2e.sh
2021-07-19T04:15:25.2337491Z [submariner-operator]$ broker=cluster1
2021-07-19T04:15:25.2349438Z [submariner-operator]$ declare -gA cluster_nodes
2021-07-19T04:15:25.2362814Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker
2021-07-19T04:15:25.2382480Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker
2021-07-19T04:15:25.2393544Z [submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker
2021-07-19T04:15:25.2406098Z [submariner-operator]$ declare -gA cluster_subm
2021-07-19T04:15:25.2417982Z [submariner-operator]$ cluster_subm[cluster1]=true
2021-07-19T04:15:25.2430856Z [submariner-operator]$ cluster_subm[cluster2]=true
2021-07-19T04:15:25.2442852Z [submariner-operator]$ cluster_subm[cluster3]=true
2021-07-19T04:15:25.2454747Z [submariner-operator]$ declare -gA cluster_cni
2021-07-19T04:15:25.2469602Z [submariner-operator]$ cluster_cni[cluster2]=weave
2021-07-19T04:15:25.2481768Z [submariner-operator]$ cluster_cni[cluster3]=weave
2021-07-19T04:15:25.2495362Z [submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-19T04:15:25.2512573Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-19T04:15:25.2529061Z [submariner-operator]$ script_name=cluster_settings
2021-07-19T04:15:25.2543120Z [submariner-operator]$ exec_name=e2e.sh
2021-07-19T04:15:25.2569727Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker
2021-07-19T04:15:25.2582881Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker
2021-07-19T04:15:25.2619252Z [submariner-operator]$ globalnet=true
2021-07-19T04:15:25.2633394Z [submariner-operator]$ timeout=2m
2021-07-19T04:15:25.2644860Z [submariner-operator]$ declare_kubeconfig
2021-07-19T04:15:25.2657236Z [submariner-operator]$ declare_kubeconfig
2021-07-19T04:15:25.2672069Z [submariner-operator]$ source /opt/shipyard/scripts/lib/kubecfg
2021-07-19T04:15:25.2686856Z [submariner-operator]$ export KUBECONFIG
2021-07-19T04:15:25.2704675Z [submariner-operator]$ KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)
2021-07-19T04:15:25.2723689Z [submariner-operator]$ find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:
2021-07-19T04:15:25.2764436Z [submariner-operator]$ . /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/lib_operator_verify_subm.sh
2021-07-19T04:15:25.2792225Z [submariner-operator]$ create_subm_vars
2021-07-19T04:15:25.2807102Z [submariner-operator]$ create_subm_vars
2021-07-19T04:15:25.2819901Z [submariner-operator]$ deployment_name=submariner
2021-07-19T04:15:25.2832978Z [submariner-operator]$ operator_deployment_name=submariner-operator
2021-07-19T04:15:25.2844670Z [submariner-operator]$ gateway_deployment_name=submariner-gateway
2021-07-19T04:15:25.2856986Z [submariner-operator]$ routeagent_deployment_name=submariner-routeagent
2021-07-19T04:15:25.2869993Z [submariner-operator]$ broker_deployment_name=submariner-k8s-broker
2021-07-19T04:15:25.2882515Z [submariner-operator]$ globalnet_deployment_name=submariner-globalnet
2021-07-19T04:15:25.2895425Z [submariner-operator]$ declare_cidrs
2021-07-19T04:15:25.2911123Z [submariner-operator]$ declare_cidrs
2021-07-19T04:15:25.2926456Z [submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
2021-07-19T04:15:25.2940301Z [submariner-operator]$ i=1
2021-07-19T04:15:25.2952669Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-19T04:15:25.2964766Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-19T04:15:25.2980462Z [submariner-operator]$ [cluster1] val=0
2021-07-19T04:15:25.2993045Z [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.0.0.0/16
2021-07-19T04:15:25.3007584Z [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.0.0.0/16
2021-07-19T04:15:25.3026076Z [submariner-operator]$ [cluster1] global_CIDRs[cluster1]=242.254.1.0/24
2021-07-19T04:15:25.3048904Z [submariner-operator]$ [cluster1] i=$(("$i"+1))
2021-07-19T04:15:25.3070821Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-19T04:15:25.3081868Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-19T04:15:25.3097527Z [submariner-operator]$ [cluster2] val=0
2021-07-19T04:15:25.3110175Z [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.0.0.0/16
2021-07-19T04:15:25.3123610Z [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.0.0.0/16
2021-07-19T04:15:25.3138915Z [submariner-operator]$ [cluster2] global_CIDRs[cluster2]=242.254.2.0/24
2021-07-19T04:15:25.3148223Z [submariner-operator]$ [cluster2] i=$(("$i"+1))
2021-07-19T04:15:25.3159714Z [submariner-operator]$ [cluster2] natEnabled=false
2021-07-19T04:15:25.3172780Z [submariner-operator]$ [cluster2] subm_gateway_image_repo=localhost:5000
2021-07-19T04:15:25.3184657Z [submariner-operator]$ [cluster2] subm_gateway_image_tag=local
2021-07-19T04:15:25.3198317Z [submariner-operator]$ [cluster2] subm_gateway_size=3
2021-07-19T04:15:25.3216182Z [submariner-operator]$ [cluster2] subm_colorcodes=blue
2021-07-19T04:15:25.3225371Z [submariner-operator]$ [cluster2] subm_debug=false
2021-07-19T04:15:25.3237621Z [submariner-operator]$ [cluster2] subm_broker=k8s
2021-07-19T04:15:25.3248178Z [submariner-operator]$ [cluster2] ce_ipsec_debug=false
2021-07-19T04:15:25.3259918Z [submariner-operator]$ [cluster2] ce_ipsec_ikeport=500
2021-07-19T04:15:25.3272209Z [submariner-operator]$ [cluster2] ce_ipsec_nattport=4500
2021-07-19T04:15:25.3283442Z [submariner-operator]$ [cluster2] subm_ns=submariner-operator
2021-07-19T04:15:25.3297641Z [submariner-operator]$ [cluster2] SUBMARINER_BROKER_NS=submariner-k8s-broker
2021-07-19T04:15:25.3314050Z [submariner-operator]$ [cluster2] with_context cluster1 broker_vars
2021-07-19T04:15:25.3326861Z [submariner-operator]$ [cluster2] with_context cluster1 broker_vars
2021-07-19T04:15:25.3343967Z [submariner-operator]$ [cluster1] broker_vars
2021-07-19T04:15:25.3355273Z [submariner-operator]$ [cluster1] broker_vars
2021-07-19T04:15:25.3368884Z [submariner-operator]$ [cluster1] SUBMARINER_BROKER_URL=$(kubectl -n default get endpoints kubernetes -o jsonpath="{.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}")
2021-07-19T04:15:25.3435029Z [submariner-operator]$ [cluster1] kubectl -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}
2021-07-19T04:15:25.3486785Z [submariner-operator]$ [cluster1] kubectl -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}
2021-07-19T04:15:25.3488819Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}
2021-07-19T04:15:25.8188116Z [submariner-operator]$ [cluster1] SUBMARINER_BROKER_CA=$(kubectl -n ${SUBMARINER_BROKER_NS} get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${SUBMARINER_BROKER_NS}-client')].data['ca\.crt']}")
2021-07-19T04:15:25.8217726Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}
2021-07-19T04:15:25.8351226Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}
2021-07-19T04:15:25.8391503Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}
2021-07-19T04:15:26.1810927Z [submariner-operator]$ [cluster1] SUBMARINER_BROKER_TOKEN=$(kubectl -n ${SUBMARINER_BROKER_NS} get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${SUBMARINER_BROKER_NS}-client')].data.token}"|base64 --decode)
2021-07-19T04:15:26.1834570Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}
2021-07-19T04:15:26.1852800Z [submariner-operator]$ [cluster1] base64 --decode
2021-07-19T04:15:26.1864024Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}
2021-07-19T04:15:26.1882048Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}
2021-07-19T04:15:26.5209516Z [submariner-operator]$ [cluster2] with_context cluster1 verify_subm_broker_secrets
2021-07-19T04:15:26.5223836Z [submariner-operator]$ [cluster2] with_context cluster1 verify_subm_broker_secrets
2021-07-19T04:15:26.5239270Z [submariner-operator]$ [cluster1] verify_subm_broker_secrets
2021-07-19T04:15:26.5253355Z [submariner-operator]$ [cluster1] verify_subm_broker_secrets
2021-07-19T04:15:26.5863296Z [submariner-operator]$ [cluster1] verify_secrets submariner-k8s-broker submariner-k8s-broker-client LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
2021-07-19T04:15:26.6995405Z [submariner-operator]$ [cluster1] verify_secrets submariner-k8s-broker submariner-k8s-broker-client LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
2021-07-19T04:15:26.7477713Z [submariner-operator]$ [cluster1] wait_for_secrets submariner-k8s-broker
2021-07-19T04:15:26.7479045Z [submariner-operator]$ [cluster1] wait_for_secrets submariner-k8s-broker
2021-07-19T04:15:26.7480343Z [submariner-operator]$ [cluster1] kubectl get secret -n submariner-k8s-broker
2021-07-19T04:15:26.7482176Z [submariner-operator]$ [cluster1] grep -q submariner-
2021-07-19T04:15:26.7483289Z [submariner-operator]$ [cluster1] kubectl get secret -n submariner-k8s-broker
2021-07-19T04:15:26.7485073Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secret -n submariner-k8s-broker
2021-07-19T04:15:26.9553076Z [submariner-operator]$ [cluster1] return
2021-07-19T04:15:26.9565677Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker
2021-07-19T04:15:26.9579959Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker
2021-07-19T04:15:26.9597218Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secrets -n submariner-k8s-broker
2021-07-19T04:15:27.2633004Z NAME                                       TYPE                                  DATA   AGE
2021-07-19T04:15:27.2634530Z cluster-cluster1-token-q7vjv               kubernetes.io/service-account-token   3      78s
2021-07-19T04:15:27.2636319Z cluster-cluster2-token-rbdtf               kubernetes.io/service-account-token   3      72s
2021-07-19T04:15:27.2637728Z default-token-jgqw5                        kubernetes.io/service-account-token   3      97s
2021-07-19T04:15:27.2639115Z submariner-k8s-broker-admin-token-qcjp2    kubernetes.io/service-account-token   3      97s
2021-07-19T04:15:27.2640973Z submariner-k8s-broker-client-token-jkngl   kubernetes.io/service-account-token   3      97s
2021-07-19T04:15:27.2707301Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}
2021-07-19T04:15:27.2719844Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}
2021-07-19T04:15:27.2732962Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}
2021-07-19T04:15:27.6225369Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner-k8s-broker-client-token-jkngl.cluster1.json
2021-07-19T04:15:27.6238074Z [submariner-operator]$ [cluster1] kubectl get secret submariner-k8s-broker-client-token-jkngl -n submariner-k8s-broker -o json
2021-07-19T04:15:27.6255275Z [submariner-operator]$ [cluster1] tee /tmp/submariner-k8s-broker-client-token-jkngl.cluster1.json
2021-07-19T04:15:27.6274576Z [submariner-operator]$ [cluster1] kubectl get secret submariner-k8s-broker-client-token-jkngl -n submariner-k8s-broker -o json
2021-07-19T04:15:27.6288624Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secret submariner-k8s-broker-client-token-jkngl -n submariner-k8s-broker -o json
2021-07-19T04:15:28.0187324Z {
2021-07-19T04:15:28.0187867Z     "apiVersion": "v1",
2021-07-19T04:15:28.0188249Z     "data": {
2021-07-19T04:15:28.0760741Z         "ca.crt": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==",
2021-07-19T04:15:28.1525446Z         "namespace": "c3VibWFyaW5lci1rOHMtYnJva2Vy",
2021-07-19T04:15:28.2063765Z         "token": "ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklrdHJVVzFyT0VWd2IyeDFXRUoyUm1aaGIzZHVUa2RNVUU1d01qTTJVbUp4TFVSWGFWVnNjSFJaVlVVaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUp6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJaUxDSnJkV0psY201bGRHVnpMbWx2TDNObGNuWnBZMlZoWTJOdmRXNTBMM05sWTNKbGRDNXVZVzFsSWpvaWMzVmliV0Z5YVc1bGNpMXJPSE10WW5KdmEyVnlMV05zYVdWdWRDMTBiMnRsYmkxcWEyNW5iQ0lzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG01aGJXVWlPaUp6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJdFkyeHBaVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXpaWEoyYVdObExXRmpZMjkxYm5RdWRXbGtJam9pWlRabE1tTXhaR1l0Tm1JMVlTMDBOV1U0TFRoaU9XWXROakJpTlRsaU9HUXhNbU5tSWl3aWMzVmlJam9pYzNsemRHVnRPbk5sY25acFkyVmhZMk52ZFc1ME9uTjFZbTFoY21sdVpYSXRhemh6TFdKeWIydGxjanB6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJdFkyeHBaVzUwSW4wLmhzVDRwNjlELVY4RWxnV01UbTU4Q29yLVZMVlNDa3N5WkxVOGJFbWpxdThqMkh4UVF5eF9EZGVJS3FqSFlURU5Rc0NPbTBDNHVGak1fVjZTZFBJa2R0SU1CcTFpVUtTbE0xRDF6cFhkbDNtNElMVXh2R1JvTXZXdW5zMkJLSngxb3lVaVVNRi1oZHlBX3hTeXdvOHMwWVhmME1QdXBCLXg4RHFhQm5mdS01QWRhUVI0R0U3Y2R2NnBOTlhRTU9Gcmc0dHpud0xpazdhUzJQSFJkU2kzVDIyeVoyNzZGNDhOaGNFUFp1QkdDNHBrdGxyNEpsd2RHQ2gzODFZc1pOS2dtLVZqWFdjNkpyeTl5SVdsRmEwWGFsdndDUTByYjBwS05jM0xlTHpoeWhmTkpScU0zanp4N3RCRGNkVjU2ZEZxcm9yY2FxV3d0bjdfLUVhSEJFSlZVQQ=="
2021-07-19T04:15:28.2579930Z     },
2021-07-19T04:15:28.2580293Z     "kind": "Secret",
2021-07-19T04:15:28.2580679Z     "metadata": {
2021-07-19T04:15:28.2581064Z         "annotations": {
2021-07-19T04:15:28.2582468Z             "kubernetes.io/service-account.name": "submariner-k8s-broker-client",
2021-07-19T04:15:28.2583788Z             "kubernetes.io/service-account.uid": "e6e2c1df-6b5a-45e8-8b9f-60b59b8d12cf"
2021-07-19T04:15:28.2584742Z         },
2021-07-19T04:15:28.2585372Z         "creationTimestamp": "2021-07-19T04:13:50Z",
2021-07-19T04:15:28.2585892Z         "managedFields": [
2021-07-19T04:15:28.2586279Z             {
2021-07-19T04:15:28.2586642Z                 "apiVersion": "v1",
2021-07-19T04:15:28.2587121Z                 "fieldsType": "FieldsV1",
2021-07-19T04:15:28.2587558Z                 "fieldsV1": {
2021-07-19T04:15:28.2588087Z                     "f:data": {
2021-07-19T04:15:28.2588586Z                         ".": {},
2021-07-19T04:15:28.2589193Z                         "f:ca.crt": {},
2021-07-19T04:15:28.2589566Z                         "f:namespace": {},
2021-07-19T04:15:28.2590140Z                         "f:token": {}
2021-07-19T04:15:28.2590473Z                     },
2021-07-19T04:15:28.2590842Z                     "f:metadata": {
2021-07-19T04:15:28.2591245Z                         "f:annotations": {
2021-07-19T04:15:28.2591791Z                             ".": {},
2021-07-19T04:15:28.2593756Z                             "f:kubernetes.io/service-account.name": {},
2021-07-19T04:15:28.2595296Z                             "f:kubernetes.io/service-account.uid": {}
2021-07-19T04:15:28.2595912Z                         }
2021-07-19T04:15:28.2596259Z                     },
2021-07-19T04:15:28.2596653Z                     "f:type": {}
2021-07-19T04:15:28.2597020Z                 },
2021-07-19T04:15:28.2597798Z                 "manager": "kube-controller-manager",
2021-07-19T04:15:28.2598616Z                 "operation": "Update",
2021-07-19T04:15:28.2599570Z                 "time": "2021-07-19T04:13:50Z"
2021-07-19T04:15:28.2599905Z             }
2021-07-19T04:15:28.2600216Z         ],
2021-07-19T04:15:28.2601519Z         "name": "submariner-k8s-broker-client-token-jkngl",
2021-07-19T04:15:28.2602983Z         "namespace": "submariner-k8s-broker",
2021-07-19T04:15:28.2603842Z         "resourceVersion": "1714",
2021-07-19T04:15:28.2605508Z         "selfLink": "/api/v1/namespaces/submariner-k8s-broker/secrets/submariner-k8s-broker-client-token-jkngl",
2021-07-19T04:15:28.2607006Z         "uid": "f295f460-d57f-4762-acf2-9d7b243fd776"
2021-07-19T04:15:28.2607527Z     },
2021-07-19T04:15:28.2609113Z     "type": "kubernetes.io/service-account-token"
2021-07-19T04:15:28.2611530Z }
2021-07-19T04:15:28.2612645Z [submariner-operator]$ [cluster1] validate_equals .kind Secret
2021-07-19T04:15:28.2613955Z [submariner-operator]$ [cluster1] validate_equals .kind Secret
2021-07-19T04:15:28.2616490Z [submariner-operator]$ [cluster1] jq -r -M .kind /tmp/submariner-k8s-broker-client-token-jkngl.cluster1.json
2021-07-19T04:15:28.2618843Z [submariner-operator]$ [cluster1] validate_equals .type kubernetes.io/service-account-token
2021-07-19T04:15:28.2620275Z [submariner-operator]$ [cluster1] validate_equals .type kubernetes.io/service-account-token
2021-07-19T04:15:28.2622169Z [submariner-operator]$ [cluster1] jq -r -M .type /tmp/submariner-k8s-broker-client-token-jkngl.cluster1.json
2021-07-19T04:15:28.2624428Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner-k8s-broker-client-token-jkngl
2021-07-19T04:15:28.2626697Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner-k8s-broker-client-token-jkngl
2021-07-19T04:15:28.2629228Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariner-k8s-broker-client-token-jkngl.cluster1.json
2021-07-19T04:15:28.2631147Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-k8s-broker
2021-07-19T04:15:28.2632583Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-k8s-broker
2021-07-19T04:15:28.2634688Z [submariner-operator]$ [cluster1] jq -r -M .metadata.namespace /tmp/submariner-k8s-broker-client-token-jkngl.cluster1.json
2021-07-19T04:15:28.2755777Z [submariner-operator]$ [cluster1] jq -r -M .data["ca.crt"] /tmp/submariner-k8s-broker-client-token-jkngl.cluster1.json
2021-07-19T04:15:28.3240189Z [submariner-operator]$ [cluster1] one_ca_crt_ok=true
2021-07-19T04:15:28.3278776Z [submariner-operator]$ [cluster2] run_subm_clusters verify_subm_deployed
2021-07-19T04:15:28.3317226Z [submariner-operator]$ [cluster2] run_subm_clusters verify_subm_deployed
2021-07-19T04:15:28.3339978Z [submariner-operator]$ [cluster2] declare -a subm_clusters
2021-07-19T04:15:28.3433164Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 verify_subm_deployed
2021-07-19T04:15:28.3451224Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 verify_subm_deployed
2021-07-19T04:15:28.3467600Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-19T04:15:28.3490836Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-19T04:15:28.3527511Z [submariner-operator]$ [cluster1] pids[cluster1]=7998
2021-07-19T04:15:28.3556114Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-19T04:15:28.3561427Z [submariner-operator]$ [cluster2] pids[cluster2]=8000
2021-07-19T04:15:28.3569124Z [submariner-operator]$ [cluster1] verify_subm_deployed
2021-07-19T04:15:28.3591103Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-19T04:15:28.3600207Z [submariner-operator]$ [cluster2] wait 8000
2021-07-19T04:15:28.3606237Z [submariner-operator]$ [cluster2] verify_subm_deployed
2021-07-19T04:15:28.3611281Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-19T04:15:28.3645280Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-19T04:15:29.3962788Z [submariner-operator]$ [cluster1] verify_subm_deployed
2021-07-19T04:15:29.3964045Z [submariner-operator]$ [cluster1] verify_endpoints_crd
2021-07-19T04:15:29.3965007Z [submariner-operator]$ [cluster1] verify_endpoints_crd
2021-07-19T04:15:29.3966215Z [submariner-operator]$ [cluster1] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-19T04:15:29.3967770Z [submariner-operator]$ [cluster1] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-19T04:15:29.3968966Z [submariner-operator]$ [cluster1] kubectl get crds endpoints.submariner.io
2021-07-19T04:15:29.3970232Z [submariner-operator]$ [cluster1] kubectl get crds endpoints.submariner.io
2021-07-19T04:15:29.3971556Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds endpoints.submariner.io
2021-07-19T04:15:29.3972410Z [cluster1] NAME                      CREATED AT
2021-07-19T04:15:29.3973831Z [cluster1] endpoints.submariner.io   2021-07-19T04:14:00Z
2021-07-19T04:15:29.3975240Z [submariner-operator]$ [cluster1] json_file=/tmp/endpoints.submariner.io.cluster1.json
2021-07-19T04:15:29.3977800Z [submariner-operator]$ [cluster1] kubectl get crd endpoints.submariner.io -o json
2021-07-19T04:15:29.3979410Z [submariner-operator]$ [cluster1] tee /tmp/endpoints.submariner.io.cluster1.json
2021-07-19T04:15:29.3980876Z [submariner-operator]$ [cluster1] kubectl get crd endpoints.submariner.io -o json
2021-07-19T04:15:29.3982213Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd endpoints.submariner.io -o json
2021-07-19T04:15:29.3983014Z [cluster1] {
2021-07-19T04:15:29.3983541Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:29.3984451Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-19T04:15:29.3985226Z [cluster1]     "metadata": {
2021-07-19T04:15:29.3985705Z [cluster1]         "annotations": {
2021-07-19T04:15:29.3986892Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-19T04:15:29.3987515Z [cluster1]         },
2021-07-19T04:15:29.3988155Z [cluster1]         "creationTimestamp": "2021-07-19T04:14:00Z",
2021-07-19T04:15:29.3988882Z [cluster1]         "generation": 1,
2021-07-19T04:15:29.3989378Z [cluster1]         "managedFields": [
2021-07-19T04:15:29.3989815Z [cluster1]             {
2021-07-19T04:15:29.3990390Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:29.3991195Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:29.3991845Z [cluster1]                 "fieldsV1": {
2021-07-19T04:15:29.3992275Z [cluster1]                     "f:metadata": {
2021-07-19T04:15:29.3992742Z [cluster1]                         "f:annotations": {
2021-07-19T04:15:29.3993523Z [cluster1]                             ".": {},
2021-07-19T04:15:29.3994646Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-19T04:15:29.3995367Z [cluster1]                         }
2021-07-19T04:15:29.3995777Z [cluster1]                     },
2021-07-19T04:15:29.3996223Z [cluster1]                     "f:spec": {
2021-07-19T04:15:29.3997025Z [cluster1]                         "f:conversion": {
2021-07-19T04:15:29.3997647Z [cluster1]                             ".": {},
2021-07-19T04:15:29.3998075Z [cluster1]                             "f:strategy": {}
2021-07-19T04:15:29.3998507Z [cluster1]                         },
2021-07-19T04:15:29.3998906Z [cluster1]                         "f:group": {},
2021-07-19T04:15:29.3999345Z [cluster1]                         "f:names": {
2021-07-19T04:15:29.3999764Z [cluster1]                             "f:kind": {},
2021-07-19T04:15:29.4000220Z [cluster1]                             "f:listKind": {},
2021-07-19T04:15:29.4000691Z [cluster1]                             "f:plural": {},
2021-07-19T04:15:29.4001143Z [cluster1]                             "f:singular": {}
2021-07-19T04:15:29.4001566Z [cluster1]                         },
2021-07-19T04:15:29.4001963Z [cluster1]                         "f:scope": {},
2021-07-19T04:15:29.4002414Z [cluster1]                         "f:versions": {}
2021-07-19T04:15:29.4002823Z [cluster1]                     },
2021-07-19T04:15:29.4003437Z [cluster1]                     "f:status": {
2021-07-19T04:15:29.4004775Z [cluster1]                         "f:acceptedNames": {
2021-07-19T04:15:29.4005360Z [cluster1]                             "f:kind": {},
2021-07-19T04:15:29.4005886Z [cluster1]                             "f:plural": {}
2021-07-19T04:15:29.4006367Z [cluster1]                         },
2021-07-19T04:15:29.4006882Z [cluster1]                         "f:conditions": {},
2021-07-19T04:15:29.4007578Z [cluster1]                         "f:storedVersions": {}
2021-07-19T04:15:29.4008038Z [cluster1]                     }
2021-07-19T04:15:29.4008400Z [cluster1]                 },
2021-07-19T04:15:29.4009210Z [cluster1]                 "manager": "submariner-operator",
2021-07-19T04:15:29.4009794Z [cluster1]                 "operation": "Update",
2021-07-19T04:15:29.4010434Z [cluster1]                 "time": "2021-07-19T04:14:06Z"
2021-07-19T04:15:29.4010829Z [cluster1]             }
2021-07-19T04:15:29.4011367Z [cluster1]         ],
2021-07-19T04:15:29.4011912Z [cluster1]         "name": "endpoints.submariner.io",
2021-07-19T04:15:29.4012747Z [cluster1]         "resourceVersion": "1909",
2021-07-19T04:15:29.4013754Z [cluster1]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/endpoints.submariner.io",
2021-07-19T04:15:29.4015021Z [cluster1]         "uid": "5d212636-e3f5-4f4e-8ff7-e18ee4491799"
2021-07-19T04:15:29.4015675Z [cluster1]     },
2021-07-19T04:15:29.4016045Z [cluster1]     "spec": {
2021-07-19T04:15:29.4016476Z [cluster1]         "conversion": {
2021-07-19T04:15:29.4016932Z [cluster1]             "strategy": "None"
2021-07-19T04:15:29.4017347Z [cluster1]         },
2021-07-19T04:15:29.4017783Z [cluster1]         "group": "submariner.io",
2021-07-19T04:15:29.4018256Z [cluster1]         "names": {
2021-07-19T04:15:29.4018700Z [cluster1]             "kind": "Endpoint",
2021-07-19T04:15:29.4019200Z [cluster1]             "listKind": "EndpointList",
2021-07-19T04:15:29.4019740Z [cluster1]             "plural": "endpoints",
2021-07-19T04:15:29.4020240Z [cluster1]             "singular": "endpoint"
2021-07-19T04:15:29.4020678Z [cluster1]         },
2021-07-19T04:15:29.4021101Z [cluster1]         "scope": "Namespaced",
2021-07-19T04:15:29.4021571Z [cluster1]         "versions": [
2021-07-19T04:15:29.4021954Z [cluster1]             {
2021-07-19T04:15:29.4022343Z [cluster1]                 "name": "v1",
2021-07-19T04:15:29.4022749Z [cluster1]                 "schema": {
2021-07-19T04:15:29.4023252Z [cluster1]                     "openAPIV3Schema": {
2021-07-19T04:15:29.4023779Z [cluster1]                         "properties": {
2021-07-19T04:15:29.4024385Z [cluster1]                             "apiVersion": {
2021-07-19T04:15:29.4026607Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-19T04:15:29.4028138Z [cluster1]                                 "type": "string"
2021-07-19T04:15:29.4028575Z [cluster1]                             },
2021-07-19T04:15:29.4028980Z [cluster1]                             "kind": {
2021-07-19T04:15:29.4031011Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-19T04:15:29.4032533Z [cluster1]                                 "type": "string"
2021-07-19T04:15:29.4032959Z [cluster1]                             },
2021-07-19T04:15:29.4033778Z [cluster1]                             "metadata": {
2021-07-19T04:15:29.4034270Z [cluster1]                                 "type": "object"
2021-07-19T04:15:29.4034893Z [cluster1]                             },
2021-07-19T04:15:29.4035336Z [cluster1]                             "spec": {
2021-07-19T04:15:29.4035816Z [cluster1]                                 "properties": {
2021-07-19T04:15:29.4036334Z [cluster1]                                     "backend": {
2021-07-19T04:15:29.4037152Z [cluster1]                                         "type": "string"
2021-07-19T04:15:29.4037583Z [cluster1]                                     },
2021-07-19T04:15:29.4038041Z [cluster1]                                     "backend_config": {
2021-07-19T04:15:29.4038666Z [cluster1]                                         "additionalProperties": {
2021-07-19T04:15:29.4039263Z [cluster1]                                             "type": "string"
2021-07-19T04:15:29.4039704Z [cluster1]                                         },
2021-07-19T04:15:29.4040159Z [cluster1]                                         "type": "object"
2021-07-19T04:15:29.4040586Z [cluster1]                                     },
2021-07-19T04:15:29.4041264Z [cluster1]                                     "cable_name": {
2021-07-19T04:15:29.4041785Z [cluster1]                                         "type": "string"
2021-07-19T04:15:29.4042218Z [cluster1]                                     },
2021-07-19T04:15:29.4042659Z [cluster1]                                     "cluster_id": {
2021-07-19T04:15:29.4043125Z [cluster1]                                         "type": "string"
2021-07-19T04:15:29.4044024Z [cluster1]                                     },
2021-07-19T04:15:29.4044529Z [cluster1]                                     "healthCheckIP": {
2021-07-19T04:15:29.4045088Z [cluster1]                                         "type": "string"
2021-07-19T04:15:29.4045566Z [cluster1]                                     },
2021-07-19T04:15:29.4046020Z [cluster1]                                     "hostname": {
2021-07-19T04:15:29.4046891Z [cluster1]                                         "type": "string"
2021-07-19T04:15:29.4047649Z [cluster1]                                     },
2021-07-19T04:15:29.4048106Z [cluster1]                                     "nat_enabled": {
2021-07-19T04:15:29.4048590Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:29.4049348Z [cluster1]                                     },
2021-07-19T04:15:29.4049786Z [cluster1]                                     "private_ip": {
2021-07-19T04:15:29.4053717Z [cluster1]                                         "type": "string"
2021-07-19T04:15:29.4054204Z [cluster1]                                     },
2021-07-19T04:15:29.4054811Z [cluster1]                                     "public_ip": {
2021-07-19T04:15:29.4055311Z [cluster1]                                         "type": "string"
2021-07-19T04:15:29.4056117Z [cluster1]                                     },
2021-07-19T04:15:29.4056555Z [cluster1]                                     "subnets": {
2021-07-19T04:15:29.4057001Z [cluster1]                                         "items": {
2021-07-19T04:15:29.4057482Z [cluster1]                                             "type": "string"
2021-07-19T04:15:29.4057919Z [cluster1]                                         },
2021-07-19T04:15:29.4058369Z [cluster1]                                         "type": "array"
2021-07-19T04:15:29.4058808Z [cluster1]                                     }
2021-07-19T04:15:29.4059196Z [cluster1]                                 },
2021-07-19T04:15:29.4059630Z [cluster1]                                 "required": [
2021-07-19T04:15:29.4060089Z [cluster1]                                     "backend",
2021-07-19T04:15:29.8307503Z [cluster1]                                   [submariner-operator]$ [cluster2] verify_subm_deployed
2021-07-19T04:15:29.8308705Z [submariner-operator]$ [cluster2] verify_endpoints_crd
2021-07-19T04:15:29.8309822Z [submariner-operator]$ [cluster2] verify_endpoints_crd
2021-07-19T04:15:29.8311158Z [submariner-operator]$ [cluster2] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-19T04:15:29.8312651Z [submariner-operator]$ [cluster2] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-19T04:15:29.8314742Z [submariner-operator]$ [cluster2] kubectl get crds endpoints.submariner.io
2021-07-19T04:15:29.8316310Z [submariner-operator]$ [cluster2] kubectl get crds endpoints.submariner.io
2021-07-19T04:15:29.8318420Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds endpoints.submariner.io
2021-07-19T04:15:29.8319495Z [cluster2] NAME                      CREATED AT
2021-07-19T04:15:29.8320672Z [cluster2] endpoints.submariner.io   2021-07-19T04:14:14Z
2021-07-19T04:15:29.8322344Z [submariner-operator]$ [cluster2] json_file=/tmp/endpoints.submariner.io.cluster2.json
2021-07-19T04:15:29.8324477Z [submariner-operator]$ [cluster2] kubectl get crd endpoints.submariner.io -o json
2021-07-19T04:15:29.8326572Z [submariner-operator]$ [cluster2] tee /tmp/endpoints.submariner.io.cluster2.json
2021-07-19T04:15:29.8328906Z [submariner-operator]$ [cluster2] kubectl get crd endpoints.submariner.io -o json
2021-07-19T04:15:29.8330373Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd endpoints.submariner.io -o json
2021-07-19T04:15:29.8331193Z [cluster2] {
2021-07-19T04:15:29.8331729Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:29.8332476Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-19T04:15:29.8333056Z [cluster2]     "metadata": {
2021-07-19T04:15:29.8333515Z [cluster2]         "annotations": {
2021-07-19T04:15:29.8335007Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-19T04:15:29.8336020Z [cluster2]         },
2021-07-19T04:15:29.8336988Z [cluster2]         "creationTimestamp": "2021-07-19T04:14:14Z",
2021-07-19T04:15:29.8337880Z [cluster2]         "generation": 1,
2021-07-19T04:15:29.8338513Z [cluster2]         "managedFields": [
2021-07-19T04:15:29.8338949Z [cluster2]             {
2021-07-19T04:15:29.8339687Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:29.8340365Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:29.8340883Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:29.8341566Z [cluster2]                     "f:metadata": {
2021-07-19T04:15:29.8342381Z [cluster2]                         "f:annotations": {
2021-07-19T04:15:29.8342818Z [cluster2]                             ".": {},
2021-07-19T04:15:29.8343872Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-19T04:15:29.8345367Z [cluster2]                         }
2021-07-19T04:15:29.8345941Z [cluster2]                     },
2021-07-19T04:15:29.8346360Z [cluster2]                     "f:spec": {
2021-07-19T04:15:29.8346853Z [cluster2]                         "f:conversion": {
2021-07-19T04:15:29.8347891Z [cluster2]                             ".": {},
2021-07-19T04:15:29.8348317Z [cluster2]                             "f:strategy": {}
2021-07-19T04:15:29.8348997Z [cluster2]                         },
2021-07-19T04:15:29.8349586Z [cluster2]                         "f:group": {},
2021-07-19T04:15:29.8350386Z [cluster2]                         "f:names": {
2021-07-19T04:15:29.8350835Z [cluster2]                             "f:kind": {},
2021-07-19T04:15:29.8351320Z [cluster2]                             "f:listKind": {},
2021-07-19T04:15:29.8351945Z [cluster2]                             "f:plural": {},
2021-07-19T04:15:29.8352584Z [cluster2]                             "f:singular": {}
2021-07-19T04:15:29.8353009Z [cluster2]                         },
2021-07-19T04:15:29.8353408Z [cluster2]                         "f:scope": {},
2021-07-19T04:15:29.8354060Z [cluster2]                         "f:versions": {}
2021-07-19T04:15:29.8354878Z [cluster2]                     },
2021-07-19T04:15:29.8355515Z [cluster2]                     "f:status": {
2021-07-19T04:15:29.8356074Z [cluster2]                         "f:acceptedNames": {
2021-07-19T04:15:29.8356794Z [cluster2]                             "f:kind": {},
2021-07-19T04:15:29.8357271Z [cluster2]                             "f:plural": {}
2021-07-19T04:15:29.8358205Z [cluster2]                         },
2021-07-19T04:15:29.8358632Z [cluster2]                         "f:conditions": {},
2021-07-19T04:15:29.8359159Z [cluster2]                         "f:storedVersions": {}
2021-07-19T04:15:29.8359624Z [cluster2]                     }
2021-07-19T04:15:29.8359982Z [cluster2]                 },
2021-07-19T04:15:29.8360758Z [cluster2]                 "manager": "submariner-operator",
2021-07-19T04:15:29.8361350Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:29.8362064Z [cluster2]                 "time": "2021-07-19T04:14:26Z"
2021-07-19T04:15:29.8362460Z [cluster2]             }
2021-07-19T04:15:29.8362828Z [cluster2]         ],
2021-07-19T04:15:29.8363358Z [cluster2]         "name": "endpoints.submariner.io",
2021-07-19T04:15:29.8364315Z [cluster2]         "resourceVersion": "1916",
2021-07-19T04:15:29.8365393Z [cluster2]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/endpoints.submariner.io",
2021-07-19T04:15:29.8366943Z [cluster2]         "uid": "1b06cab8-7e05-4b5c-9f5f-44e6fd0cc33a"
2021-07-19T04:15:29.8367525Z [cluster2]     },
2021-07-19T04:15:29.8367898Z [cluster2]     "spec": {
2021-07-19T04:15:29.8368344Z [cluster2]         "conversion": {
2021-07-19T04:15:29.8372670Z [cluster2]             "strategy": "None"
2021-07-19T04:15:29.8373180Z [cluster2]         },
2021-07-19T04:15:29.8373891Z [cluster2]         "group": "submariner.io",
2021-07-19T04:15:29.8374441Z [cluster2]         "names": {
2021-07-19T04:15:29.8374936Z [cluster2]             "kind": "Endpoint",
2021-07-19T04:15:29.8428411Z [cluster2]             "listKind": "EndpointList",
2021-07-19T04:15:29.8429122Z [cluster2]             "plural": "endpoints",
2021-07-19T04:15:29.8429994Z [cluster2]             "singular": "endpoint"
2021-07-19T04:15:29.8430651Z [cluster2]         },
2021-07-19T04:15:29.8431120Z [cluster2]         "scope": "Namespaced",
2021-07-19T04:15:29.8431631Z [cluster2]         "versions": [
2021-07-19T04:15:29.8432044Z [cluster2]             {
2021-07-19T04:15:29.8432464Z [cluster2]                 "name": "v1",
2021-07-19T04:15:29.8432903Z [cluster2]                 "schema": {
2021-07-19T04:15:29.8433437Z [cluster2]                     "openAPIV3Schema": {
2021-07-19T04:15:29.8434005Z [cluster2]                         "properties": {
2021-07-19T04:15:29.8434730Z [cluster2]                             "apiVersion": {
2021-07-19T04:15:29.8437198Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-19T04:15:29.8438948Z [cluster2]                                 "type": "string"
2021-07-19T04:15:29.8439427Z [cluster2]                             },
2021-07-19T04:15:29.8439874Z [cluster2]                             "kind": {
2021-07-19T04:15:29.8442647Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-19T04:15:29.8444557Z [cluster2]                                 "type": "string"
2021-07-19T04:15:29.8445069Z [cluster2]                             },
2021-07-19T04:15:29.8445707Z [cluster2]                             "metadata": {
2021-07-19T04:15:29.8446224Z [cluster2]                                 "type": "object"
2021-07-19T04:15:29.8446674Z [cluster2]                             },
2021-07-19T04:15:29.8447121Z [cluster2]                             "spec": {
2021-07-19T04:15:29.8447615Z [cluster2]                                 "properties": {
2021-07-19T04:15:29.8448143Z [cluster2]                                     "backend": {
2021-07-19T04:15:29.8448639Z [cluster2]                                         "type": "string"
2021-07-19T04:15:29.8449125Z [cluster2]                                     },
2021-07-19T04:15:29.8449619Z [cluster2]                                     "backend_config": {
2021-07-19T04:15:29.8450252Z [cluster2]                                         "additionalProperties": {
2021-07-19T04:15:29.8450903Z [cluster2]                                             "type": "string"
2021-07-19T04:15:29.8451376Z [cluster2]                                         },
2021-07-19T04:15:29.8451865Z [cluster2]                                         "type": "object"
2021-07-19T04:15:29.8452330Z [cluster2]                                     },
2021-07-19T04:15:29.8452806Z [cluster2]                                     "cable_name": {
2021-07-19T04:15:29.8453495Z [cluster2]                                         "type": "string"
2021-07-19T04:15:29.8453986Z [cluster2]                                     },
2021-07-19T04:15:29.8454869Z [cluster2]                                     "cluster_id": {
2021-07-19T04:15:29.8455413Z [cluster2]                                         "type": "string"
2021-07-19T04:15:29.8462167Z [cluster2]                                     },
2021-07-19T04:15:29.8462673Z [cluster2]                                     "healthCheckIP": {
2021-07-19T04:15:29.8463239Z [cluster2]                                         "type": "string"
2021-07-19T04:15:29.8463709Z [cluster2]                                     },
2021-07-19T04:15:29.8464329Z [cluster2]                                     "hostname": {
2021-07-19T04:15:29.8464853Z [cluster2]                                         "type": "string"
2021-07-19T04:15:29.8465317Z [cluster2]                                     },
2021-07-19T04:15:29.8465987Z [cluster2]                                     "nat_enabled": {
2021-07-19T04:15:29.8466727Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:29.8467561Z [cluster2]                                     },
2021-07-19T04:15:29.8468027Z [cluster2]                                     "private_ip": {
2021-07-19T04:15:29.8468559Z [cluster2]                                         "type": "string"
2021-07-19T04:15:29.8469042Z [cluster2]                                     },
2021-07-19T04:15:29.8469675Z [cluster2]                                     "public_ip": {
2021-07-19T04:15:29.8471193Z [cluster2]                                         "type": "string"
2021-07-19T04:15:29.8471702Z [cluster2]                                     },
2021-07-19T04:15:29.8472209Z [cluster2]                                     "subnets": {
2021-07-19T04:15:29.8472721Z [cluster2]                                         "items": {
2021-07-19T04:15:29.8473277Z [cluster2]                                             "type": "string"
2021-07-19T04:15:29.8474015Z [cluster2]                                         },
2021-07-19T04:15:29.8474504Z [cluster2]                                         "type": "array"
2021-07-19T04:15:29.8475156Z [cluster2]                                     }
2021-07-19T04:15:29.8475581Z [cluster2]                                 },
2021-07-19T04:15:29.8476065Z [cluster2]                                 "required": [
2021-07-19T04:15:29.8476569Z [cluster2]                                     "backend",
2021-07-19T04:15:30.0240744Z [cluster2]                                     "cable_name",
2021-07-19T04:15:30.0241267Z [cluster1]                                     "cluster_id",
2021-07-19T04:15:30.0241721Z [cluster1]                                     "hostname",
2021-07-19T04:15:30.0242197Z [cluster1]                                     "nat_enabled",
2021-07-19T04:15:30.0242664Z [cluster1]                                     "private_ip",
2021-07-19T04:15:30.0243390Z [cluster1]                                     "public_ip",
2021-07-19T04:15:30.0243849Z [cluster1]                                     "subnets"
2021-07-19T04:15:30.0244290Z [cluster1]                                 ],
2021-07-19T04:15:30.0244743Z [cluster1]                                 "type": "object"
2021-07-19T04:15:30.0245166Z [cluster1]                             }
2021-07-19T04:15:30.0245558Z [cluster1]                         },
2021-07-19T04:15:30.0246138Z [cluster1]                         "required": [
2021-07-19T04:15:30.0246585Z [cluster1]                             "spec"
2021-07-19T04:15:30.0246963Z [cluster1]                         ],
2021-07-19T04:15:30.0247384Z [cluster1]                         "type": "object"
2021-07-19T04:15:30.0247781Z [cluster1]                     }
2021-07-19T04:15:30.0248148Z [cluster1]                 },
2021-07-19T04:15:30.0248541Z [cluster1]                 "served": true,
2021-07-19T04:15:30.0248992Z [cluster1]                 "storage": true
2021-07-19T04:15:30.0249396Z [cluster1]             }
2021-07-19T04:15:30.0249907Z [cluster1]         ]
2021-07-19T04:15:30.0250278Z [cluster1]     },
2021-07-19T04:15:30.0250632Z [cluster1]     "status": {
2021-07-19T04:15:30.0251079Z [cluster1]         "acceptedNames": {
2021-07-19T04:15:30.0251548Z [cluster1]             "kind": "Endpoint",
2021-07-19T04:15:30.0252055Z [cluster1]             "listKind": "EndpointList",
2021-07-19T04:15:30.0252741Z [cluster1]             "plural": "endpoints",
2021-07-19T04:15:30.0253462Z [cluster1]             "singular": "endpoint"
2021-07-19T04:15:30.0253894Z [cluster1]         },
2021-07-19T04:15:30.0254305Z [cluster1]         "conditions": [
2021-07-19T04:15:30.0254704Z [cluster1]             {
2021-07-19T04:15:30.0256425Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:14:00Z",
2021-07-19T04:15:30.0257020Z [cluster1]                 "message": "no conflicts found",
2021-07-19T04:15:30.0257529Z [cluster1]                 "reason": "NoConflicts",
2021-07-19T04:15:30.0258007Z [cluster1]                 "status": "True",
2021-07-19T04:15:30.0258477Z [cluster1]                 "type": "NamesAccepted"
2021-07-19T04:15:30.0258918Z [cluster1]             },
2021-07-19T04:15:30.0259250Z [cluster1]             {
2021-07-19T04:15:30.0259928Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:14:00Z",
2021-07-19T04:15:30.0260558Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-19T04:15:30.0261225Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-19T04:15:30.0261921Z [cluster1]                 "status": "True",
2021-07-19T04:15:30.0262369Z [cluster1]                 "type": "Established"
2021-07-19T04:15:30.0263053Z [cluster1]             }
2021-07-19T04:15:30.0263638Z [cluster1]         ],
2021-07-19T04:15:30.0264083Z [cluster1]         "storedVersions": [
2021-07-19T04:15:30.0264514Z [cluster1]             "v1"
2021-07-19T04:15:30.0264885Z [cluster1]         ]
2021-07-19T04:15:30.0265403Z [cluster1]     }
2021-07-19T04:15:30.0265769Z [cluster1] }
2021-07-19T04:15:30.0267259Z [submariner-operator]$ [cluster1] validate_equals .metadata.name endpoints.submariner.io
2021-07-19T04:15:30.0268564Z [submariner-operator]$ [cluster1] validate_equals .metadata.name endpoints.submariner.io
2021-07-19T04:15:30.0270028Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/endpoints.submariner.io.cluster1.json
2021-07-19T04:15:30.0271759Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-19T04:15:30.0273077Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-19T04:15:30.0274698Z [submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/endpoints.submariner.io.cluster1.json
2021-07-19T04:15:30.0276247Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-19T04:15:30.0277505Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-19T04:15:30.0278824Z [submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/endpoints.submariner.io.cluster1.json
2021-07-19T04:15:30.0280141Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Endpoint
2021-07-19T04:15:30.0281191Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Endpoint
2021-07-19T04:15:30.0282957Z [submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/endpoints.submariner.io.cluster1.json
2021-07-19T04:15:30.0285012Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Endpoint
2021-07-19T04:15:30.0286991Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Endpoint
2021-07-19T04:15:30.0288665Z [submariner-operator]$ [cluster1] jq -r -M .status.acceptedNames.kind /tmp/endpoints.submariner.io.cluster1.json
2021-07-19T04:15:30.0291129Z [submariner-operator]$ [cluster1] verify_clusters_crd
2021-07-19T04:15:30.0292406Z [submariner-operator]$ [cluster1] verify_clusters_crd
2021-07-19T04:15:30.0294038Z [submariner-operator]$ [cluster1] validate_crd clusters.submariner.io v1 Cluster
2021-07-19T04:15:30.0303222Z [submariner-operator]$ [cluster1] validate_crd clusters.submariner.io v1 Cluster
2021-07-19T04:15:30.0304907Z [submariner-operator]$ [cluster1] kubectl get crds clusters.submariner.io
2021-07-19T04:15:30.0306392Z [submariner-operator]$ [cluster1] kubectl get crds clusters.submariner.io
2021-07-19T04:15:30.2265266Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd  "cable_name",
2021-07-19T04:15:30.2266062Z [cluster2]                                     "cluster_id",
2021-07-19T04:15:30.2266706Z [cluster2]                                     "hostname",
2021-07-19T04:15:30.2267218Z [cluster2]                                     "nat_enabled",
2021-07-19T04:15:30.2267709Z [cluster2]                                     "private_ip",
2021-07-19T04:15:30.2268233Z [cluster2]                                     "public_ip",
2021-07-19T04:15:30.2268859Z [cluster2]                                     "subnets"
2021-07-19T04:15:30.2269295Z [cluster2]                                 ],
2021-07-19T04:15:30.2269728Z [cluster2]                                 "type": "object"
2021-07-19T04:15:30.2270169Z [cluster2]                             }
2021-07-19T04:15:30.2270565Z [cluster2]                         },
2021-07-19T04:15:30.2270981Z [cluster2]                         "required": [
2021-07-19T04:15:30.2271650Z [cluster2]                             "spec"
2021-07-19T04:15:30.2272038Z [cluster2]                         ],
2021-07-19T04:15:30.2272473Z [cluster2]                         "type": "object"
2021-07-19T04:15:30.2272886Z [cluster2]                     }
2021-07-19T04:15:30.2273265Z [cluster2]                 },
2021-07-19T04:15:30.2273672Z [cluster2]                 "served": true,
2021-07-19T04:15:30.2274136Z [cluster2]                 "storage": true
2021-07-19T04:15:30.2274536Z [cluster2]             }
2021-07-19T04:15:30.2275078Z [cluster2]         ]
2021-07-19T04:15:30.2275637Z [cluster2]     },
2021-07-19T04:15:30.2276023Z [cluster2]     "status": {
2021-07-19T04:15:30.2276485Z [cluster2]         "acceptedNames": {
2021-07-19T04:15:30.2276966Z [cluster2]             "kind": "Endpoint",
2021-07-19T04:15:30.2277493Z [cluster2]             "listKind": "EndpointList",
2021-07-19T04:15:30.2278199Z [cluster2]             "plural": "endpoints",
2021-07-19T04:15:30.2279080Z [cluster2]             "singular": "endpoint"
2021-07-19T04:15:30.2279496Z [cluster2]         },
2021-07-19T04:15:30.2279892Z [cluster2]         "conditions": [
2021-07-19T04:15:30.2280275Z [cluster2]             {
2021-07-19T04:15:30.2281028Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:14Z",
2021-07-19T04:15:30.2281617Z [cluster2]                 "message": "no conflicts found",
2021-07-19T04:15:30.2282154Z [cluster2]                 "reason": "NoConflicts",
2021-07-19T04:15:30.2282645Z [cluster2]                 "status": "True",
2021-07-19T04:15:30.2283131Z [cluster2]                 "type": "NamesAccepted"
2021-07-19T04:15:30.2283765Z [cluster2]             },
2021-07-19T04:15:30.2284117Z [cluster2]             {
2021-07-19T04:15:30.2285005Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:14Z",
2021-07-19T04:15:30.2286066Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-19T04:15:30.2287112Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-19T04:15:30.2287686Z [cluster2]                 "status": "True",
2021-07-19T04:15:30.2288178Z [cluster2]                 "type": "Established"
2021-07-19T04:15:30.2288603Z [cluster2]             }
2021-07-19T04:15:30.2288974Z [cluster2]         ],
2021-07-19T04:15:30.2289602Z [cluster2]         "storedVersions": [
2021-07-19T04:15:30.2290049Z [cluster2]             "v1"
2021-07-19T04:15:30.2290534Z [cluster2]         ]
2021-07-19T04:15:30.2290877Z [cluster2]     }
2021-07-19T04:15:30.2291358Z [cluster2] }
2021-07-19T04:15:30.2292413Z [submariner-operator]$ [cluster2] validate_equals .metadata.name endpoints.submariner.io
2021-07-19T04:15:30.2293798Z [submariner-operator]$ [cluster2] validate_equals .metadata.name endpoints.submariner.io
2021-07-19T04:15:30.2295794Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/endpoints.submariner.io.cluster2.json
2021-07-19T04:15:30.2297774Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-19T04:15:30.2299077Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-19T04:15:30.2300464Z [submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/endpoints.submariner.io.cluster2.json
2021-07-19T04:15:30.2301910Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-19T04:15:30.2303041Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-19T04:15:30.2304467Z [submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/endpoints.submariner.io.cluster2.json
2021-07-19T04:15:30.2306071Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Endpoint
2021-07-19T04:15:30.2307390Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Endpoint
2021-07-19T04:15:30.2309053Z [submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/endpoints.submariner.io.cluster2.json
2021-07-19T04:15:30.2310667Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Endpoint
2021-07-19T04:15:30.2312255Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Endpoint
2021-07-19T04:15:30.2313968Z [submariner-operator]$ [cluster2] jq -r -M .status.acceptedNames.kind /tmp/endpoints.submariner.io.cluster2.json
2021-07-19T04:15:30.2315804Z [submariner-operator]$ [cluster2] verify_clusters_crd
2021-07-19T04:15:30.2316909Z [submariner-operator]$ [cluster2] verify_clusters_crd
2021-07-19T04:15:30.2317985Z [submariner-operator]$ [cluster2] validate_crd clusters.submariner.io v1 Cluster
2021-07-19T04:15:30.2319231Z [submariner-operator]$ [cluster2] validate_crd clusters.submariner.io v1 Cluster
2021-07-19T04:15:30.2320424Z [submariner-operator]$ [cluster2] kubectl get crds clusters.submariner.io
2021-07-19T04:15:30.2322567Z [submariner-operator]$ [cluster2] kubectl get crds clusters.submariner.io
2021-07-19T04:15:30.9045965Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds clusters.submariner.io
2021-07-19T04:15:30.9047046Z [cluster1] NAME                     CREATED AT
2021-07-19T04:15:30.9047858Z [cluster1] clusters.submariner.io   2021-07-19T04:14:00Z
2021-07-19T04:15:30.9049045Z [submariner-operator]$ [cluster1] json_file=/tmp/clusters.submariner.io.cluster1.json
2021-07-19T04:15:30.9050425Z [submariner-operator]$ [cluster1] kubectl get crd clusters.submariner.io -o json
2021-07-19T04:15:30.9051609Z [submariner-operator]$ [cluster1] kubectl get crd clusters.submariner.io -o json
2021-07-19T04:15:30.9053477Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd clusters.submariner.io -o json
2021-07-19T04:15:30.9055128Z [submariner-operator]$ [cluster1] tee /tmp/clusters.submariner.io.cluster1.json
2021-07-19T04:15:30.9056218Z [cluster1] {
2021-07-19T04:15:30.9056917Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:30.9057615Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-19T04:15:30.9058175Z [cluster1]     "metadata": {
2021-07-19T04:15:30.9058589Z [cluster1]         "annotations": {
2021-07-19T04:15:30.9059449Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-19T04:15:30.9060039Z [cluster1]         },
2021-07-19T04:15:30.9060681Z [cluster1]         "creationTimestamp": "2021-07-19T04:14:00Z",
2021-07-19T04:15:30.9061823Z [cluster1]         "generation": 1,
2021-07-19T04:15:30.9062342Z [cluster1]         "managedFields": [
2021-07-19T04:15:30.9062799Z [cluster1]             {
2021-07-19T04:15:30.9063364Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:30.9064040Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:30.9064895Z [cluster1]                 "fieldsV1": {
2021-07-19T04:15:30.9065330Z [cluster1]                     "f:metadata": {
2021-07-19T04:15:30.9065771Z [cluster1]                         "f:annotations": {
2021-07-19T04:15:30.9066206Z [cluster1]                             ".": {},
2021-07-19T04:15:30.9067044Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-19T04:15:30.9067673Z [cluster1]                         }
2021-07-19T04:15:30.9068036Z [cluster1]                     },
2021-07-19T04:15:30.9068403Z [cluster1]                     "f:spec": {
2021-07-19T04:15:30.9068842Z [cluster1]                         "f:conversion": {
2021-07-19T04:15:30.9069250Z [cluster1]                             ".": {},
2021-07-19T04:15:30.9069672Z [cluster1]                             "f:strategy": {}
2021-07-19T04:15:30.9070059Z [cluster1]                         },
2021-07-19T04:15:30.9070452Z [cluster1]                         "f:group": {},
2021-07-19T04:15:30.9070851Z [cluster1]                         "f:names": {
2021-07-19T04:15:30.9071264Z [cluster1]                             "f:kind": {},
2021-07-19T04:15:30.9072226Z [cluster1]                             "f:listKind": {},
2021-07-19T04:15:30.9072943Z [cluster1]                             "f:plural": {},
2021-07-19T04:15:30.9073613Z [cluster1]                             "f:singular": {}
2021-07-19T04:15:30.9074067Z [cluster1]                         },
2021-07-19T04:15:30.9074530Z [cluster1]                         "f:scope": {},
2021-07-19T04:15:30.9075017Z [cluster1]                         "f:versions": {}
2021-07-19T04:15:30.9075618Z [cluster1]                     },
2021-07-19T04:15:30.9076018Z [cluster1]                     "f:status": {
2021-07-19T04:15:30.9076519Z [cluster1]                         "f:acceptedNames": {
2021-07-19T04:15:30.9077312Z [cluster1]                             "f:kind": {},
2021-07-19T04:15:30.9077738Z [cluster1]                             "f:plural": {}
2021-07-19T04:15:30.9078135Z [cluster1]                         },
2021-07-19T04:15:30.9078531Z [cluster1]                         "f:conditions": {},
2021-07-19T04:15:30.9079027Z [cluster1]                         "f:storedVersions": {}
2021-07-19T04:15:30.9079453Z [cluster1]                     }
2021-07-19T04:15:30.9079807Z [cluster1]                 },
2021-07-19T04:15:30.9080692Z [cluster1]                 "manager": "submariner-operator",
2021-07-19T04:15:30.9081597Z [cluster1]                 "operation": "Update",
2021-07-19T04:15:30.9082585Z [cluster1]                 "time": "2021-07-19T04:14:06Z"
2021-07-19T04:15:30.9083012Z [cluster1]             }
2021-07-19T04:15:30.9083548Z [cluster1]         ],
2021-07-19T04:15:30.9084131Z [cluster1]         "name": "clusters.submariner.io",
2021-07-19T04:15:30.9084963Z [cluster1]         "resourceVersion": "1906",
2021-07-19T04:15:30.9086216Z [cluster1]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.submariner.io",
2021-07-19T04:15:30.9087415Z [cluster1]         "uid": "a1a068fc-6bb6-46b5-826c-e51e6d8237bf"
2021-07-19T04:15:30.9087929Z [cluster1]     },
2021-07-19T04:15:30.9088290Z [cluster1]     "spec": {
2021-07-19T04:15:30.9088828Z [cluster1]         "conversion": {
2021-07-19T04:15:30.9089272Z [cluster1]             "strategy": "None"
2021-07-19T04:15:30.9089648Z [cluster1]         },
2021-07-19T04:15:30.9090081Z [cluster1]         "group": "submariner.io",
2021-07-19T04:15:30.9090640Z [cluster1]         "names": {
2021-07-19T04:15:30.9091054Z [cluster1]             "kind": "Cluster",
2021-07-19T04:15:30.9091712Z [cluster1]             "listKind": "ClusterList",
2021-07-19T04:15:30.9092695Z [cluster1]             "plural": "clusters",
2021-07-19T04:15:30.9093234Z [cluster1]             "singular": "cluster"
2021-07-19T04:15:30.9093658Z [cluster1]         },
2021-07-19T04:15:30.9094114Z [cluster1]         "scope": "Namespaced",
2021-07-19T04:15:30.9094591Z [cluster1]         "versions": [
2021-07-19T04:15:30.9095007Z [cluster1]             {
2021-07-19T04:15:30.9095719Z [cluster1]                 "name": "v1",
2021-07-19T04:15:30.9096309Z [cluster1]                 "schema": {
2021-07-19T04:15:30.9096943Z [cluster1]                     "openAPIV3Schema": {
2021-07-19T04:15:30.9097456Z [cluster1]                         "properties": {
2021-07-19T04:15:30.9097927Z [cluster1]                             "apiVersion": {
2021-07-19T04:15:30.9129122Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-19T04:15:30.9130815Z [cluster1]                                 "type": "string"
2021-07-19T04:15:30.9131614Z [cluster1]                             },
2021-07-19T04:15:30.9132224Z [cluster1]                             "kind": {
2021-07-19T04:15:30.9134425Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-19T04:15:30.9136377Z [cluster1]                                 "type": "string"
2021-07-19T04:15:30.9136808Z [cluster1]                             },
2021-07-19T04:15:30.9137252Z [cluster1]                             "metadata": {
2021-07-19T04:15:30.9137736Z [cluster1]                                 "type": "object"
2021-07-19T04:15:30.9138169Z [cluster1]                             },
2021-07-19T04:15:30.9138592Z [cluster1]                             "spec": {
2021-07-19T04:15:30.9139052Z [cluster1]                                 "properties": {
2021-07-19T04:15:30.9139558Z [cluster1]                                     "cluster_cidr": {
2021-07-19T04:15:30.9140021Z [cluster1]                                         "items": {
2021-07-19T04:15:30.9140507Z [cluster1]                                             "type": "string"
2021-07-19T04:15:30.9141158Z [cluster1]                                         },
2021-07-19T04:15:30.9141967Z [cluster1]                                         "type": "array"
2021-07-19T04:15:30.9142867Z [cluster1]                                     },
2021-07-19T04:15:30.9143362Z [cluster1]                                     "cluster_id": {
2021-07-19T04:15:30.9144087Z [cluster1]                                         "type": "string"
2021-07-19T04:15:30.9144559Z [cluster1]                                     },
2021-07-19T04:15:30.9145039Z [cluster1]                                     "color_codes": {
2021-07-19T04:15:30.9145526Z [cluster1]                                         "items": {
2021-07-19T04:15:30.9146338Z [cluster1]                                             "type": "string"
2021-07-19T04:15:30.9146801Z [cluster1]                                         },
2021-07-19T04:15:30.9147236Z [cluster1]                                         "type": "array"
2021-07-19T04:15:30.9147687Z [cluster1]                                     },
2021-07-19T04:15:30.9148118Z [cluster1]                                     "global_cidr": {
2021-07-19T04:15:30.9148589Z [cluster1]                                         "items": {
2021-07-19T04:15:30.9149050Z [cluster1]                                             "type": "string"
2021-07-19T04:15:30.9149509Z [cluster1]                                         },
2021-07-19T04:15:30.9149941Z [cluster1]                                         "type": "array"
2021-07-19T04:15:30.9150495Z [cluster1]                                     },
2021-07-19T04:15:30.9151001Z [cluster1]                                     "service_cidr": {
2021-07-19T04:15:30.9151463Z [cluster1]                                         "items": {
2021-07-19T04:15:30.9152123Z [cluster1]                                             "type": "string"
2021-07-19T04:15:30.9152775Z [cluster1]                                         },
2021-07-19T04:15:30.9153468Z [cluster1]                                         "type": "array"
2021-07-19T04:15:30.9154163Z [cluster1]                                     }
2021-07-19T04:15:30.9154606Z [cluster1]                                 },
2021-07-19T04:15:30.9155110Z [cluster1]                                 "required": [
2021-07-19T04:15:30.9155798Z [cluster1]                                     "cluster_cidr",
2021-07-19T04:15:30.9156336Z [cluster1]                                     "cluster_id",
2021-07-19T04:15:30.9157193Z [cluster1]                                     "color_codes",
2021-07-19T04:15:30.9157674Z [cluster1]                                     "global_cidr",
2021-07-19T04:15:30.9158139Z [cluster1]                                     "service_cidr"
2021-07-19T04:15:30.9158583Z [cluster1]                                 ],
2021-07-19T04:15:30.9159162Z [cluster1]                                 "type": "object"
2021-07-19T04:15:30.9159591Z [cluster1]                             }
2021-07-19T04:15:30.9159977Z [cluster1]                         },
2021-07-19T04:15:30.9160489Z [cluster1]                         "required": [
2021-07-19T04:15:30.9161095Z [cluster1]                             "spec"
2021-07-19T04:15:30.9161496Z [cluster1]                         ],
2021-07-19T04:15:30.9161944Z [cluster1]                         "type": "object"
2021-07-19T04:15:30.9162370Z [cluster1]                     }
2021-07-19T04:15:30.9162760Z [cluster1]                 },
2021-07-19T04:15:30.9163177Z [cluster1]                 "served": true,
2021-07-19T04:15:30.9163657Z [cluster1]                 "storage": true
2021-07-19T04:15:30.9164080Z [cluster1]             }
2021-07-19T04:15:30.9164434Z [cluster1]         ]
2021-07-19T04:15:30.9164899Z [cluster1]     },
2021-07-19T04:15:30.9165277Z [cluster1]     "status": {
2021-07-19T04:15:30.9165750Z [cluster1]         "acceptedNames": {
2021-07-19T04:15:30.9166237Z [cluster1]             "kind": "Cluster",
2021-07-19T04:15:30.9166760Z [cluster1]             "listKind": "ClusterList",
2021-07-19T04:15:30.9167292Z [cluster1]             "plural": "clusters",
2021-07-19T04:15:30.9167805Z [cluster1]             "singular": "cluster"
2021-07-19T04:15:30.9168227Z [cluster1]         },
2021-07-19T04:15:30.9168644Z [cluster1]         "conditions": [
2021-07-19T04:15:30.9169051Z [cluster1]             {
2021-07-19T04:15:31.1289227Z [cluster1]                 "lastTransitionTime": "2021-07-19Ts clusters.submariner.io
2021-07-19T04:15:31.1291335Z [cluster2] NAME                     CREATED AT
2021-07-19T04:15:31.1292161Z [cluster2] clusters.submariner.io   2021-07-19T04:14:14Z
2021-07-19T04:15:31.1293279Z [submariner-operator]$ [cluster2] json_file=/tmp/clusters.submariner.io.cluster2.json
2021-07-19T04:15:31.1294790Z [submariner-operator]$ [cluster2] kubectl get crd clusters.submariner.io -o json
2021-07-19T04:15:31.1296415Z [submariner-operator]$ [cluster2] kubectl get crd clusters.submariner.io -o json
2021-07-19T04:15:31.1297951Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd clusters.submariner.io -o json
2021-07-19T04:15:31.1299447Z [submariner-operator]$ [cluster2] tee /tmp/clusters.submariner.io.cluster2.json
2021-07-19T04:15:31.1300564Z [cluster2] {
2021-07-19T04:15:31.1301088Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:31.1301782Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-19T04:15:31.1302345Z [cluster2]     "metadata": {
2021-07-19T04:15:31.1302767Z [cluster2]         "annotations": {
2021-07-19T04:15:31.1303822Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-19T04:15:31.1304454Z [cluster2]         },
2021-07-19T04:15:31.1305135Z [cluster2]         "creationTimestamp": "2021-07-19T04:14:14Z",
2021-07-19T04:15:31.1305642Z [cluster2]         "generation": 1,
2021-07-19T04:15:31.1306114Z [cluster2]         "managedFields": [
2021-07-19T04:15:31.1306535Z [cluster2]             {
2021-07-19T04:15:31.1307425Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:31.1308051Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:31.1308512Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:31.1308948Z [cluster2]                     "f:metadata": {
2021-07-19T04:15:31.1309399Z [cluster2]                         "f:annotations": {
2021-07-19T04:15:31.1309831Z [cluster2]                             ".": {},
2021-07-19T04:15:31.1310785Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-19T04:15:31.1311400Z [cluster2]                         }
2021-07-19T04:15:31.1311745Z [cluster2]                     },
2021-07-19T04:15:31.1312132Z [cluster2]                     "f:spec": {
2021-07-19T04:15:31.1312559Z [cluster2]                         "f:conversion": {
2021-07-19T04:15:31.1312958Z [cluster2]                             ".": {},
2021-07-19T04:15:31.1313364Z [cluster2]                             "f:strategy": {}
2021-07-19T04:15:31.1313746Z [cluster2]                         },
2021-07-19T04:15:31.1314316Z [cluster2]                         "f:group": {},
2021-07-19T04:15:31.1314708Z [cluster2]                         "f:names": {
2021-07-19T04:15:31.1315108Z [cluster2]                             "f:kind": {},
2021-07-19T04:15:31.1315516Z [cluster2]                             "f:listKind": {},
2021-07-19T04:15:31.1315951Z [cluster2]                             "f:plural": {},
2021-07-19T04:15:31.1316641Z [cluster2]                             "f:singular": {}
2021-07-19T04:15:31.1317416Z [cluster2]                         },
2021-07-19T04:15:31.1318021Z [cluster2]                         "f:scope": {},
2021-07-19T04:15:31.1318667Z [cluster2]                         "f:versions": {}
2021-07-19T04:15:31.1319127Z [cluster2]                     },
2021-07-19T04:15:31.1319713Z [cluster2]                     "f:status": {
2021-07-19T04:15:31.1320394Z [cluster2]                         "f:acceptedNames": {
2021-07-19T04:15:31.1321033Z [cluster2]                             "f:kind": {},
2021-07-19T04:15:31.1321614Z [cluster2]                             "f:plural": {}
2021-07-19T04:15:31.1321985Z [cluster2]                         },
2021-07-19T04:15:31.1322399Z [cluster2]                         "f:conditions": {},
2021-07-19T04:15:31.1322884Z [cluster2]                         "f:storedVersions": {}
2021-07-19T04:15:31.1323302Z [cluster2]                     }
2021-07-19T04:15:31.1323651Z [cluster2]                 },
2021-07-19T04:15:31.1324344Z [cluster2]                 "manager": "submariner-operator",
2021-07-19T04:15:31.1326109Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:31.1327047Z [cluster2]                 "time": "2021-07-19T04:14:26Z"
2021-07-19T04:15:31.1327532Z [cluster2]             }
2021-07-19T04:15:31.1327940Z [cluster2]         ],
2021-07-19T04:15:31.1328763Z [cluster2]         "name": "clusters.submariner.io",
2021-07-19T04:15:31.1329619Z [cluster2]         "resourceVersion": "1914",
2021-07-19T04:15:31.1331319Z [cluster2]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.submariner.io",
2021-07-19T04:15:31.1332624Z [cluster2]         "uid": "fd3c7ff0-bc42-42ac-8023-4bf2ef6db230"
2021-07-19T04:15:31.1333360Z [cluster2]     },
2021-07-19T04:15:31.1333723Z [cluster2]     "spec": {
2021-07-19T04:15:31.1334119Z [cluster2]         "conversion": {
2021-07-19T04:15:31.1334723Z [cluster2]             "strategy": "None"
2021-07-19T04:15:31.1335094Z [cluster2]         },
2021-07-19T04:15:31.1336042Z [cluster2]         "group": "submariner.io",
2021-07-19T04:15:31.1336686Z [cluster2]         "names": {
2021-07-19T04:15:31.1337117Z [cluster2]             "kind": "Cluster",
2021-07-19T04:15:31.1337606Z [cluster2]             "listKind": "ClusterList",
2021-07-19T04:15:31.1338343Z [cluster2]             "plural": "clusters",
2021-07-19T04:15:31.1338845Z [cluster2]             "singular": "cluster"
2021-07-19T04:15:31.1339264Z [cluster2]         },
2021-07-19T04:15:31.1339710Z [cluster2]         "scope": "Namespaced",
2021-07-19T04:15:31.1340190Z [cluster2]         "versions": [
2021-07-19T04:15:31.1340596Z [cluster2]             {
2021-07-19T04:15:31.1341295Z [cluster2]                 "name": "v1",
2021-07-19T04:15:31.1341850Z [cluster2]                 "schema": {
2021-07-19T04:15:31.1342298Z [cluster2]                     "openAPIV3Schema": {
2021-07-19T04:15:31.1342791Z [cluster2]                         "properties": {
2021-07-19T04:15:31.1343242Z [cluster2]                             "apiVersion": {
2021-07-19T04:15:31.1366195Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-19T04:15:31.1368828Z [cluster2]                                 "type": "string"
2021-07-19T04:15:31.1369319Z [cluster2]                             },
2021-07-19T04:15:31.1370119Z [cluster2]                             "kind": {
2021-07-19T04:15:31.1372661Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-19T04:15:31.1375016Z [cluster2]                                 "type": "string"
2021-07-19T04:15:31.1375455Z [cluster2]                             },
2021-07-19T04:15:31.1376039Z [cluster2]                             "metadata": {
2021-07-19T04:15:31.1376701Z [cluster2]                                 "type": "object"
2021-07-19T04:15:31.1377728Z [cluster2]                             },
2021-07-19T04:15:31.1378150Z [cluster2]                             "spec": {
2021-07-19T04:15:31.1378812Z [cluster2]                                 "properties": {
2021-07-19T04:15:31.1379345Z [cluster2]                                     "cluster_cidr": {
2021-07-19T04:15:31.1379832Z [cluster2]                                         "items": {
2021-07-19T04:15:31.1380651Z [cluster2]                                             "type": "string"
2021-07-19T04:15:31.1381086Z [cluster2]                                         },
2021-07-19T04:15:31.1381531Z [cluster2]                                         "type": "array"
2021-07-19T04:15:31.1382133Z [cluster2]                                     },
2021-07-19T04:15:31.1382548Z [cluster2]                                     "cluster_id": {
2021-07-19T04:15:31.1383010Z [cluster2]                                         "type": "string"
2021-07-19T04:15:31.1383596Z [cluster2]                                     },
2021-07-19T04:15:31.1384009Z [cluster2]                                     "color_codes": {
2021-07-19T04:15:31.1384427Z [cluster2]                                         "items": {
2021-07-19T04:15:31.1384883Z [cluster2]                                             "type": "string"
2021-07-19T04:15:31.1385292Z [cluster2]                                         },
2021-07-19T04:15:31.1385712Z [cluster2]                                         "type": "array"
2021-07-19T04:15:31.1386126Z [cluster2]                                     },
2021-07-19T04:15:31.1386888Z [cluster2]                                     "global_cidr": {
2021-07-19T04:15:31.1387743Z [cluster2]                                         "items": {
2021-07-19T04:15:31.1388319Z [cluster2]                                             "type": "string"
2021-07-19T04:15:31.1388791Z [cluster2]                                         },
2021-07-19T04:15:31.1389219Z [cluster2]                                         "type": "array"
2021-07-19T04:15:31.1389654Z [cluster2]                                     },
2021-07-19T04:15:31.1390091Z [cluster2]                                     "service_cidr": {
2021-07-19T04:15:31.1390878Z [cluster2]                                         "items": {
2021-07-19T04:15:31.1391324Z [cluster2]                                             "type": "string"
2021-07-19T04:15:31.1391733Z [cluster2]                                         },
2021-07-19T04:15:31.1392147Z [cluster2]                                         "type": "array"
2021-07-19T04:15:31.1392545Z [cluster2]                                     }
2021-07-19T04:15:31.1392919Z [cluster2]                                 },
2021-07-19T04:15:31.1393315Z [cluster2]                                 "required": [
2021-07-19T04:15:31.1393766Z [cluster2]                                     "cluster_cidr",
2021-07-19T04:15:31.1394215Z [cluster2]                                     "cluster_id",
2021-07-19T04:15:31.1394651Z [cluster2]                                     "color_codes",
2021-07-19T04:15:31.1395091Z [cluster2]                                     "global_cidr",
2021-07-19T04:15:31.1395524Z [cluster2]                                     "service_cidr"
2021-07-19T04:15:31.1396030Z [cluster2]                                 ],
2021-07-19T04:15:31.1396609Z [cluster2]                                 "type": "object"
2021-07-19T04:15:31.1397415Z [cluster2]                             }
2021-07-19T04:15:31.1398169Z [cluster2]                         },
2021-07-19T04:15:31.1398602Z [cluster2]                         "required": [
2021-07-19T04:15:31.1399035Z [cluster2]                             "spec"
2021-07-19T04:15:31.1399447Z [cluster2]                         ],
2021-07-19T04:15:31.1399892Z [cluster2]                         "type": "object"
2021-07-19T04:15:31.1400318Z [cluster2]                     }
2021-07-19T04:15:31.1401171Z [cluster2]                 },
2021-07-19T04:15:31.1401548Z [cluster2]                 "served": true,
2021-07-19T04:15:31.1401977Z [cluster2]                 "storage": true
2021-07-19T04:15:31.1402345Z [cluster2]             }
2021-07-19T04:15:31.1402692Z [cluster2]         ]
2021-07-19T04:15:31.1403011Z [cluster2]     },
2021-07-19T04:15:31.1403369Z [cluster2]     "status": {
2021-07-19T04:15:31.1403782Z [cluster2]         "acceptedNames": {
2021-07-19T04:15:31.1404231Z [cluster2]             "kind": "Cluster",
2021-07-19T04:15:31.1405100Z [cluster2]             "listKind": "ClusterList",
2021-07-19T04:15:31.1405622Z [cluster2]             "plural": "clusters",
2021-07-19T04:15:31.1406299Z [cluster2]             "singular": "cluster"
2021-07-19T04:15:31.1406717Z [cluster2]         },
2021-07-19T04:15:31.1407341Z [cluster2]         "conditions": [
2021-07-19T04:15:31.1407765Z [cluster2]             {
2021-07-19T04:15:32.1040765Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:00Z",
2021-07-19T04:15:32.1041814Z [cluster1]                 "message": "no conflicts found",
2021-07-19T04:15:32.1043008Z [cluster1]                 "reason": "NoConflicts",
2021-07-19T04:15:32.1043692Z [cluster1]                 "status": "True",
2021-07-19T04:15:32.1044479Z [cluster1]                 "type": "NamesAccepted"
2021-07-19T04:15:32.1045003Z [cluster1]             },
2021-07-19T04:15:32.1045414Z [cluster1]             {
2021-07-19T04:15:32.1046364Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:14:00Z",
2021-07-19T04:15:32.1047081Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-19T04:15:32.1048016Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-19T04:15:32.1048709Z [cluster1]                 "status": "True",
2021-07-19T04:15:32.1049462Z [cluster1]                 "type": "Established"
2021-07-19T04:15:32.1050036Z [cluster1]             }
2021-07-19T04:15:32.1050420Z [cluster1]         ],
2021-07-19T04:15:32.1050873Z [cluster1]         "storedVersions": [
2021-07-19T04:15:32.1051348Z [cluster1]             "v1"
2021-07-19T04:15:32.1051727Z [cluster1]         ]
2021-07-19T04:15:32.1052103Z [cluster1]     }
2021-07-19T04:15:32.1052458Z [cluster1] }
2021-07-19T04:15:32.1053532Z [submariner-operator]$ [cluster1] validate_equals .metadata.name clusters.submariner.io
2021-07-19T04:15:32.1054944Z [submariner-operator]$ [cluster1] validate_equals .metadata.name clusters.submariner.io
2021-07-19T04:15:32.1056992Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/clusters.submariner.io.cluster1.json
2021-07-19T04:15:32.1059158Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-19T04:15:32.1060350Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-19T04:15:32.1062156Z [submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/clusters.submariner.io.cluster1.json
2021-07-19T04:15:32.1063529Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-19T04:15:32.1064681Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-19T04:15:32.1066189Z [submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/clusters.submariner.io.cluster1.json
2021-07-19T04:15:32.1067598Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Cluster
2021-07-19T04:15:32.1068936Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Cluster
2021-07-19T04:15:32.1070365Z [submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/clusters.submariner.io.cluster1.json
2021-07-19T04:15:32.1072076Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Cluster
2021-07-19T04:15:32.1073435Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Cluster
2021-07-19T04:15:32.1075146Z [submariner-operator]$ [cluster1] jq -r -M .status.acceptedNames.kind /tmp/clusters.submariner.io.cluster1.json
2021-07-19T04:15:32.1076689Z [submariner-operator]$ [cluster1] verify_subm_crd
2021-07-19T04:15:32.1077528Z [submariner-operator]$ [cluster1] verify_subm_crd
2021-07-19T04:15:32.1079476Z [submariner-operator]$ [cluster1] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-19T04:15:32.1081058Z [submariner-operator]$ [cluster1] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-19T04:15:32.1082622Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-19T04:15:32.1083862Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-19T04:15:32.1085201Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds submariners.submariner.io
2021-07-19T04:15:32.1086079Z [cluster1] NAME                        CREATED AT
2021-07-19T04:15:32.1086847Z [cluster1] submariners.submariner.io   2021-07-19T04:13:50Z
2021-07-19T04:15:32.1088303Z [submariner-operator]$ [cluster1] json_file=/tmp/submariners.submariner.io.cluster1.json
2021-07-19T04:15:32.1089774Z [submariner-operator]$ [cluster1] kubectl get crd submariners.submariner.io -o json
2021-07-19T04:15:32.1091253Z [submariner-operator]$ [cluster1] tee /tmp/submariners.submariner.io.cluster1.json
2021-07-19T04:15:32.1092735Z [submariner-operator]$ [cluster1] kubectl get crd submariners.submariner.io -o json
2021-07-19T04:15:32.1094159Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd submariners.submariner.io -o json
2021-07-19T04:15:32.1095000Z [cluster1] {
2021-07-19T04:15:32.1103538Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:32.1104519Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-19T04:15:32.1105249Z [cluster1]     "metadata": {
2021-07-19T04:15:32.1105840Z [cluster1]         "annotations": {
2021-07-19T04:15:32.1106843Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-19T04:15:32.1107584Z [cluster1]         },
2021-07-19T04:15:32.1108295Z [cluster1]         "creationTimestamp": "2021-07-19T04:13:50Z",
2021-07-19T04:15:32.1108823Z [cluster1]         "generation": 1,
2021-07-19T04:15:32.1109493Z [cluster1]         "managedFields": [
2021-07-19T04:15:32.1110056Z [cluster1]             {
2021-07-19T04:15:32.1110775Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:32.1111398Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:32.1111893Z [cluster1]                 "fieldsV1": {
2021-07-19T04:15:32.1112445Z [cluster1]                     "f:metadata": {
2021-07-19T04:15:32.1146897Z [cluster1]                         "f:annotations": {
2021-07-19T04:15:32.1147517Z [cluster1]                             ".": {},
2021-07-19T04:15:32.1148703Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-19T04:15:32.1149509Z [cluster1]                         }
2021-07-19T04:15:32.1150030Z [cluster1]                     },
2021-07-19T04:15:32.1150761Z [cluster1]                     "f:spec": {
2021-07-19T04:15:32.1151349Z [cluster1]                         "f:conversion": {
2021-07-19T04:15:32.1151820Z [cluster1]                             ".": {},
2021-07-19T04:15:32.1152429Z [cluster1]                             "f:strategy": {}
2021-07-19T04:15:32.1153173Z [cluster1]                         },
2021-07-19T04:15:32.1153775Z [cluster1]                         "f:group": {},
2021-07-19T04:15:32.1154194Z [cluster1]                         "f:names": {
2021-07-19T04:15:32.1154620Z [cluster1]                             "f:kind": {},
2021-07-19T04:15:32.1155053Z [cluster1]                             "f:listKind": {},
2021-07-19T04:15:32.1156195Z [cluster1]                             "f:plural": {},
2021-07-19T04:15:32.1156992Z [cluster1]                             "f:singular": {}
2021-07-19T04:15:32.1157402Z [cluster1]                         },
2021-07-19T04:15:32.1157783Z [cluster1]                         "f:scope": {},
2021-07-19T04:15:32.1158218Z [cluster1]                         "f:versions": {}
2021-07-19T04:15:32.1158610Z [cluster1]                     },
2021-07-19T04:15:32.1159009Z [cluster1]                     "f:status": {
2021-07-19T04:15:32.1159478Z [cluster1]                         "f:acceptedNames": {
2021-07-19T04:15:32.1160000Z [cluster1]                             "f:kind": {},
2021-07-19T04:15:32.1160796Z [cluster1]                             "f:plural": {}
2021-07-19T04:15:32.1161204Z [cluster1]                         },
2021-07-19T04:15:32.1161651Z [cluster1]                         "f:conditions": {},
2021-07-19T04:15:32.1162169Z [cluster1]                         "f:storedVersions": {}
2021-07-19T04:15:32.1162640Z [cluster1]                     }
2021-07-19T04:15:32.1163013Z [cluster1]                 },
2021-07-19T04:15:32.1163618Z [cluster1]                 "manager": "subctl",
2021-07-19T04:15:32.1164264Z [cluster1]                 "operation": "Update",
2021-07-19T04:15:32.1164997Z [cluster1]                 "time": "2021-07-19T04:14:02Z"
2021-07-19T04:15:32.1165406Z [cluster1]             }
2021-07-19T04:15:32.1165742Z [cluster1]         ],
2021-07-19T04:15:32.1166281Z [cluster1]         "name": "submariners.submariner.io",
2021-07-19T04:15:32.1166901Z [cluster1]         "resourceVersion": "1863",
2021-07-19T04:15:32.1167871Z [cluster1]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/submariners.submariner.io",
2021-07-19T04:15:32.1169050Z [cluster1]         "uid": "4202ca53-4e40-48a8-85c5-8a01acfbdad2"
2021-07-19T04:15:32.1169543Z [cluster1]     },
2021-07-19T04:15:32.1169890Z [cluster1]     "spec": {
2021-07-19T04:15:32.1170303Z [cluster1]         "conversion": {
2021-07-19T04:15:32.1170943Z [cluster1]             "strategy": "None"
2021-07-19T04:15:32.1171818Z [cluster1]         },
2021-07-19T04:15:32.1172573Z [cluster1]         "group": "submariner.io",
2021-07-19T04:15:32.1173067Z [cluster1]         "names": {
2021-07-19T04:15:32.1173555Z [cluster1]             "kind": "Submariner",
2021-07-19T04:15:32.1174139Z [cluster1]             "listKind": "SubmarinerList",
2021-07-19T04:15:32.1175063Z [cluster1]             "plural": "submariners",
2021-07-19T04:15:32.1176165Z [cluster1]             "singular": "submariner"
2021-07-19T04:15:32.1176827Z [cluster1]         },
2021-07-19T04:15:32.1177239Z [cluster1]         "scope": "Namespaced",
2021-07-19T04:15:32.1177694Z [cluster1]         "versions": [
2021-07-19T04:15:32.1178068Z [cluster1]             {
2021-07-19T04:15:32.1178473Z [cluster1]                 "name": "v1alpha1",
2021-07-19T04:15:32.1178901Z [cluster1]                 "schema": {
2021-07-19T04:15:32.1179364Z [cluster1]                     "openAPIV3Schema": {
2021-07-19T04:15:32.1180040Z [cluster1]                         "description": "Submariner is the Schema for the submariners API",
2021-07-19T04:15:32.1181216Z [cluster1]                         "properties": {
2021-07-19T04:15:32.1181908Z [cluster1]                             "apiVersion": {
2021-07-19T04:15:32.1184488Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-19T04:15:32.1186203Z [cluster1]                                 "type": "string"
2021-07-19T04:15:32.1186631Z [cluster1]                             },
2021-07-19T04:15:32.1188034Z [cluster1]                             "kind": {
2021-07-19T04:15:32.1190054Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-19T04:15:32.1202864Z [cluster1]                                 "type": "string"
2021-07-19T04:15:32.1203508Z [cluster1]                             },
2021-07-19T04:15:32.1204555Z [cluster1]                             "metadata": {
2021-07-19T04:15:32.1205206Z [cluster1]                                 "type": "object"
2021-07-19T04:15:32.1205655Z [cluster1]                             },
2021-07-19T04:15:32.1212819Z [cluster1]                             "spec": {
2021-07-19T04:15:32.1213783Z [cluster1]                                 "description": "SubmarinerSpec defines the desired state of Submariner",
2021-07-19T04:15:32.1214514Z [cluster1]                                 "properties": {
2021-07-19T04:15:32.1215177Z [cluster1]                                     "broker": {
2021-07-19T04:15:32.1215850Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1216305Z [cluster1]                                     },
2021-07-19T04:15:32.1216848Z [cluster1]                                     "brokerK8sApiServer": {
2021-07-19T04:15:32.1217580Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1223998Z [cluster1]                                     },
2021-07-19T04:15:32.1224777Z [cluster1]                                     "brokerK8sApiServerToken": {
2021-07-19T04:15:32.1225370Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1225802Z [cluster1]                                     },
2021-07-19T04:15:32.1226228Z [cluster1]                                     "brokerK8sCA": {
2021-07-19T04:15:32.1226712Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1227142Z [cluster1]                                     },
2021-07-19T04:15:32.1228169Z [cluster1]                                     "brokerK8sRemoteNamespace": {
2021-07-19T04:15:32.1230301Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1230822Z [cluster1]                                     },
2021-07-19T04:15:32.1231371Z [cluster1]                                     "cableDriver": {
2021-07-19T04:15:32.1232306Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1233474Z [cluster1]                                     },
2021-07-19T04:15:32.1234099Z [cluster1]                                     "ceIPSecDebug": {
2021-07-19T04:15:32.1234608Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.1235052Z [cluster1]                                     },
2021-07-19T04:15:32.1235750Z [cluster1]                                     "ceIPSecForceUDPEncaps": {
2021-07-19T04:15:32.1239045Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.1239991Z [cluster1]                                     },
2021-07-19T04:15:32.1241346Z [cluster1]                                     "ceIPSecIKEPort": {
2021-07-19T04:15:32.1241930Z [cluster1]                                         "type": "integer"
2021-07-19T04:15:32.1242584Z [cluster1]                                     },
2021-07-19T04:15:32.1243078Z [cluster1]                                     "ceIPSecNATTPort": {
2021-07-19T04:15:32.1243971Z [cluster1]                                         "type": "integer"
2021-07-19T04:15:32.1244578Z [cluster1]                                     },
2021-07-19T04:15:32.1245014Z [cluster1]                                     "ceIPSecPSK": {
2021-07-19T04:15:32.1245496Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1245911Z [cluster1]                                     },
2021-07-19T04:15:32.1246462Z [cluster1]                                     "ceIPSecPreferredServer": {
2021-07-19T04:15:32.1247058Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.1247497Z [cluster1]                                     },
2021-07-19T04:15:32.1247943Z [cluster1]                                     "clusterCIDR": {
2021-07-19T04:15:32.1248415Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1248845Z [cluster1]                                     },
2021-07-19T04:15:32.1249262Z [cluster1]                                     "clusterID": {
2021-07-19T04:15:32.1249737Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1250342Z [cluster1]                                     },
2021-07-19T04:15:32.1250805Z [cluster1]                                     "colorCodes": {
2021-07-19T04:15:32.1251296Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1251747Z [cluster1]                                     },
2021-07-19T04:15:32.1252383Z [cluster1]                                     "connectionHealthCheck": {
2021-07-19T04:15:32.1253002Z [cluster1]                                         "properties": {
2021-07-19T04:15:32.1253819Z [cluster1]                                             "enabled": {
2021-07-19T04:15:32.1254284Z [cluster1]                                                 "type": "boolean"
2021-07-19T04:15:32.1254731Z [cluster1]                                             },
2021-07-19T04:15:32.1255202Z [cluster1]                                             "intervalSeconds": {
2021-07-19T04:15:32.1256465Z [cluster1]                                                 "description": "The interval at which health check pings are sent.",
2021-07-19T04:15:32.1257283Z [cluster1]                                                 "format": "int64",
2021-07-19T04:15:32.1257763Z [cluster1]                                                 "type": "integer"
2021-07-19T04:15:32.1258691Z [cluster1]                                             },
2021-07-19T04:15:32.1259521Z [cluster1]                                             "maxPacketLossCount": {
2021-07-19T04:15:32.1260637Z [cluster1]                                                 "description": "The maximum number of packets lost at which the health checker will mark the connection as down.",
2021-07-19T04:15:32.1261604Z [cluster1]                                                 "format": "int64",
2021-07-19T04:15:32.1262136Z [cluster1]                                                 "type": "integer"
2021-07-19T04:15:32.1262611Z [cluster1]                                             }
2021-07-19T04:15:32.1263026Z [cluster1]                                         },
2021-07-19T04:15:32.1263811Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.1264758Z [cluster1]                                     },
2021-07-19T04:15:32.1265835Z [cluster1]                                     "coreDNSCustomConfig": {
2021-07-19T04:15:32.1266625Z [cluster1]                                         "properties": {
2021-07-19T04:15:32.1267440Z [cluster1]                                             "configMapName": {
2021-07-19T04:15:32.1268190Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1268660Z [cluster1]                                             },
2021-07-19T04:15:32.1269148Z [cluster1]                                             "namespace": {
2021-07-19T04:15:32.1269669Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1270918Z [cluster1]                                             }
2021-07-19T04:15:32.1271541Z [cluster1]                                         },
2021-07-19T04:15:32.1272255Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.1273112Z [cluster1]                                     },
2021-07-19T04:15:32.1274140Z [cluster1]                                     "customDomains": {
2021-07-19T04:15:32.1274867Z [cluster1]                                         "items": {
2021-07-19T04:15:32.1275699Z [cluster1]                                             "type": "string"
2021-07-19T04:15:32.1277402Z [cluster1]                                         },
2021-07-19T04:15:32.1278627Z [cluster1]                                         "type": "array",
2021-07-19T04:15:32.1280735Z [cluster1]                                         "x-kubernetes-list-type": "set"
2021-07-19T04:15:32.1281573Z [cluster1]                                     },
2021-07-19T04:15:32.1282079Z [cluster1]                                     "debug": {
2021-07-19T04:15:32.1282649Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.1283163Z [cluster1]                                     },
2021-07-19T04:15:32.1283694Z [cluster1]                                     "globalCIDR": {
2021-07-19T04:15:32.1285405Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1286929Z [cluster1]                                     },
2021-07-19T04:15:32.1287880Z [cluster1]                                     "imageOverrides": {
2021-07-19T04:15:32.1336310Z [cluster1]                                         "additionalProperties": {
2021-07-19T04:15:32.1336906Z [cluster1]                                             "type": "string"
2021-07-19T04:15:32.1337367Z [cluster1]                                         },
2021-07-19T04:15:32.1338068Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.1338514Z [cluster1]                                     },
2021-07-19T04:15:32.1339068Z [cluster1]                                     "loadBalancerEnabled": {
2021-07-19T04:15:32.1339645Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.1340496Z [cluster1]                                     },
2021-07-19T04:15:32.1341527Z [cluster1]                                     "namespace": {
2021-07-19T04:15:32.1342231Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1342894Z [cluster1]                                     },
2021-07-19T04:15:32.1343386Z [cluster1]                                     "natEnabled": {
2021-07-19T04:15:32.1344236Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.1344673Z [cluster1]                                     },
2021-07-19T04:15:32.1345129Z [cluster1]                                     "repository": {
2021-07-19T04:15:32.1345606Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1346055Z [cluster1]                                     },
2021-07-19T04:15:32.1346498Z [cluster1]                                     "serviceCIDR": {
2021-07-19T04:15:32.1346996Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1347611Z [cluster1]                                     },
2021-07-19T04:15:32.1348179Z [cluster1]                                     "serviceDiscoveryEnabled": {
2021-07-19T04:15:32.1348819Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.1349269Z [cluster1]                                     },
2021-07-19T04:15:32.1349712Z [cluster1]                                     "version": {
2021-07-19T04:15:32.1350190Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1350646Z [cluster1]                                     }
2021-07-19T04:15:32.1351039Z [cluster1]                                 },
2021-07-19T04:15:32.1351489Z [cluster1]                                 "required": [
2021-07-19T04:15:32.1352199Z [cluster1]                                     "broker",
2021-07-19T04:15:32.1352756Z [cluster1]                                     "brokerK8sApiServer",
2021-07-19T04:15:32.1353517Z [cluster1]                                     "brokerK8sApiServerToken",
2021-07-19T04:15:32.1354140Z [cluster1]                                     "brokerK8sCA",
2021-07-19T04:15:32.1354786Z [cluster1]                                     "brokerK8sRemoteNamespace",
2021-07-19T04:15:32.1355438Z [cluster1]                                     "ceIPSecDebug",
2021-07-19T04:15:32.1355982Z [cluster1]                                     "ceIPSecPSK",
2021-07-19T04:15:32.1356494Z [cluster1]                                     "clusterCIDR",
2021-07-19T04:15:32.1357005Z [cluster1]                                     "clusterID",
2021-07-19T04:15:32.1357483Z [cluster1]                                     "debug",
2021-07-19T04:15:32.1357947Z [cluster1]                                     "namespace",
2021-07-19T04:15:32.1358460Z [cluster1]                                     "natEnabled",
2021-07-19T04:15:32.1358964Z [cluster1]                                     "serviceCIDR"
2021-07-19T04:15:32.1359426Z [cluster1]                                 ],
2021-07-19T04:15:32.1360035Z [cluster1]                                 "type": "object"
2021-07-19T04:15:32.1360466Z [cluster1]                             },
2021-07-19T04:15:32.1361246Z [cluster1]                             "status": {
2021-07-19T04:15:32.1362119Z [cluster1]                                 "description": "SubmarinerStatus defines the observed state of Submariner",
2021-07-19T04:15:32.1416587Z [cluster1]                                 "properties": {
2021-07-19T04:15:32.1417075Z [cluster1]                                     "clusterCIDR": {
2021-07-19T04:15:32.1417565Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1417980Z [cluster1]                                     },
2021-07-19T04:15:32.1418426Z [cluster1]                                     "clusterID": {
2021-07-19T04:15:32.1418880Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1419311Z [cluster1]                                     },
2021-07-19T04:15:32.1419747Z [cluster1]                                     "colorCodes": {
2021-07-19T04:15:32.1420385Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1421050Z [cluster1]                                     },
2021-07-19T04:15:32.1421655Z [cluster1]                                     "deploymentInfo": {
2021-07-19T04:15:32.1422211Z [cluster1]                                         "properties": {
2021-07-19T04:15:32.1422727Z [cluster1]                                             "cloudProvider": {
2021-07-19T04:15:32.1423257Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1423855Z [cluster1]                                             },
2021-07-19T04:15:32.1424331Z [cluster1]                                             "kubernetesType": {
2021-07-19T04:15:32.1424836Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1425246Z [cluster1]                                             },
2021-07-19T04:15:32.1425774Z [cluster1]                                             "kubernetesTypeVersion": {
2021-07-19T04:15:32.1426334Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1426766Z [cluster1]                                             },
2021-07-19T04:15:32.1427235Z [cluster1]                                             "kubernetesVersion": {
2021-07-19T04:15:32.1427762Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1428187Z [cluster1]                                             }
2021-07-19T04:15:32.1428564Z [cluster1]                                         },
2021-07-19T04:15:32.1428987Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.1429513Z [cluster1]                                     },
2021-07-19T04:15:32.1430347Z [cluster1]                                     "gatewayDaemonSetStatus": {
2021-07-19T04:15:32.1432270Z [cluster1]                                         "properties": {
2021-07-19T04:15:32.1432986Z [cluster1]                                             "lastResourceVersion": {
2021-07-19T04:15:32.1433674Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1434218Z [cluster1]                                             },
2021-07-19T04:15:32.1435214Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-19T04:15:32.1436032Z [cluster1]                                                 "type": "boolean"
2021-07-19T04:15:32.1436659Z [cluster1]                                             },
2021-07-19T04:15:32.1437344Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-19T04:15:32.1437952Z [cluster1]                                                 "items": {
2021-07-19T04:15:32.1439007Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-19T04:15:32.1440481Z [cluster1]                                                     "properties": {
2021-07-19T04:15:32.1441030Z [cluster1]                                                         "running": {
2021-07-19T04:15:32.1441690Z [cluster1]                                                             "description": "Details about a running container",
2021-07-19T04:15:32.1442386Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.1442942Z [cluster1]                                                                 "startedAt": {
2021-07-19T04:15:32.1444338Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-19T04:15:32.1445229Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.1445762Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1446214Z [cluster1]                                                                 }
2021-07-19T04:15:32.1446925Z [cluster1]                                                             },
2021-07-19T04:15:32.1447385Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.1447835Z [cluster1]                                                         },
2021-07-19T04:15:32.1448272Z [cluster1]                                                         "terminated": {
2021-07-19T04:15:32.1448899Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-19T04:15:32.1449530Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.1450026Z [cluster1]                                                                 "containerID": {
2021-07-19T04:15:32.1451624Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-19T04:15:32.1452392Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1452915Z [cluster1]                                                                 },
2021-07-19T04:15:32.1453777Z [cluster1]                                                                 "exitCode": {
2021-07-19T04:15:32.1454681Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-19T04:15:32.1536592Z [cluster1]                                                                     "format": "int32",
2021-07-19T04:15:32.1537592Z [cluster1]                                                                     "type": "integer"
2021-07-19T04:15:32.1538043Z [cluster1]                                                                 },
2021-07-19T04:15:32.1538504Z [cluster1]                                                                 "finishedAt": {
2021-07-19T04:15:32.1539143Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-19T04:15:32.1540130Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.1540840Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1541306Z [cluster1]                                                                 },
2021-07-19T04:15:32.1541784Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.1542490Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-19T04:15:32.1543218Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1543844Z [cluster1]                                                                 },
2021-07-19T04:15:32.1544285Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.1544945Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-19T04:15:32.1545591Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1546045Z [cluster1]                                                                 },
2021-07-19T04:15:32.1546469Z [cluster1]                                                                 "signal": {
2021-07-19T04:15:32.1547102Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-19T04:15:32.1547747Z [cluster1]                                                                     "format": "int32",
2021-07-19T04:15:32.1548383Z [cluster1]                                                                     "type": "integer"
2021-07-19T04:15:32.1548868Z [cluster1]                                                                 },
2021-07-19T04:15:32.1549311Z [cluster1]                                                                 "startedAt": {
2021-07-19T04:15:32.1549996Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-19T04:15:32.1551171Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.1551904Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1552404Z [cluster1]                                                                 }
2021-07-19T04:15:32.1552843Z [cluster1]                                                             },
2021-07-19T04:15:32.1553332Z [cluster1]                                                             "required": [
2021-07-19T04:15:32.1554527Z [cluster1]                                                                 "exitCode"
2021-07-19T04:15:32.1555008Z [cluster1]                                                             ],
2021-07-19T04:15:32.1555485Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.1556105Z [cluster1]                                                         },
2021-07-19T04:15:32.1556552Z [cluster1]                                                         "waiting": {
2021-07-19T04:15:32.1557310Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-19T04:15:32.1557940Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.1558673Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.1559354Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-19T04:15:32.1560021Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1560846Z [cluster1]                                                                 },
2021-07-19T04:15:32.1561312Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.1561960Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-19T04:15:32.1562644Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1563205Z [cluster1]                                                                 }
2021-07-19T04:15:32.1563770Z [cluster1]                                                             },
2021-07-19T04:15:32.1564219Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.1564654Z [cluster1]                                                         }
2021-07-19T04:15:32.1565060Z [cluster1]                                                     },
2021-07-19T04:15:32.1565483Z [cluster1]                                                     "type": "object"
2021-07-19T04:15:32.1565916Z [cluster1]                                                 },
2021-07-19T04:15:32.1566347Z [cluster1]                                                 "type": "array"
2021-07-19T04:15:32.1566759Z [cluster1]                                             },
2021-07-19T04:15:32.1567174Z [cluster1]                                             "status": {
2021-07-19T04:15:32.1568039Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-19T04:15:32.1568884Z [cluster1]                                                 "properties": {
2021-07-19T04:15:32.1569484Z [cluster1]                                                     "collisionCount": {
2021-07-19T04:15:32.1571391Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-19T04:15:32.1572587Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1573116Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1573771Z [cluster1]                                                     },
2021-07-19T04:15:32.1574546Z [cluster1]                                                     "conditions": {
2021-07-19T04:15:32.1612733Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-19T04:15:32.1613559Z [cluster1]                                                         "items": {
2021-07-19T04:15:32.1614297Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-19T04:15:32.1615069Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.1615755Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-19T04:15:32.1616844Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-19T04:15:32.1617864Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.1618373Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1618809Z [cluster1]                                                                 },
2021-07-19T04:15:32.1619242Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.1619910Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-19T04:15:32.1621059Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1621515Z [cluster1]                                                                 },
2021-07-19T04:15:32.1621967Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.1622855Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-19T04:15:32.1623793Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1624258Z [cluster1]                                                                 },
2021-07-19T04:15:32.1624678Z [cluster1]                                                                 "status": {
2021-07-19T04:15:32.1625315Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-19T04:15:32.1625965Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1626404Z [cluster1]                                                                 },
2021-07-19T04:15:32.1626842Z [cluster1]                                                                 "type": {
2021-07-19T04:15:32.1627407Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-19T04:15:32.1628377Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1628859Z [cluster1]                                                                 }
2021-07-19T04:15:32.1629286Z [cluster1]                                                             },
2021-07-19T04:15:32.1629752Z [cluster1]                                                             "required": [
2021-07-19T04:15:32.1630219Z [cluster1]                                                                 "status",
2021-07-19T04:15:32.1630679Z [cluster1]                                                                 "type"
2021-07-19T04:15:32.1631111Z [cluster1]                                                             ],
2021-07-19T04:15:32.1631732Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.1632165Z [cluster1]                                                         },
2021-07-19T04:15:32.1632605Z [cluster1]                                                         "type": "array"
2021-07-19T04:15:32.1634266Z [cluster1]                                                     },
2021-07-19T04:15:32.1634948Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-19T04:15:32.1636619Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.1638087Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1639214Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1639784Z [cluster1]                                                     },
2021-07-19T04:15:32.1640457Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-19T04:15:32.1642559Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.1644044Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1645059Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1645569Z [cluster1]                                                     },
2021-07-19T04:15:32.1646114Z [cluster1]                                                     "numberAvailable": {
2021-07-19T04:15:32.1684044Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.1685521Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1686073Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1686581Z [cluster1]                                                     },
2021-07-19T04:15:32.1687156Z [cluster1]                                                     "numberMisscheduled": {
2021-07-19T04:15:32.1688871Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.1689972Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1690601Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1691072Z [cluster1]                                                     },
2021-07-19T04:15:32.1691857Z [cluster1]                                                     "numberReady": {
2021-07-19T04:15:32.1692675Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-19T04:15:32.1693461Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1694117Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1694581Z [cluster1]                                                     },
2021-07-19T04:15:32.1695814Z [cluster1]                                                     "numberUnavailable": {
2021-07-19T04:15:32.1697014Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.1698274Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1699118Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1699757Z [cluster1]                                                     },
2021-07-19T04:15:32.1700279Z [cluster1]                                                     "observedGeneration": {
2021-07-19T04:15:32.1701375Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-19T04:15:32.1702360Z [cluster1]                                                         "format": "int64",
2021-07-19T04:15:32.1703000Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1703442Z [cluster1]                                                     },
2021-07-19T04:15:32.1704160Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-19T04:15:32.1705330Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-19T04:15:32.1706453Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.1707138Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.1707629Z [cluster1]                                                     }
2021-07-19T04:15:32.1708244Z [cluster1]                                                 },
2021-07-19T04:15:32.1709048Z [cluster1]                                                 "required": [
2021-07-19T04:15:32.1709939Z [cluster1]                                                     "currentNumberScheduled",
2021-07-19T04:15:32.1710644Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-19T04:15:32.1711423Z [cluster1]                                                     "numberMisscheduled",
2021-07-19T04:15:32.1711998Z [cluster1]                                                     "numberReady"
2021-07-19T04:15:32.1712460Z [cluster1]                                                 ],
2021-07-19T04:15:32.1712897Z [cluster1]                                                 "type": "object"
2021-07-19T04:15:32.1713342Z [cluster1]                                             }
2021-07-19T04:15:32.1713726Z [cluster1]                                         },
2021-07-19T04:15:32.1714157Z [cluster1]                                         "required": [
2021-07-19T04:15:32.1714858Z [cluster1]                                             "mismatchedContainerImages"
2021-07-19T04:15:32.1715654Z [cluster1]                                         ],
2021-07-19T04:15:32.1716135Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.1754092Z [cluster1]                                     },
2021-07-19T04:15:32.1754739Z [cluster1]                                     "gateways": {
2021-07-19T04:15:32.1756112Z [cluster1]                                         "items": {
2021-07-19T04:15:32.1757020Z [cluster1]                                             "properties": {
2021-07-19T04:15:32.1757634Z [cluster1]                                                 "connections": {
2021-07-19T04:15:32.1758495Z [cluster1]                                                     "items": {
2021-07-19T04:15:32.1759329Z [cluster1]                                                         "properties": {
2021-07-19T04:15:32.1760026Z [cluster1]                                                             "endpoint": {
2021-07-19T04:15:32.1760555Z [cluster1]                                                                 "properties": {
2021-07-19T04:15:32.1761069Z [cluster1]                                                                     "backend": {
2021-07-19T04:15:32.1761588Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1762074Z [cluster1]                                                                     },
2021-07-19T04:15:32.1762575Z [cluster1]                                                                     "backend_config": {
2021-07-19T04:15:32.1763384Z [cluster1]                                                                         "additionalProperties": {
2021-07-19T04:15:32.1764197Z [cluster1]                                                                             "type": "string"
2021-07-19T04:15:32.1765475Z [cluster1]                                                                         },
2021-07-19T04:15:32.1766098Z [cluster1]                                                                         "type": "object"
2021-07-19T04:15:32.1766659Z [cluster1]                                                                     },
2021-07-19T04:15:32.1767207Z [cluster1]                                                                     "cable_name": {
2021-07-19T04:15:32.1767795Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1768956Z [cluster1]                                                                     },
2021-07-19T04:15:32.1769391Z [cluster1]                                                                     "cluster_id": {
2021-07-19T04:15:32.1769885Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1770328Z [cluster1]                                                                     },
2021-07-19T04:15:32.1770827Z [cluster1]                                                                     "healthCheckIP": {
2021-07-19T04:15:32.1771350Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1771791Z [cluster1]                                                                     },
2021-07-19T04:15:32.1772243Z [cluster1]                                                                     "hostname": {
2021-07-19T04:15:32.1772719Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1773349Z [cluster1]                                                                     },
2021-07-19T04:15:32.1773814Z [cluster1]                                                                     "nat_enabled": {
2021-07-19T04:15:32.1774311Z [cluster1]                                                                         "type": "boolean"
2021-07-19T04:15:32.1774791Z [cluster1]                                                                     },
2021-07-19T04:15:32.1775430Z [cluster1]                                                                     "private_ip": {
2021-07-19T04:15:32.1776548Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1777521Z [cluster1]                                                                     },
2021-07-19T04:15:32.1778559Z [cluster1]                                                                     "public_ip": {
2021-07-19T04:15:32.1779250Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1779711Z [cluster1]                                                                     },
2021-07-19T04:15:32.1780170Z [cluster1]                                                                     "subnets": {
2021-07-19T04:15:32.1780633Z [cluster1]                                                                         "items": {
2021-07-19T04:15:32.1781138Z [cluster1]                                                                             "type": "string"
2021-07-19T04:15:32.1781611Z [cluster1]                                                                         },
2021-07-19T04:15:32.1782102Z [cluster1]                                                                         "type": "array"
2021-07-19T04:15:32.1782754Z [cluster1]                                                                     }
2021-07-19T04:15:32.1783180Z [cluster1]                                                                 },
2021-07-19T04:15:32.1783803Z [cluster1]                                                                 "required": [
2021-07-19T04:15:32.1784277Z [cluster1]                                                                     "backend",
2021-07-19T04:15:32.1784959Z [cluster1]                                                                     "cable_name",
2021-07-19T04:15:32.1785791Z [cluster1]                                                                     "cluster_id",
2021-07-19T04:15:32.1786611Z [cluster1]                                                                     "hostname",
2021-07-19T04:15:32.1787165Z [cluster1]                                                                     "nat_enabled",
2021-07-19T04:15:32.1787703Z [cluster1]                                                                     "private_ip",
2021-07-19T04:15:32.1788256Z [cluster1]                                                                     "public_ip",
2021-07-19T04:15:32.1788973Z [cluster1]                                                                     "subnets"
2021-07-19T04:15:32.1789433Z [cluster1]                                                                 ],
2021-07-19T04:15:32.1789915Z [cluster1]                                                                 "type": "object"
2021-07-19T04:15:32.1790374Z [cluster1]                                                             },
2021-07-19T04:15:32.1790864Z [cluster1]                                                             "latencyRTT": {
2021-07-19T04:15:32.1791749Z [cluster1]                                                                 "description": "LatencySpec describes the round trip time information for a packet between the gateway pods of two clusters.",
2021-07-19T04:15:32.1792646Z [cluster1]                                                                 "properties": {
2021-07-19T04:15:32.1793172Z [cluster1]                                                                     "average": {
2021-07-19T04:15:32.1793672Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1794162Z [cluster1]                                                                     },
2021-07-19T04:15:32.1794602Z [cluster1]                                                                     "last": {
2021-07-19T04:15:32.1795278Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1795796Z [cluster1]                                                                     },
2021-07-19T04:15:32.1796265Z [cluster1]                                                                     "max": {
2021-07-19T04:15:32.1796803Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1797385Z [cluster1]                                                                     },
2021-07-19T04:15:32.1797877Z [cluster1]                                                                     "min": {
2021-07-19T04:15:32.1798732Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1799214Z [cluster1]                                                                     },
2021-07-19T04:15:32.1799691Z [cluster1]                                                                     "stdDev": {
2021-07-19T04:15:32.1800186Z [cluster1]                                                                         "type": "string"
2021-07-19T04:15:32.1800837Z [cluster1]                                                                     }
2021-07-19T04:15:32.1801246Z [cluster1]                                                                 },
2021-07-19T04:15:32.1801710Z [cluster1]                                                                 "type": "object"
2021-07-19T04:15:32.1802168Z [cluster1]                                                             },
2021-07-19T04:15:32.1802613Z [cluster1]                                                             "status": {
2021-07-19T04:15:32.1803096Z [cluster1]                                                                 "type": "string"
2021-07-19T04:15:32.1803727Z [cluster1]                                                             },
2021-07-19T04:15:32.1804864Z [cluster1]                                                             "statusMessage": {
2021-07-19T04:15:32.1805883Z [cluster1]                                                                 "type": "string"
2021-07-19T04:15:32.1806418Z [cluster1]                                                             },
2021-07-19T04:15:32.1871848Z [cluster1]                                                             "usingIP": {
2021-07-19T04:15:32.1872388Z [cluster1]                                                                 "type": "string"
2021-07-19T04:15:32.1872883Z [cluster1]                                                             },
2021-07-19T04:15:32.1873345Z [cluster1]                                                             "usingNAT": {
2021-07-19T04:15:32.1873863Z [cluster1]                                                                 "type": "boolean"
2021-07-19T04:15:32.1874492Z [cluster1]                                                             }
2021-07-19T04:15:32.1874928Z [cluster1]                                                         },
2021-07-19T04:15:32.1875621Z [cluster1]                                                         "required": [
2021-07-19T04:15:32.1876159Z [cluster1]                                                             "endpoint",
2021-07-19T04:15:32.1876683Z [cluster1]                                                             "status",
2021-07-19T04:15:32.1877226Z [cluster1]                                                             "statusMessage"
2021-07-19T04:15:32.1877756Z [cluster1]                                                         ],
2021-07-19T04:15:32.1878566Z [cluster1]                                                         "type": "object"
2021-07-19T04:15:32.1879042Z [cluster1]                                                     },
2021-07-19T04:15:32.1879510Z [cluster1]                                                     "type": "array"
2021-07-19T04:15:32.1879950Z [cluster1]                                                 },
2021-07-19T04:15:32.1880402Z [cluster1]                                                 "haStatus": {
2021-07-19T04:15:32.1880890Z [cluster1]                                                     "type": "string"
2021-07-19T04:15:32.1881349Z [cluster1]                                                 },
2021-07-19T04:15:32.1881823Z [cluster1]                                                 "localEndpoint": {
2021-07-19T04:15:32.1882362Z [cluster1]                                                     "properties": {
2021-07-19T04:15:32.1882852Z [cluster1]                                                         "backend": {
2021-07-19T04:15:32.1883632Z [cluster1]                                                             "type": "string"
2021-07-19T04:15:32.1884118Z [cluster1]                                                         },
2021-07-19T04:15:32.1884569Z [cluster1]                                                         "backend_config": {
2021-07-19T04:15:32.1885358Z [cluster1]                                                             "additionalProperties": {
2021-07-19T04:15:32.1885989Z [cluster1]                                                                 "type": "string"
2021-07-19T04:15:32.1886506Z [cluster1]                                                             },
2021-07-19T04:15:32.1887013Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.1887504Z [cluster1]                                                         },
2021-07-19T04:15:32.1888319Z [cluster1]                                                         "cable_name": {
2021-07-19T04:15:32.1888965Z [cluster1]                                                             "type": "string"
2021-07-19T04:15:32.1889421Z [cluster1]                                                         },
2021-07-19T04:15:32.1889855Z [cluster1]                                                         "cluster_id": {
2021-07-19T04:15:32.1890346Z [cluster1]                                                             "type": "string"
2021-07-19T04:15:32.1890802Z [cluster1]                                                         },
2021-07-19T04:15:32.1891374Z [cluster1]                                                         "healthCheckIP": {
2021-07-19T04:15:32.1891900Z [cluster1]                                                             "type": "string"
2021-07-19T04:15:32.1892342Z [cluster1]                                                         },
2021-07-19T04:15:32.1892845Z [cluster1]                                                         "hostname": {
2021-07-19T04:15:32.1893512Z [cluster1]                                                             "type": "string"
2021-07-19T04:15:32.1893982Z [cluster1]                                                         },
2021-07-19T04:15:32.1894621Z [cluster1]                                                         "nat_enabled": {
2021-07-19T04:15:32.1895754Z [cluster1]                                                             "type": "boolean"
2021-07-19T04:15:32.1896297Z [cluster1]                                                         },
2021-07-19T04:15:32.1896961Z [cluster1]                                                         "private_ip": {
2021-07-19T04:15:32.1897507Z [cluster1]                                                             "type": "string"
2021-07-19T04:15:32.1897990Z [cluster1]                                                         },
2021-07-19T04:15:32.1898977Z [cluster1]                                                         "public_ip": {
2021-07-19T04:15:32.1899477Z [cluster1]                                                             "type": "string"
2021-07-19T04:15:32.1899924Z [cluster1]                                                         },
2021-07-19T04:15:32.1900368Z [cluster1]                                                         "subnets": {
2021-07-19T04:15:32.1900819Z [cluster1]                                                             "items": {
2021-07-19T04:15:32.1901299Z [cluster1]                                                                 "type": "string"
2021-07-19T04:15:32.1901748Z [cluster1]                                                             },
2021-07-19T04:15:32.1902204Z [cluster1]                                                             "type": "array"
2021-07-19T04:15:32.1902653Z [cluster1]                                                         }
2021-07-19T04:15:32.1903095Z [cluster1]                                                     },
2021-07-19T04:15:32.1903544Z [cluster1]                                                     "required": [
2021-07-19T04:15:32.1904211Z [cluster1]                                                         "backend",
2021-07-19T04:15:32.1904879Z [cluster1]                                                         "cable_name",
2021-07-19T04:15:32.1905778Z [cluster1]                                                         "cluster_id",
2021-07-19T04:15:32.1906340Z [cluster1]                                                         "hostname",
2021-07-19T04:15:32.1906909Z [cluster1]                                                         "nat_enabled",
2021-07-19T04:15:32.1907625Z [cluster1]                                                         "private_ip",
2021-07-19T04:15:32.1908329Z [cluster1]                                                         "public_ip",
2021-07-19T04:15:32.1909143Z [cluster1]                                                         "subnets"
2021-07-19T04:15:32.1909584Z [cluster1]                                                     ],
2021-07-19T04:15:32.1910025Z [cluster1]                                                     "type": "object"
2021-07-19T04:15:32.1910474Z [cluster1]                                                 },
2021-07-19T04:15:32.1910948Z [cluster1]                                                 "statusFailure": {
2021-07-19T04:15:32.1911443Z [cluster1]                                                     "type": "string"
2021-07-19T04:15:32.1911886Z [cluster1]                                                 },
2021-07-19T04:15:32.1912424Z [cluster1]                                                 "version": {
2021-07-19T04:15:32.1913003Z [cluster1]                                                     "type": "string"
2021-07-19T04:15:32.1913436Z [cluster1]                                                 }
2021-07-19T04:15:32.1913839Z [cluster1]                                             },
2021-07-19T04:15:32.1914258Z [cluster1]                                             "required": [
2021-07-19T04:15:32.1914877Z [cluster1]                                                 "connections",
2021-07-19T04:15:32.1915578Z [cluster1]                                                 "haStatus",
2021-07-19T04:15:32.1916118Z [cluster1]                                                 "localEndpoint",
2021-07-19T04:15:32.1916793Z [cluster1]                                                 "statusFailure",
2021-07-19T04:15:32.1917335Z [cluster1]                                                 "version"
2021-07-19T04:15:32.1917806Z [cluster1]                                             ],
2021-07-19T04:15:32.1918442Z [cluster1]                                             "type": "object"
2021-07-19T04:15:32.1918899Z [cluster1]                                         },
2021-07-19T04:15:32.1919342Z [cluster1]                                         "type": "array"
2021-07-19T04:15:32.1920097Z [cluster1]                                     },
2021-07-19T04:15:32.1920544Z [cluster1]                                     "globalCIDR": {
2021-07-19T04:15:32.1921024Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.1921477Z [cluster1]                                     },
2021-07-19T04:15:32.1922051Z [cluster1]                                     "globalnetDaemonSetStatus": {
2021-07-19T04:15:32.1976492Z [cluster1]                                         "properties": {
2021-07-19T04:15:32.1977120Z [cluster1]                                             "lastResourceVersion": {
2021-07-19T04:15:32.1977742Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.1978411Z [cluster1]                                             },
2021-07-19T04:15:32.1979166Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-19T04:15:32.1979820Z [cluster1]                                                 "type": "boolean"
2021-07-19T04:15:32.1980379Z [cluster1]                                             },
2021-07-19T04:15:32.1981023Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-19T04:15:32.1981769Z [cluster1]                                                 "items": {
2021-07-19T04:15:32.1982895Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-19T04:15:32.1984135Z [cluster1]                                                     "properties": {
2021-07-19T04:15:32.1984608Z [cluster1]                                                         "running": {
2021-07-19T04:15:32.1985432Z [cluster1]                                                             "description": "Details about a running container",
2021-07-19T04:15:32.1986108Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.1986670Z [cluster1]                                                                 "startedAt": {
2021-07-19T04:15:32.1987842Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-19T04:15:32.1989056Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.1989589Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1990045Z [cluster1]                                                                 }
2021-07-19T04:15:32.1990467Z [cluster1]                                                             },
2021-07-19T04:15:32.1991192Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.1991646Z [cluster1]                                                         },
2021-07-19T04:15:32.1992109Z [cluster1]                                                         "terminated": {
2021-07-19T04:15:32.1992750Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-19T04:15:32.1993394Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.1993986Z [cluster1]                                                                 "containerID": {
2021-07-19T04:15:32.1995576Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-19T04:15:32.1996626Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.1997200Z [cluster1]                                                                 },
2021-07-19T04:15:32.1997769Z [cluster1]                                                                 "exitCode": {
2021-07-19T04:15:32.1998583Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-19T04:15:32.2000034Z [cluster1]                                                                     "format": "int32",
2021-07-19T04:15:32.2000547Z [cluster1]                                                                     "type": "integer"
2021-07-19T04:15:32.2001053Z [cluster1]                                                                 },
2021-07-19T04:15:32.2001525Z [cluster1]                                                                 "finishedAt": {
2021-07-19T04:15:32.2002169Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-19T04:15:32.2003061Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.2003595Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2004050Z [cluster1]                                                                 },
2021-07-19T04:15:32.2004607Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.2005735Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-19T04:15:32.2006705Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2007696Z [cluster1]                                                                 },
2021-07-19T04:15:32.2008588Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.2009266Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-19T04:15:32.2009934Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2010564Z [cluster1]                                                                 },
2021-07-19T04:15:32.2011012Z [cluster1]                                                                 "signal": {
2021-07-19T04:15:32.2011687Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-19T04:15:32.2012372Z [cluster1]                                                                     "format": "int32",
2021-07-19T04:15:32.2012889Z [cluster1]                                                                     "type": "integer"
2021-07-19T04:15:32.2013502Z [cluster1]                                                                 },
2021-07-19T04:15:32.2013964Z [cluster1]                                                                 "startedAt": {
2021-07-19T04:15:32.2014687Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-19T04:15:32.2016053Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.2016632Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2017148Z [cluster1]                                                                 }
2021-07-19T04:15:32.2017597Z [cluster1]                                                             },
2021-07-19T04:15:32.2018105Z [cluster1]                                                             "required": [
2021-07-19T04:15:32.2018898Z [cluster1]                                                                 "exitCode"
2021-07-19T04:15:32.2019381Z [cluster1]                                                             ],
2021-07-19T04:15:32.2019858Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2020315Z [cluster1]                                                         },
2021-07-19T04:15:32.2020781Z [cluster1]                                                         "waiting": {
2021-07-19T04:15:32.2021734Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-19T04:15:32.2022382Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.2022885Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.2023594Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-19T04:15:32.2024950Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2025634Z [cluster1]                                                                 },
2021-07-19T04:15:32.2026515Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.2027527Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-19T04:15:32.2028289Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2028967Z [cluster1]                                                                 }
2021-07-19T04:15:32.2029560Z [cluster1]                                                             },
2021-07-19T04:15:32.2030041Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2030508Z [cluster1]                                                         }
2021-07-19T04:15:32.2030934Z [cluster1]                                                     },
2021-07-19T04:15:32.2031487Z [cluster1]                                                     "type": "object"
2021-07-19T04:15:32.2031958Z [cluster1]                                                 },
2021-07-19T04:15:32.2032416Z [cluster1]                                                 "type": "array"
2021-07-19T04:15:32.2032926Z [cluster1]                                             },
2021-07-19T04:15:32.2033372Z [cluster1]                                             "status": {
2021-07-19T04:15:32.2034085Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-19T04:15:32.2034848Z [cluster1]                                                 "properties": {
2021-07-19T04:15:32.2035569Z [cluster1]                                                     "collisionCount": {
2021-07-19T04:15:32.2036909Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-19T04:15:32.2039836Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2040464Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2041038Z [cluster1]                                                     },
2021-07-19T04:15:32.2041880Z [cluster1]                                                     "conditions": {
2021-07-19T04:15:32.2088063Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-19T04:15:32.2089198Z [cluster1]                                                         "items": {
2021-07-19T04:15:32.2089954Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-19T04:15:32.2090754Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.2091321Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-19T04:15:32.2092117Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-19T04:15:32.2093080Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.2093605Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2094076Z [cluster1]                                                                 },
2021-07-19T04:15:32.2094524Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.2095452Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-19T04:15:32.2096368Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2097296Z [cluster1]                                                                 },
2021-07-19T04:15:32.2097814Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.2098940Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-19T04:15:32.2099663Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2100155Z [cluster1]                                                                 },
2021-07-19T04:15:32.2100602Z [cluster1]                                                                 "status": {
2021-07-19T04:15:32.2101297Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-19T04:15:32.2102263Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2102742Z [cluster1]                                                                 },
2021-07-19T04:15:32.2103172Z [cluster1]                                                                 "type": {
2021-07-19T04:15:32.2103889Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-19T04:15:32.2104517Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2105325Z [cluster1]                                                                 }
2021-07-19T04:15:32.2106140Z [cluster1]                                                             },
2021-07-19T04:15:32.2106632Z [cluster1]                                                             "required": [
2021-07-19T04:15:32.2107309Z [cluster1]                                                                 "status",
2021-07-19T04:15:32.2107814Z [cluster1]                                                                 "type"
2021-07-19T04:15:32.2108607Z [cluster1]                                                             ],
2021-07-19T04:15:32.2109404Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2109850Z [cluster1]                                                         },
2021-07-19T04:15:32.2110304Z [cluster1]                                                         "type": "array"
2021-07-19T04:15:32.2110743Z [cluster1]                                                     },
2021-07-19T04:15:32.2111308Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-19T04:15:32.2112531Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.2113610Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2114111Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2114565Z [cluster1]                                                     },
2021-07-19T04:15:32.2115532Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-19T04:15:32.2117025Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.2118362Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2118865Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2119316Z [cluster1]                                                     },
2021-07-19T04:15:32.2119898Z [cluster1]                                                     "numberAvailable": {
2021-07-19T04:15:32.2120956Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.2121927Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2122433Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2122885Z [cluster1]                                                     },
2021-07-19T04:15:32.2123404Z [cluster1]                                                     "numberMisscheduled": {
2021-07-19T04:15:32.2124565Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.2125643Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2126141Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2126592Z [cluster1]                                                     },
2021-07-19T04:15:32.2127054Z [cluster1]                                                     "numberReady": {
2021-07-19T04:15:32.2127966Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-19T04:15:32.2128757Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2129255Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2129709Z [cluster1]                                                     },
2021-07-19T04:15:32.2130215Z [cluster1]                                                     "numberUnavailable": {
2021-07-19T04:15:32.2131254Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.2132211Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2132709Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2133158Z [cluster1]                                                     },
2021-07-19T04:15:32.2133676Z [cluster1]                                                     "observedGeneration": {
2021-07-19T04:15:32.2134456Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-19T04:15:32.2135169Z [cluster1]                                                         "format": "int64",
2021-07-19T04:15:32.2166892Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2167417Z [cluster1]                                                     },
2021-07-19T04:15:32.2168473Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-19T04:15:32.2169847Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-19T04:15:32.2170564Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2171277Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2171754Z [cluster1]                                                     }
2021-07-19T04:15:32.2172355Z [cluster1]                                                 },
2021-07-19T04:15:32.2172836Z [cluster1]                                                 "required": [
2021-07-19T04:15:32.2173625Z [cluster1]                                                     "currentNumberScheduled",
2021-07-19T04:15:32.2174345Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-19T04:15:32.2175247Z [cluster1]                                                     "numberMisscheduled",
2021-07-19T04:15:32.2176074Z [cluster1]                                                     "numberReady"
2021-07-19T04:15:32.2176565Z [cluster1]                                                 ],
2021-07-19T04:15:32.2177056Z [cluster1]                                                 "type": "object"
2021-07-19T04:15:32.2177530Z [cluster1]                                             }
2021-07-19T04:15:32.2177965Z [cluster1]                                         },
2021-07-19T04:15:32.2178822Z [cluster1]                                         "required": [
2021-07-19T04:15:32.2179461Z [cluster1]                                             "mismatchedContainerImages"
2021-07-19T04:15:32.2180065Z [cluster1]                                         ],
2021-07-19T04:15:32.2180503Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.2181127Z [cluster1]                                     },
2021-07-19T04:15:32.2181615Z [cluster1]                                     "loadBalancerStatus": {
2021-07-19T04:15:32.2182424Z [cluster1]                                         "properties": {
2021-07-19T04:15:32.2182881Z [cluster1]                                             "status": {
2021-07-19T04:15:32.2183996Z [cluster1]                                                 "description": "LoadBalancerStatus represents the status of a load-balancer.",
2021-07-19T04:15:32.2184752Z [cluster1]                                                 "properties": {
2021-07-19T04:15:32.2185458Z [cluster1]                                                     "ingress": {
2021-07-19T04:15:32.2186879Z [cluster1]                                                         "description": "Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points.",
2021-07-19T04:15:32.2187850Z [cluster1]                                                         "items": {
2021-07-19T04:15:32.2189464Z [cluster1]                                                             "description": "LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.",
2021-07-19T04:15:32.2190498Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.2190998Z [cluster1]                                                                 "hostname": {
2021-07-19T04:15:32.2192130Z [cluster1]                                                                     "description": "Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers)",
2021-07-19T04:15:32.2192955Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2193419Z [cluster1]                                                                 },
2021-07-19T04:15:32.2193857Z [cluster1]                                                                 "ip": {
2021-07-19T04:15:32.2216361Z [cluster1]                                                                     "description": "IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers)",
2021-07-19T04:15:32.2217549Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2218067Z [cluster1]                                                                 }
2021-07-19T04:15:32.2218850Z [cluster1]                                                             },
2021-07-19T04:15:32.2219441Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2219936Z [cluster1]                                                         },
2021-07-19T04:15:32.2220555Z [cluster1]                                                         "type": "array"
2021-07-19T04:15:32.2220977Z [cluster1]                                                     }
2021-07-19T04:15:32.2221374Z [cluster1]                                                 },
2021-07-19T04:15:32.2221801Z [cluster1]                                                 "type": "object"
2021-07-19T04:15:32.2222424Z [cluster1]                                             }
2021-07-19T04:15:32.2222802Z [cluster1]                                         },
2021-07-19T04:15:32.2223236Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.2223668Z [cluster1]                                     },
2021-07-19T04:15:32.2224095Z [cluster1]                                     "natEnabled": {
2021-07-19T04:15:32.2224584Z [cluster1]                                         "type": "boolean"
2021-07-19T04:15:32.2225200Z [cluster1]                                     },
2021-07-19T04:15:32.2225917Z [cluster1]                                     "networkPlugin": {
2021-07-19T04:15:32.2226450Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.2226921Z [cluster1]                                     },
2021-07-19T04:15:32.2227547Z [cluster1]                                     "routeAgentDaemonSetStatus": {
2021-07-19T04:15:32.2228349Z [cluster1]                                         "properties": {
2021-07-19T04:15:32.2229258Z [cluster1]                                             "lastResourceVersion": {
2021-07-19T04:15:32.2229842Z [cluster1]                                                 "type": "string"
2021-07-19T04:15:32.2230671Z [cluster1]                                             },
2021-07-19T04:15:32.2231583Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-19T04:15:32.2232557Z [cluster1]                                                 "type": "boolean"
2021-07-19T04:15:32.2233005Z [cluster1]                                             },
2021-07-19T04:15:32.2233770Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-19T04:15:32.2234555Z [cluster1]                                                 "items": {
2021-07-19T04:15:32.2236100Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-19T04:15:32.2237560Z [cluster1]                                                     "properties": {
2021-07-19T04:15:32.2274099Z [cluster1]                                                         "running": {
2021-07-19T04:15:32.2275128Z [cluster1]                                                             "description": "Details about a running container",
2021-07-19T04:15:32.2276226Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.2276800Z [cluster1]                                                                 "startedAt": {
2021-07-19T04:15:32.2277968Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-19T04:15:32.2279374Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.2279936Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2280489Z [cluster1]                                                                 }
2021-07-19T04:15:32.2280928Z [cluster1]                                                             },
2021-07-19T04:15:32.2281586Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2282078Z [cluster1]                                                         },
2021-07-19T04:15:32.2282561Z [cluster1]                                                         "terminated": {
2021-07-19T04:15:32.2283214Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-19T04:15:32.2283887Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.2284438Z [cluster1]                                                                 "containerID": {
2021-07-19T04:15:32.2285923Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-19T04:15:32.2286720Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2287241Z [cluster1]                                                                 },
2021-07-19T04:15:32.2287776Z [cluster1]                                                                 "exitCode": {
2021-07-19T04:15:32.2288866Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-19T04:15:32.2289545Z [cluster1]                                                                     "format": "int32",
2021-07-19T04:15:32.2290192Z [cluster1]                                                                     "type": "integer"
2021-07-19T04:15:32.2290665Z [cluster1]                                                                 },
2021-07-19T04:15:32.2291154Z [cluster1]                                                                 "finishedAt": {
2021-07-19T04:15:32.2291823Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-19T04:15:32.2292761Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.2293767Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2294277Z [cluster1]                                                                 },
2021-07-19T04:15:32.2294974Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.2297842Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-19T04:15:32.2299179Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2299664Z [cluster1]                                                                 },
2021-07-19T04:15:32.2300113Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.2300818Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-19T04:15:32.2301515Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2301991Z [cluster1]                                                                 },
2021-07-19T04:15:32.2302450Z [cluster1]                                                                 "signal": {
2021-07-19T04:15:32.2303114Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-19T04:15:32.2304180Z [cluster1]                                                                     "format": "int32",
2021-07-19T04:15:32.2305059Z [cluster1]                                                                     "type": "integer"
2021-07-19T04:15:32.2305935Z [cluster1]                                                                 },
2021-07-19T04:15:32.2306621Z [cluster1]                                                                 "startedAt": {
2021-07-19T04:15:32.2307582Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-19T04:15:32.2308921Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.2309810Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2310314Z [cluster1]                                                                 }
2021-07-19T04:15:32.2310750Z [cluster1]                                                             },
2021-07-19T04:15:32.2311239Z [cluster1]                                                             "required": [
2021-07-19T04:15:32.2311770Z [cluster1]                                                                 "exitCode"
2021-07-19T04:15:32.2312323Z [cluster1]                                                             ],
2021-07-19T04:15:32.2312817Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2313465Z [cluster1]                                                         },
2021-07-19T04:15:32.2314103Z [cluster1]                                                         "waiting": {
2021-07-19T04:15:32.2315112Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-19T04:15:32.2316167Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.2316924Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.2317683Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-19T04:15:32.2318620Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2319270Z [cluster1]                                                                 },
2021-07-19T04:15:32.2319752Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.2320438Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-19T04:15:32.2321111Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2321613Z [cluster1]                                                                 }
2021-07-19T04:15:32.2322045Z [cluster1]                                                             },
2021-07-19T04:15:32.2322539Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2323010Z [cluster1]                                                         }
2021-07-19T04:15:32.2323456Z [cluster1]                                                     },
2021-07-19T04:15:32.2324020Z [cluster1]                                                     "type": "object"
2021-07-19T04:15:32.2324479Z [cluster1]                                                 },
2021-07-19T04:15:32.2325141Z [cluster1]                                                 "type": "array"
2021-07-19T04:15:32.2326272Z [cluster1]                                             },
2021-07-19T04:15:32.2326951Z [cluster1]                                             "status": {
2021-07-19T04:15:32.2327910Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-19T04:15:32.2328904Z [cluster1]                                                 "properties": {
2021-07-19T04:15:32.2329648Z [cluster1]                                                     "collisionCount": {
2021-07-19T04:15:32.2330934Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-19T04:15:32.2332137Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2332830Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2333336Z [cluster1]                                                     },
2021-07-19T04:15:32.2333841Z [cluster1]                                                     "conditions": {
2021-07-19T04:15:32.2334985Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-19T04:15:32.2335999Z [cluster1]                                                         "items": {
2021-07-19T04:15:32.2336836Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-19T04:15:32.2337707Z [cluster1]                                                             "properties": {
2021-07-19T04:15:32.2338349Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-19T04:15:32.2339195Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-19T04:15:32.2340386Z [cluster1]                                                                     "format": "date-time",
2021-07-19T04:15:32.2342177Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2342794Z [cluster1]                                                                 },
2021-07-19T04:15:32.2343368Z [cluster1]                                                                 "message": {
2021-07-19T04:15:32.2344237Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-19T04:15:32.2345446Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2346096Z [cluster1]                                                                 },
2021-07-19T04:15:32.2346570Z [cluster1]                                                                 "reason": {
2021-07-19T04:15:32.2348329Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-19T04:15:32.2349140Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2349755Z [cluster1]                                                                 },
2021-07-19T04:15:32.2350495Z [cluster1]                                                                 "status": {
2021-07-19T04:15:32.2351850Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-19T04:15:32.2352654Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2353238Z [cluster1]                                                                 },
2021-07-19T04:15:32.2353755Z [cluster1]                                                                 "type": {
2021-07-19T04:15:32.2354486Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-19T04:15:32.2355179Z [cluster1]                                                                     "type": "string"
2021-07-19T04:15:32.2355906Z [cluster1]                                                                 }
2021-07-19T04:15:32.2385896Z [cluster1]                                                             },
2021-07-19T04:15:32.2386887Z [cluster1]                                                             "required": [
2021-07-19T04:15:32.2387432Z [cluster1]                                                                 "status",
2021-07-19T04:15:32.2387936Z [cluster1]                                                                 "type"
2021-07-19T04:15:32.2392906Z [cluster1]                                                             ],
2021-07-19T04:15:32.2393965Z [cluster1]                                                             "type": "object"
2021-07-19T04:15:32.2394678Z [cluster1]                                                         },
2021-07-19T04:15:32.2395407Z [cluster1]                                                         "type": "array"
2021-07-19T04:15:32.2396101Z [cluster1]                                                     },
2021-07-19T04:15:32.2398732Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-19T04:15:32.2400617Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.2403052Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2403969Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2404953Z [cluster1]                                                     },
2021-07-19T04:15:32.2406280Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-19T04:15:32.2408974Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.2410533Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2411096Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2411587Z [cluster1]                                                     },
2021-07-19T04:15:32.2412137Z [cluster1]                                                     "numberAvailable": {
2021-07-19T04:15:32.2413494Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.2415004Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2415800Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2416343Z [cluster1]                                                     },
2021-07-19T04:15:32.2417270Z [cluster1]                                                     "numberMisscheduled": {
2021-07-19T04:15:32.2418535Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.2419730Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2421308Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2421832Z [cluster1]                                                     },
2021-07-19T04:15:32.2422600Z [cluster1]                                                     "numberReady": {
2021-07-19T04:15:32.2423641Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-19T04:15:32.2424558Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2431079Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2431588Z [cluster1]                                                     },
2021-07-19T04:15:32.2432153Z [cluster1]                                                     "numberUnavailable": {
2021-07-19T04:15:32.2433303Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.2434711Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2435276Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2436148Z [cluster1]                                                     },
2021-07-19T04:15:32.2436763Z [cluster1]                                                     "observedGeneration": {
2021-07-19T04:15:32.2437677Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-19T04:15:32.2439087Z [cluster1]                                                         "format": "int64",
2021-07-19T04:15:32.2439907Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2440394Z [cluster1]                                                     },
2021-07-19T04:15:32.2441016Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-19T04:15:32.2442921Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-19T04:15:32.2443830Z [cluster1]                                                         "format": "int32",
2021-07-19T04:15:32.2444427Z [cluster1]                                                         "type": "integer"
2021-07-19T04:15:32.2444992Z [cluster1]                                                     }
2021-07-19T04:15:32.2445498Z [cluster1]                                                 },
2021-07-19T04:15:32.2446223Z [cluster1]                                                 "required": [
2021-07-19T04:15:32.2446909Z [cluster1]                                                     "currentNumberScheduled",
2021-07-19T04:15:32.2447696Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-19T04:15:32.2448613Z [cluster1]                                                     "numberMisscheduled",
2021-07-19T04:15:32.2449228Z [cluster1]                                                     "numberReady"
2021-07-19T04:15:32.2449741Z [cluster1]                                                 ],
2021-07-19T04:15:32.2450244Z [cluster1]                                                 "type": "object"
2021-07-19T04:15:32.2450717Z [cluster1]                                             }
2021-07-19T04:15:32.2451154Z [cluster1]                                         },
2021-07-19T04:15:32.2451614Z [cluster1]                                         "required": [
2021-07-19T04:15:32.2452293Z [cluster1]                                             "mismatchedContainerImages"
2021-07-19T04:15:32.2452924Z [cluster1]                                         ],
2021-07-19T04:15:32.2453405Z [cluster1]                                         "type": "object"
2021-07-19T04:15:32.2453878Z [cluster1]                                     },
2021-07-19T04:15:32.2454354Z [cluster1]                                     "serviceCIDR": {
2021-07-19T04:15:32.2454891Z [cluster1]                                         "type": "string"
2021-07-19T04:15:32.2456164Z [cluster1]                                     }
2021-07-19T04:15:32.2456670Z [cluster1]                                 },
2021-07-19T04:15:32.2457170Z [cluster1]                                 "required": [
2021-07-19T04:15:32.2457881Z [cluster1]                                     "clusterID",
2021-07-19T04:15:32.2460347Z [cluster1]                                     "natEnabled"
2021-07-19T04:15:32.2461640Z [cluster1]                                 ],
2021-07-19T04:15:32.2462120Z [cluster1]                                 "type": "object"
2021-07-19T04:15:32.2462576Z [cluster1]                             }
2021-07-19T04:15:32.2462990Z [cluster1]                         },
2021-07-19T04:15:32.2463430Z [cluster1]                         "type": "object"
2021-07-19T04:15:32.2463883Z [cluster1]                     }
2021-07-19T04:15:32.2464265Z [cluster1]                 },
2021-07-19T04:15:32.2464708Z [cluster1]                 "served": true,
2021-07-19T04:15:32.2465391Z [cluster1]                 "storage": true,
2021-07-19T04:15:32.2466213Z [cluster1]                 "subresources": {
2021-07-19T04:15:32.2466726Z [cluster1]                     "status": {}
2021-07-19T04:15:32.2467184Z [cluster1]                 }
2021-07-19T04:15:32.2467745Z [cluster1]             }
2021-07-19T04:15:32.2468323Z [cluster1]         ]
2021-07-19T04:15:32.2468781Z [cluster1]     },
2021-07-19T04:15:32.2469166Z [cluster1]     "status": {
2021-07-19T04:15:32.2469659Z [cluster1]         "acceptedNames": {
2021-07-19T04:15:32.2470417Z [cluster1]             "kind": "Submariner",
2021-07-19T04:15:32.4516407Z [04:14:14Z",
2021-07-19T04:15:32.4527144Z [cluster2]                 "message": "no conflicts found",
2021-07-19T04:15:32.4528373Z [cluster2]                 "reason": "NoConflicts",
2021-07-19T04:15:32.4529490Z [cluster2]                 "status": "True",
2021-07-19T04:15:32.4530289Z [cluster2]                 "type": "NamesAccepted"
2021-07-19T04:15:32.4531715Z [cluster2]             },
2021-07-19T04:15:32.4532103Z [cluster2]             {
2021-07-19T04:15:32.4533536Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:14Z",
2021-07-19T04:15:32.4534365Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-19T04:15:32.4535216Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-19T04:15:32.4536044Z [cluster2]                 "status": "True",
2021-07-19T04:15:32.4536670Z [cluster2]                 "type": "Established"
2021-07-19T04:15:32.4537255Z [cluster2]             }
2021-07-19T04:15:32.4537639Z [cluster2]         ],
2021-07-19T04:15:32.4538068Z [cluster2]         "storedVersions": [
2021-07-19T04:15:32.4538696Z [cluster2]             "v1"
2021-07-19T04:15:32.4539263Z [cluster2]         ]
2021-07-19T04:15:32.4539662Z [cluster2]     }
2021-07-19T04:15:32.4540031Z [cluster2] }
2021-07-19T04:15:32.4541165Z [submariner-operator]$ [cluster2] validate_equals .metadata.name clusters.submariner.io
2021-07-19T04:15:32.4543392Z [submariner-operator]$ [cluster2] validate_equals .metadata.name clusters.submariner.io
2021-07-19T04:15:32.4545043Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/clusters.submariner.io.cluster2.json
2021-07-19T04:15:32.4546403Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-19T04:15:32.4547459Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-19T04:15:32.4548986Z [submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/clusters.submariner.io.cluster2.json
2021-07-19T04:15:32.4550368Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-19T04:15:32.4551516Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-19T04:15:32.4553032Z [submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/clusters.submariner.io.cluster2.json
2021-07-19T04:15:32.4554533Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Cluster
2021-07-19T04:15:32.4555825Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Cluster
2021-07-19T04:15:32.4557483Z [submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/clusters.submariner.io.cluster2.json
2021-07-19T04:15:32.4559132Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Cluster
2021-07-19T04:15:32.4560470Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Cluster
2021-07-19T04:15:32.4562585Z [submariner-operator]$ [cluster2] jq -r -M .status.acceptedNames.kind /tmp/clusters.submariner.io.cluster2.json
2021-07-19T04:15:32.4563884Z [submariner-operator]$ [cluster2] verify_subm_crd
2021-07-19T04:15:32.4564693Z [submariner-operator]$ [cluster2] verify_subm_crd
2021-07-19T04:15:32.4565788Z [submariner-operator]$ [cluster2] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-19T04:15:32.4567112Z [submariner-operator]$ [cluster2] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-19T04:15:32.4568794Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-19T04:15:32.4570820Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-19T04:15:32.4572470Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds submariners.submariner.io
2021-07-19T04:15:32.4573648Z [cluster2] NAME                        CREATED AT
2021-07-19T04:15:32.4574560Z [cluster2] submariners.submariner.io   2021-07-19T04:14:02Z
2021-07-19T04:15:32.4576241Z [submariner-operator]$ [cluster2] json_file=/tmp/submariners.submariner.io.cluster2.json
2021-07-19T04:15:32.4577849Z [submariner-operator]$ [cluster2] kubectl get crd submariners.submariner.io -o json
2021-07-19T04:15:32.4579843Z [submariner-operator]$ [cluster2] tee /tmp/submariners.submariner.io.cluster2.json
2021-07-19T04:15:32.4581493Z [submariner-operator]$ [cluster2] kubectl get crd submariners.submariner.io -o json
2021-07-19T04:15:32.4583552Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd submariners.submariner.io -o json
2021-07-19T04:15:32.4584443Z [cluster2] {
2021-07-19T04:15:32.4584987Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:32.4585929Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-19T04:15:32.4586885Z [cluster2]     "metadata": {
2021-07-19T04:15:32.4587465Z [cluster2]         "annotations": {
2021-07-19T04:15:32.4589010Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-19T04:15:32.4589767Z [cluster2]         },
2021-07-19T04:15:32.4590527Z [cluster2]         "creationTimestamp": "2021-07-19T04:14:02Z",
2021-07-19T04:15:32.4591903Z [cluster2]         "generation": 1,
2021-07-19T04:15:32.4592390Z [cluster2]         "managedFields": [
2021-07-19T04:15:32.4592815Z [cluster2]             {
2021-07-19T04:15:32.4593375Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:32.4594000Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:32.4594498Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:32.4594921Z [cluster2]                     "f:status": {
2021-07-19T04:15:32.4595476Z [cluster2]                         "f:acceptedNames": {
2021-07-19T04:15:32.4595947Z [cluster2]                             "f:kind": {},
2021-07-19T04:15:32.4596396Z [cluster2]                             "f:listKind": {},
2021-07-19T04:15:32.4596851Z [cluster2]                             "f:plural": {},
2021-07-19T04:15:32.4597314Z [cluster2]                             "f:singular": {}
2021-07-19T04:15:32.4597742Z [cluster2]                         },
2021-07-19T04:15:32.4598160Z [cluster2]                         "f:conditions": {}
2021-07-19T04:15:32.4598773Z [cluster2]                     }
2021-07-19T04:15:32.4599155Z [cluster2]                 },
2021-07-19T04:15:32.4600017Z [cluster2]                 "manager": "kube-apiserver",
2021-07-19T04:15:32.4600619Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:32.4601346Z [cluster2]                 "time": "2021-07-19T04:14:02Z"
2021-07-19T04:15:32.4601924Z [cluster2]             },
2021-07-19T04:15:32.4602293Z [cluster2]             {
2021-07-19T04:15:32.4602831Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-19T04:15:32.4603477Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:32.4603977Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:32.4604420Z [cluster2]                     "f:metadata": {
2021-07-19T04:15:32.4604895Z [cluster2]                         "f:annotations": {
2021-07-19T04:15:32.4605322Z [cluster2]                             ".": {},
2021-07-19T04:15:32.4606170Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-19T04:15:32.4619360Z [cluster2]                         }
2021-07-19T04:15:32.4620183Z [cluster2]                     },
2021-07-19T04:15:32.4620626Z [cluster2]                     "f:spec": {
2021-07-19T04:15:32.4621136Z [cluster2]                         "f:conversion": {
2021-07-19T04:15:32.4621814Z [cluster2]                             ".": {},
2021-07-19T04:15:32.4622780Z [cluster2]                             "f:strategy": {}
2021-07-19T04:15:32.4623261Z [cluster2]                         },
2021-07-19T04:15:32.4623658Z [cluster2]                         "f:group": {},
2021-07-19T04:15:32.4624270Z [cluster2]                         "f:names": {
2021-07-19T04:15:32.4625126Z [cluster2]                             "f:kind": {},
2021-07-19T04:15:32.4625584Z [cluster2]                             "f:listKind": {},
2021-07-19T04:15:32.4626032Z [cluster2]                             "f:plural": {},
2021-07-19T04:15:32.4626494Z [cluster2]                             "f:singular": {}
2021-07-19T04:15:32.4626902Z [cluster2]                         },
2021-07-19T04:15:32.4627486Z [cluster2]                         "f:scope": {},
2021-07-19T04:15:32.4627968Z [cluster2]                         "f:versions": {}
2021-07-19T04:15:32.4628423Z [cluster2]                     },
2021-07-19T04:15:32.4629032Z [cluster2]                     "f:status": {
2021-07-19T04:15:32.4629541Z [cluster2]                         "f:storedVersions": {}
2021-07-19T04:15:32.4630036Z [cluster2]                     }
2021-07-19T04:15:32.4630420Z [cluster2]                 },
2021-07-19T04:15:32.4630879Z [cluster2]                 "manager": "subctl",
2021-07-19T04:15:32.4631400Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:32.4637728Z [cluster2]                 "time": "2021-07-19T04:14:02Z"
2021-07-19T04:15:32.4638141Z [cluster2]             }
2021-07-19T04:15:32.4638943Z [cluster2]         ],
2021-07-19T04:15:32.4639547Z [cluster2]         "name": "submariners.submariner.io",
2021-07-19T04:15:32.4640226Z [cluster2]         "resourceVersion": "1734",
2021-07-19T04:15:32.4641311Z [cluster2]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/submariners.submariner.io",
2021-07-19T04:15:32.4642965Z [cluster2]         "uid": "23fe25ee-faf9-4702-a5b8-254d5191de42"
2021-07-19T04:15:32.4643522Z [cluster2]     },
2021-07-19T04:15:32.4643886Z [cluster2]     "spec": {
2021-07-19T04:15:32.4644313Z [cluster2]         "conversion": {
2021-07-19T04:15:32.4644759Z [cluster2]             "strategy": "None"
2021-07-19T04:15:32.4645174Z [cluster2]         },
2021-07-19T04:15:32.4645608Z [cluster2]         "group": "submariner.io",
2021-07-19T04:15:32.4646896Z [cluster2]         "names": {
2021-07-19T04:15:32.4647425Z [cluster2]             "kind": "Submariner",
2021-07-19T04:15:32.4647954Z [cluster2]             "listKind": "SubmarinerList",
2021-07-19T04:15:32.4654099Z [cluster2]             "plural": "submariners",
2021-07-19T04:15:32.4654627Z [cluster2]             "singular": "submariner"
2021-07-19T04:15:32.4655083Z [cluster2]         },
2021-07-19T04:15:32.4655669Z [cluster2]         "scope": "Namespaced",
2021-07-19T04:15:32.4656315Z [cluster2]         "versions": [
2021-07-19T04:15:32.4656724Z [cluster2]             {
2021-07-19T04:15:32.4657147Z [cluster2]                 "name": "v1alpha1",
2021-07-19T04:15:32.4657580Z [cluster2]                 "schema": {
2021-07-19T04:15:32.4658081Z [cluster2]                     "openAPIV3Schema": {
2021-07-19T04:15:32.4659108Z [cluster2]                         "description": "Submariner is the Schema for the submariners API",
2021-07-19T04:15:32.4659806Z [cluster2]                         "properties": {
2021-07-19T04:15:32.4660344Z [cluster2]                             "apiVersion": {
2021-07-19T04:15:32.4662717Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-19T04:15:32.4664319Z [cluster2]                                 "type": "string"
2021-07-19T04:15:32.4664764Z [cluster2]                             },
2021-07-19T04:15:32.4665163Z [cluster2]                             "kind": {
2021-07-19T04:15:32.4667103Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-19T04:15:32.4669077Z [cluster2]                                 "type": "string"
2021-07-19T04:15:32.4669530Z [cluster2]                             },
2021-07-19T04:15:32.4670167Z [cluster2]                             "metadata": {
2021-07-19T04:15:32.4670659Z [cluster2]                                 "type": "object"
2021-07-19T04:15:32.4671124Z [cluster2]                             },
2021-07-19T04:15:32.4671558Z [cluster2]                             "spec": {
2021-07-19T04:15:32.4672641Z [cluster2]                                 "description": "SubmarinerSpec defines the desired state of Submariner",
2021-07-19T04:15:32.4673366Z [cluster2]                                 "properties": {
2021-07-19T04:15:32.4673832Z [cluster2]                                     "broker": {
2021-07-19T04:15:32.4674306Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4674739Z [cluster2]                                     },
2021-07-19T04:15:32.4675272Z [cluster2]                                     "brokerK8sApiServer": {
2021-07-19T04:15:32.4676000Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4676468Z [cluster2]                                     },
2021-07-19T04:15:32.4677056Z [cluster2]                                     "brokerK8sApiServerToken": {
2021-07-19T04:15:32.4677828Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4678469Z [cluster2]                                     },
2021-07-19T04:15:32.4679152Z [cluster2]                                     "brokerK8sCA": {
2021-07-19T04:15:32.4679682Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4680321Z [cluster2]                                     },
2021-07-19T04:15:32.4680960Z [cluster2]                                     "brokerK8sRemoteNamespace": {
2021-07-19T04:15:32.4682134Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4682945Z [cluster2]                                     },
2021-07-19T04:15:32.4683409Z [cluster2]                                     "cableDriver": {
2021-07-19T04:15:32.4684056Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4684489Z [cluster2]                                     },
2021-07-19T04:15:32.4684937Z [cluster2]                                     "ceIPSecDebug": {
2021-07-19T04:15:32.4685535Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.4685977Z [cluster2]                                     },
2021-07-19T04:15:32.4686519Z [cluster2]                                     "ceIPSecForceUDPEncaps": {
2021-07-19T04:15:32.4688001Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.4695324Z [cluster2]                                     },
2021-07-19T04:15:32.4696822Z [cluster2]                                     "ceIPSecIKEPort": {
2021-07-19T04:15:32.4697376Z [cluster2]                                         "type": "integer"
2021-07-19T04:15:32.4697840Z [cluster2]                                     },
2021-07-19T04:15:32.4698751Z [cluster2]                                     "ceIPSecNATTPort": {
2021-07-19T04:15:32.4699801Z [cluster2]                                         "type": "integer"
2021-07-19T04:15:32.4700257Z [cluster2]                                     },
2021-07-19T04:15:32.4700750Z [cluster2]                                     "ceIPSecPSK": {
2021-07-19T04:15:32.4701274Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4702130Z [cluster2]                                     },
2021-07-19T04:15:32.4702899Z [cluster2]                                     "ceIPSecPreferredServer": {
2021-07-19T04:15:32.4703509Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.4703957Z [cluster2]                                     },
2021-07-19T04:15:32.4704570Z [cluster2]                                     "clusterCIDR": {
2021-07-19T04:15:32.4705225Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4705646Z [cluster2]                                     },
2021-07-19T04:15:32.4706267Z [cluster2]                                     "clusterID": {
2021-07-19T04:15:32.4706757Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4707187Z [cluster2]                                     },
2021-07-19T04:15:32.4707643Z [cluster2]                                     "colorCodes": {
2021-07-19T04:15:32.4708306Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4708997Z [cluster2]                                     },
2021-07-19T04:15:32.4709587Z [cluster2]                                     "connectionHealthCheck": {
2021-07-19T04:15:32.4710408Z [cluster2]                                         "properties": {
2021-07-19T04:15:32.4711092Z [cluster2]                                             "enabled": {
2021-07-19T04:15:32.4711600Z [cluster2]                                                 "type": "boolean"
2021-07-19T04:15:32.4721364Z [cluster2]                                             },
2021-07-19T04:15:32.4722264Z [cluster2]                                             "intervalSeconds": {
2021-07-19T04:15:32.4723154Z [cluster2]                                                 "description": "The interval at which health check pings are sent.",
2021-07-19T04:15:32.4723815Z [cluster2]                                                 "format": "int64",
2021-07-19T04:15:32.4724330Z [cluster2]                                                 "type": "integer"
2021-07-19T04:15:32.4724801Z [cluster2]                                             },
2021-07-19T04:15:32.4725314Z [cluster2]                                             "maxPacketLossCount": {
2021-07-19T04:15:32.4726189Z [cluster2]                                                 "description": "The maximum number of packets lost at which the health checker will mark the connection as down.",
2021-07-19T04:15:32.4726977Z [cluster2]                                                 "format": "int64",
2021-07-19T04:15:32.4727490Z [cluster2]                                                 "type": "integer"
2021-07-19T04:15:32.4727959Z [cluster2]                                             }
2021-07-19T04:15:32.4728353Z [cluster2]                                         },
2021-07-19T04:15:32.4729331Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.4729817Z [cluster2]                                     },
2021-07-19T04:15:32.4730384Z [cluster2]                                     "coreDNSCustomConfig": {
2021-07-19T04:15:32.4730998Z [cluster2]                                         "properties": {
2021-07-19T04:15:32.4731568Z [cluster2]                                             "configMapName": {
2021-07-19T04:15:32.4732616Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.4733258Z [cluster2]                                             },
2021-07-19T04:15:32.4733721Z [cluster2]                                             "namespace": {
2021-07-19T04:15:32.4734218Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.4734675Z [cluster2]                                             }
2021-07-19T04:15:32.4735067Z [cluster2]                                         },
2021-07-19T04:15:32.4735721Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.4736169Z [cluster2]                                     },
2021-07-19T04:15:32.4736648Z [cluster2]                                     "customDomains": {
2021-07-19T04:15:32.4737145Z [cluster2]                                         "items": {
2021-07-19T04:15:32.4737605Z [cluster2]                                             "type": "string"
2021-07-19T04:15:32.4738288Z [cluster2]                                         },
2021-07-19T04:15:32.4739081Z [cluster2]                                         "type": "array",
2021-07-19T04:15:32.4740089Z [cluster2]                                         "x-kubernetes-list-type": "set"
2021-07-19T04:15:32.4742366Z [cluster2]                                     },
2021-07-19T04:15:32.4742965Z [cluster2]                                     "debug": {
2021-07-19T04:15:32.4743426Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.4744417Z [cluster2]                                     },
2021-07-19T04:15:32.4744897Z [cluster2]                                     "globalCIDR": {
2021-07-19T04:15:32.4746037Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4746955Z [cluster2]                                     },
2021-07-19T04:15:32.4747426Z [cluster2]                                     "imageOverrides": {
2021-07-19T04:15:32.4748433Z [cluster2]                                         "additionalProperties": {
2021-07-19T04:15:32.4786042Z [cluster2]                                             "type": "string"
2021-07-19T04:15:32.4786634Z [cluster2]                                         },
2021-07-19T04:15:32.4787081Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.4787536Z [cluster2]                                     },
2021-07-19T04:15:32.4788066Z [cluster2]                                     "loadBalancerEnabled": {
2021-07-19T04:15:32.4789137Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.4789628Z [cluster2]                                     },
2021-07-19T04:15:32.4790092Z [cluster2]                                     "namespace": {
2021-07-19T04:15:32.4790614Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4791069Z [cluster2]                                     },
2021-07-19T04:15:32.4791552Z [cluster2]                                     "natEnabled": {
2021-07-19T04:15:32.4792232Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.4792689Z [cluster2]                                     },
2021-07-19T04:15:32.4793144Z [cluster2]                                     "repository": {
2021-07-19T04:15:32.4793624Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4794069Z [cluster2]                                     },
2021-07-19T04:15:32.4794679Z [cluster2]                                     "serviceCIDR": {
2021-07-19T04:15:32.4795207Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4795637Z [cluster2]                                     },
2021-07-19T04:15:32.4796214Z [cluster2]                                     "serviceDiscoveryEnabled": {
2021-07-19T04:15:32.4797006Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.4797434Z [cluster2]                                     },
2021-07-19T04:15:32.4797851Z [cluster2]                                     "version": {
2021-07-19T04:15:32.4798303Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4798935Z [cluster2]                                     }
2021-07-19T04:15:32.4799341Z [cluster2]                                 },
2021-07-19T04:15:32.4799990Z [cluster2]                                 "required": [
2021-07-19T04:15:32.4800482Z [cluster2]                                     "broker",
2021-07-19T04:15:32.4801136Z [cluster2]                                     "brokerK8sApiServer",
2021-07-19T04:15:32.4802582Z [cluster2]                                     "brokerK8sApiServerToken",
2021-07-19T04:15:32.4803316Z [cluster2]                                     "brokerK8sCA",
2021-07-19T04:15:32.4804112Z [cluster2]                                     "brokerK8sRemoteNamespace",
2021-07-19T04:15:32.4804723Z [cluster2]                                     "ceIPSecDebug",
2021-07-19T04:15:32.4805231Z [cluster2]                                     "ceIPSecPSK",
2021-07-19T04:15:32.4805811Z [cluster2]                                     "clusterCIDR",
2021-07-19T04:15:32.4806288Z [cluster2]                                     "clusterID",
2021-07-19T04:15:32.4806725Z [cluster2]                                     "debug",
2021-07-19T04:15:32.4807187Z [cluster2]                                     "namespace",
2021-07-19T04:15:32.4807850Z [cluster2]                                     "natEnabled",
2021-07-19T04:15:32.4808553Z [cluster2]                                     "serviceCIDR"
2021-07-19T04:15:32.4809337Z [cluster2]                                 ],
2021-07-19T04:15:32.4809827Z [cluster2]                                 "type": "object"
2021-07-19T04:15:32.4810329Z [cluster2]                             },
2021-07-19T04:15:32.4810783Z [cluster2]                             "status": {
2021-07-19T04:15:32.4812411Z [cluster2]                                 "description": "SubmarinerStatus defines the observed state of Submariner",
2021-07-19T04:15:32.4813146Z [cluster2]                                 "properties": {
2021-07-19T04:15:32.4813657Z [cluster2]                                     "clusterCIDR": {
2021-07-19T04:15:32.4814161Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4814591Z [cluster2]                                     },
2021-07-19T04:15:32.4816024Z [cluster2]                                     "clusterID": {
2021-07-19T04:15:32.4816609Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4817139Z [cluster2]                                     },
2021-07-19T04:15:32.4817662Z [cluster2]                                     "colorCodes": {
2021-07-19T04:15:32.4818247Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.4819375Z [cluster2]                                     },
2021-07-19T04:15:32.4820090Z [cluster2]                                     "deploymentInfo": {
2021-07-19T04:15:32.4820762Z [cluster2]                                         "properties": {
2021-07-19T04:15:32.4821532Z [cluster2]                                             "cloudProvider": {
2021-07-19T04:15:32.4822097Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.4822568Z [cluster2]                                             },
2021-07-19T04:15:32.4823483Z [cluster2]                                             "kubernetesType": {
2021-07-19T04:15:32.4824674Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.4825270Z [cluster2]                                             },
2021-07-19T04:15:32.4826401Z [cluster2]                                             "kubernetesTypeVersion": {
2021-07-19T04:15:32.4827538Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.4828367Z [cluster2]                                             },
2021-07-19T04:15:32.4829126Z [cluster2]                                             "kubernetesVersion": {
2021-07-19T04:15:32.4829796Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.4830313Z [cluster2]                                             }
2021-07-19T04:15:32.4830799Z [cluster2]                                         },
2021-07-19T04:15:32.4831404Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.4832441Z [cluster2]                                     },
2021-07-19T04:15:32.4833552Z [cluster2]                                     "gatewayDaemonSetStatus": {
2021-07-19T04:15:32.4834192Z [cluster2]                                         "properties": {
2021-07-19T04:15:32.4835001Z [cluster2]                                             "lastResourceVersion": {
2021-07-19T04:15:32.4835821Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.4836350Z [cluster2]                                             },
2021-07-19T04:15:32.4837124Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-19T04:15:32.4838328Z [cluster2]                                                 "type": "boolean"
2021-07-19T04:15:32.4839202Z [cluster2]                                             },
2021-07-19T04:15:32.4839871Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-19T04:15:32.4840592Z [cluster2]                                                 "items": {
2021-07-19T04:15:32.4841893Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-19T04:15:32.4843383Z [cluster2]                                                     "properties": {
2021-07-19T04:15:32.4843940Z [cluster2]                                                         "running": {
2021-07-19T04:15:32.4844924Z [cluster2]                                                             "description": "Details about a running container",
2021-07-19T04:15:32.4845607Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.4846148Z [cluster2]                                                                 "startedAt": {
2021-07-19T04:15:32.4847829Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-19T04:15:32.4849228Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.4849869Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4850455Z [cluster2]                                                                 }
2021-07-19T04:15:32.4850972Z [cluster2]                                                             },
2021-07-19T04:15:32.4852300Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.4900781Z [cluster2]                                                         },
2021-07-19T04:15:32.4901356Z [cluster2]                                                         "terminated": {
2021-07-19T04:15:32.4902153Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-19T04:15:32.4903549Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.4904142Z [cluster2]                                                                 "containerID": {
2021-07-19T04:15:32.4905358Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-19T04:15:32.4906126Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4906659Z [cluster2]                                                                 },
2021-07-19T04:15:32.4907180Z [cluster2]                                                                 "exitCode": {
2021-07-19T04:15:32.4907923Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-19T04:15:32.4908856Z [cluster2]                                                                     "format": "int32",
2021-07-19T04:15:32.4909453Z [cluster2]                                                                     "type": "integer"
2021-07-19T04:15:32.4909978Z [cluster2]                                                                 },
2021-07-19T04:15:32.4910515Z [cluster2]                                                                 "finishedAt": {
2021-07-19T04:15:32.4911255Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-19T04:15:32.4912566Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.4913140Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4913656Z [cluster2]                                                                 },
2021-07-19T04:15:32.4914164Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.4914922Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-19T04:15:32.4917248Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4917882Z [cluster2]                                                                 },
2021-07-19T04:15:32.4918632Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.4919484Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-19T04:15:32.4920311Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4920881Z [cluster2]                                                                 },
2021-07-19T04:15:32.4921552Z [cluster2]                                                                 "signal": {
2021-07-19T04:15:32.4922778Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-19T04:15:32.4923524Z [cluster2]                                                                     "format": "int32",
2021-07-19T04:15:32.4924081Z [cluster2]                                                                     "type": "integer"
2021-07-19T04:15:32.4924674Z [cluster2]                                                                 },
2021-07-19T04:15:32.4925179Z [cluster2]                                                                 "startedAt": {
2021-07-19T04:15:32.4925952Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-19T04:15:32.4927061Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.4927799Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4928342Z [cluster2]                                                                 }
2021-07-19T04:15:32.4928795Z [cluster2]                                                             },
2021-07-19T04:15:32.4929303Z [cluster2]                                                             "required": [
2021-07-19T04:15:32.4930151Z [cluster2]                                                                 "exitCode"
2021-07-19T04:15:32.4930696Z [cluster2]                                                             ],
2021-07-19T04:15:32.4931390Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.4931880Z [cluster2]                                                         },
2021-07-19T04:15:32.4932367Z [cluster2]                                                         "waiting": {
2021-07-19T04:15:32.4933019Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-19T04:15:32.4933712Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.4934252Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.4934998Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-19T04:15:32.4936189Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4936688Z [cluster2]                                                                 },
2021-07-19T04:15:32.4937186Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.4937880Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-19T04:15:32.4938982Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4939494Z [cluster2]                                                                 }
2021-07-19T04:15:32.4939940Z [cluster2]                                                             },
2021-07-19T04:15:32.4940448Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.4975868Z [cluster2]                                                         }
2021-07-19T04:15:32.4976364Z [cluster2]                                                     },
2021-07-19T04:15:32.4976826Z [cluster2]                                                     "type": "object"
2021-07-19T04:15:32.4977297Z [cluster2]                                                 },
2021-07-19T04:15:32.4977756Z [cluster2]                                                 "type": "array"
2021-07-19T04:15:32.4978387Z [cluster2]                                             },
2021-07-19T04:15:32.4979497Z [cluster2]                                             "status": {
2021-07-19T04:15:32.4980261Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-19T04:15:32.4981071Z [cluster2]                                                 "properties": {
2021-07-19T04:15:32.4981862Z [cluster2]                                                     "collisionCount": {
2021-07-19T04:15:32.4983442Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-19T04:15:32.4984566Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.4985061Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.4985668Z [cluster2]                                                     },
2021-07-19T04:15:32.4986147Z [cluster2]                                                     "conditions": {
2021-07-19T04:15:32.4987290Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-19T04:15:32.4988239Z [cluster2]                                                         "items": {
2021-07-19T04:15:32.4989453Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-19T04:15:32.4990330Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.4990967Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-19T04:15:32.4991824Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-19T04:15:32.4993165Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.4993705Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4994189Z [cluster2]                                                                 },
2021-07-19T04:15:32.4994661Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.4995537Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-19T04:15:32.4996292Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4996758Z [cluster2]                                                                 },
2021-07-19T04:15:32.4997230Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.4998166Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-19T04:15:32.4998838Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.4999315Z [cluster2]                                                                 },
2021-07-19T04:15:32.4999769Z [cluster2]                                                                 "status": {
2021-07-19T04:15:32.5000457Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-19T04:15:32.5001152Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5001626Z [cluster2]                                                                 },
2021-07-19T04:15:32.5002089Z [cluster2]                                                                 "type": {
2021-07-19T04:15:32.5002694Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-19T04:15:32.5003337Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5003801Z [cluster2]                                                                 }
2021-07-19T04:15:32.5004251Z [cluster2]                                                             },
2021-07-19T04:15:32.5004724Z [cluster2]                                                             "required": [
2021-07-19T04:15:32.5005204Z [cluster2]                                                                 "status",
2021-07-19T04:15:32.5005685Z [cluster2]                                                                 "type"
2021-07-19T04:15:32.5006218Z [cluster2]                                                             ],
2021-07-19T04:15:32.5006718Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5007180Z [cluster2]                                                         },
2021-07-19T04:15:32.5007653Z [cluster2]                                                         "type": "array"
2021-07-19T04:15:32.5008112Z [cluster2]                                                     },
2021-07-19T04:15:32.5008676Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-19T04:15:32.5010551Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5011913Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5012805Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5013286Z [cluster2]                                                     },
2021-07-19T04:15:32.5013852Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-19T04:15:32.5015172Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5018357Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5019027Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5019601Z [cluster2]                                                     },
2021-07-19T04:15:32.5020192Z [cluster2]                                                     "numberAvailable": {
2021-07-19T04:15:32.5021758Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.5022951Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5023450Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5023935Z [cluster2]                                                     },
2021-07-19T04:15:32.5024453Z [cluster2]                                                     "numberMisscheduled": {
2021-07-19T04:15:32.5025666Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5026966Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5027488Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5027979Z [cluster2]                                                     },
2021-07-19T04:15:32.5028460Z [cluster2]                                                     "numberReady": {
2021-07-19T04:15:32.5029349Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-19T04:15:32.5030206Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5030719Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5031203Z [cluster2]                                                     },
2021-07-19T04:15:32.5031876Z [cluster2]                                                     "numberUnavailable": {
2021-07-19T04:15:32.5033014Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.5034052Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5034756Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5035260Z [cluster2]                                                     },
2021-07-19T04:15:32.5035799Z [cluster2]                                                     "observedGeneration": {
2021-07-19T04:15:32.5036804Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-19T04:15:32.5072452Z [cluster2]                                                         "format": "int64",
2021-07-19T04:15:32.5073010Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5073529Z [cluster2]                                                     },
2021-07-19T04:15:32.5074140Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-19T04:15:32.5075012Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-19T04:15:32.5076334Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5076871Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5077387Z [cluster2]                                                     }
2021-07-19T04:15:32.5077821Z [cluster2]                                                 },
2021-07-19T04:15:32.5078310Z [cluster2]                                                 "required": [
2021-07-19T04:15:32.5078947Z [cluster2]                                                     "currentNumberScheduled",
2021-07-19T04:15:32.5079725Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-19T04:15:32.5080463Z [cluster2]                                                     "numberMisscheduled",
2021-07-19T04:15:32.5081275Z [cluster2]                                                     "numberReady"
2021-07-19T04:15:32.5081803Z [cluster2]                                                 ],
2021-07-19T04:15:32.5082282Z [cluster2]                                                 "type": "object"
2021-07-19T04:15:32.5082773Z [cluster2]                                             }
2021-07-19T04:15:32.5083200Z [cluster2]                                         },
2021-07-19T04:15:32.5083917Z [cluster2]                                         "required": [
2021-07-19T04:15:32.5084810Z [cluster2]                                             "mismatchedContainerImages"
2021-07-19T04:15:32.5085495Z [cluster2]                                         ],
2021-07-19T04:15:32.5086171Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.5086802Z [cluster2]                                     },
2021-07-19T04:15:32.5087273Z [cluster2]                                     "gateways": {
2021-07-19T04:15:32.5087922Z [cluster2]                                         "items": {
2021-07-19T04:15:32.5088459Z [cluster2]                                             "properties": {
2021-07-19T04:15:32.5089025Z [cluster2]                                                 "connections": {
2021-07-19T04:15:32.5089576Z [cluster2]                                                     "items": {
2021-07-19T04:15:32.5090130Z [cluster2]                                                         "properties": {
2021-07-19T04:15:32.5090705Z [cluster2]                                                             "endpoint": {
2021-07-19T04:15:32.5091389Z [cluster2]                                                                 "properties": {
2021-07-19T04:15:32.5091971Z [cluster2]                                                                     "backend": {
2021-07-19T04:15:32.5092546Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5093261Z [cluster2]                                                                     },
2021-07-19T04:15:32.5093846Z [cluster2]                                                                     "backend_config": {
2021-07-19T04:15:32.5094598Z [cluster2]                                                                         "additionalProperties": {
2021-07-19T04:15:32.5095319Z [cluster2]                                                                             "type": "string"
2021-07-19T04:15:32.5096379Z [cluster2]                                                                         },
2021-07-19T04:15:32.5097005Z [cluster2]                                                                         "type": "object"
2021-07-19T04:15:32.5097552Z [cluster2]                                                                     },
2021-07-19T04:15:32.5098080Z [cluster2]                                                                     "cable_name": {
2021-07-19T04:15:32.5098639Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5099183Z [cluster2]                                                                     },
2021-07-19T04:15:32.5099798Z [cluster2]                                                                     "cluster_id": {
2021-07-19T04:15:32.5100368Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5100893Z [cluster2]                                                                     },
2021-07-19T04:15:32.5101474Z [cluster2]                                                                     "healthCheckIP": {
2021-07-19T04:15:32.5102098Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5102613Z [cluster2]                                                                     },
2021-07-19T04:15:32.5103143Z [cluster2]                                                                     "hostname": {
2021-07-19T04:15:32.5103699Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5104241Z [cluster2]                                                                     },
2021-07-19T04:15:32.5109664Z [cluster2]                                                                     "nat_enabled": {
2021-07-19T04:15:32.5110315Z [cluster2]                                                                         "type": "boolean"
2021-07-19T04:15:32.5110863Z [cluster2]                                                                     },
2021-07-19T04:15:32.5111400Z [cluster2]                                                                     "private_ip": {
2021-07-19T04:15:32.5111983Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5112512Z [cluster2]                                                                     },
2021-07-19T04:15:32.5113043Z [cluster2]                                                                     "public_ip": {
2021-07-19T04:15:32.5113619Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5114147Z [cluster2]                                                                     },
2021-07-19T04:15:32.5114669Z [cluster2]                                                                     "subnets": {
2021-07-19T04:15:32.5115201Z [cluster2]                                                                         "items": {
2021-07-19T04:15:32.5116311Z [cluster2]                                                                             "type": "string"
2021-07-19T04:15:32.5116861Z [cluster2]                                                                         },
2021-07-19T04:15:32.5117558Z [cluster2]                                                                         "type": "array"
2021-07-19T04:15:32.5118080Z [cluster2]                                                                     }
2021-07-19T04:15:32.5118549Z [cluster2]                                                                 },
2021-07-19T04:15:32.5120582Z [cluster2]                                                                 "required": [
2021-07-19T04:15:32.5121176Z [cluster2]                                                                     "backend",
2021-07-19T04:15:32.5121777Z [cluster2]                                                                     "cable_name",
2021-07-19T04:15:32.5122385Z [cluster2]                                                                     "cluster_id",
2021-07-19T04:15:32.5122986Z [cluster2]                                                                     "hostname",
2021-07-19T04:15:32.5123603Z [cluster2]                                                                     "nat_enabled",
2021-07-19T04:15:32.5124203Z [cluster2]                                                                     "private_ip",
2021-07-19T04:15:32.5124812Z [cluster2]                                                                     "public_ip",
2021-07-19T04:15:32.5125392Z [cluster2]                                                                     "subnets"
2021-07-19T04:15:32.5126137Z [cluster2]                                                                 ],
2021-07-19T04:15:32.5126713Z [cluster2]                                                                 "type": "object"
2021-07-19T04:15:32.5127262Z [cluster2]                                                             },
2021-07-19T04:15:32.5127835Z [cluster2]                                                             "latencyRTT": {
2021-07-19T04:15:32.5129162Z [cluster2]                                                                 "description": "LatencySpec describes the round trip time information for a packet between the gateway pods of two clusters.",
2021-07-19T04:15:32.5130131Z [cluster2]                                                                 "properties": {
2021-07-19T04:15:32.5131047Z [cluster2]                                                                     "average": {
2021-07-19T04:15:32.5131585Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5132112Z [cluster2]                                                                     },
2021-07-19T04:15:32.5132586Z [cluster2]                                                                     "last": {
2021-07-19T04:15:32.5133333Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5134064Z [cluster2]                                                                     },
2021-07-19T04:15:32.5134606Z [cluster2]                                                                     "max": {
2021-07-19T04:15:32.5135196Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5136127Z [cluster2]                                                                     },
2021-07-19T04:15:32.5136952Z [cluster2]                                                                     "min": {
2021-07-19T04:15:32.5137481Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5137999Z [cluster2]                                                                     },
2021-07-19T04:15:32.5138500Z [cluster2]                                                                     "stdDev": {
2021-07-19T04:15:32.5139032Z [cluster2]                                                                         "type": "string"
2021-07-19T04:15:32.5139652Z [cluster2]                                                                     }
2021-07-19T04:15:32.5140129Z [cluster2]                                                                 },
2021-07-19T04:15:32.5140638Z [cluster2]                                                                 "type": "object"
2021-07-19T04:15:32.5141132Z [cluster2]                                                             },
2021-07-19T04:15:32.5201560Z [cluster2]                                                             "status": {
2021-07-19T04:15:32.5202150Z [cluster2]                                                                 "type": "string"
2021-07-19T04:15:32.5202715Z [cluster2]                                                             },
2021-07-19T04:15:32.5203275Z [cluster2]                                                             "statusMessage": {
2021-07-19T04:15:32.5203865Z [cluster2]                                                                 "type": "string"
2021-07-19T04:15:32.5204398Z [cluster2]                                                             },
2021-07-19T04:15:32.5204904Z [cluster2]                                                             "usingIP": {
2021-07-19T04:15:32.5205482Z [cluster2]                                                                 "type": "string"
2021-07-19T04:15:32.5206129Z [cluster2]                                                             },
2021-07-19T04:15:32.5206639Z [cluster2]                                                             "usingNAT": {
2021-07-19T04:15:32.5207400Z [cluster2]                                                                 "type": "boolean"
2021-07-19T04:15:32.5207922Z [cluster2]                                                             }
2021-07-19T04:15:32.5208409Z [cluster2]                                                         },
2021-07-19T04:15:32.5208907Z [cluster2]                                                         "required": [
2021-07-19T04:15:32.5209478Z [cluster2]                                                             "endpoint",
2021-07-19T04:15:32.5210030Z [cluster2]                                                             "status",
2021-07-19T04:15:32.5210590Z [cluster2]                                                             "statusMessage"
2021-07-19T04:15:32.5211150Z [cluster2]                                                         ],
2021-07-19T04:15:32.5211657Z [cluster2]                                                         "type": "object"
2021-07-19T04:15:32.5212178Z [cluster2]                                                     },
2021-07-19T04:15:32.5212672Z [cluster2]                                                     "type": "array"
2021-07-19T04:15:32.5213189Z [cluster2]                                                 },
2021-07-19T04:15:32.5213913Z [cluster2]                                                 "haStatus": {
2021-07-19T04:15:32.5214483Z [cluster2]                                                     "type": "string"
2021-07-19T04:15:32.5214998Z [cluster2]                                                 },
2021-07-19T04:15:32.5215724Z [cluster2]                                                 "localEndpoint": {
2021-07-19T04:15:32.5216495Z [cluster2]                                                     "properties": {
2021-07-19T04:15:32.5217059Z [cluster2]                                                         "backend": {
2021-07-19T04:15:32.5217606Z [cluster2]                                                             "type": "string"
2021-07-19T04:15:32.5218121Z [cluster2]                                                         },
2021-07-19T04:15:32.5218797Z [cluster2]                                                         "backend_config": {
2021-07-19T04:15:32.5219435Z [cluster2]                                                             "additionalProperties": {
2021-07-19T04:15:32.5221484Z [cluster2]                                                                 "type": "string"
2021-07-19T04:15:32.5222228Z [cluster2]                                                             },
2021-07-19T04:15:32.5222802Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5223525Z [cluster2]                                                         },
2021-07-19T04:15:32.5224005Z [cluster2]                                                         "cable_name": {
2021-07-19T04:15:32.5224550Z [cluster2]                                                             "type": "string"
2021-07-19T04:15:32.5225042Z [cluster2]                                                         },
2021-07-19T04:15:32.5225933Z [cluster2]                                                         "cluster_id": {
2021-07-19T04:15:32.5226473Z [cluster2]                                                             "type": "string"
2021-07-19T04:15:32.5226961Z [cluster2]                                                         },
2021-07-19T04:15:32.5227672Z [cluster2]                                                         "healthCheckIP": {
2021-07-19T04:15:32.5228262Z [cluster2]                                                             "type": "string"
2021-07-19T04:15:32.5228780Z [cluster2]                                                         },
2021-07-19T04:15:32.5229278Z [cluster2]                                                         "hostname": {
2021-07-19T04:15:32.5229839Z [cluster2]                                                             "type": "string"
2021-07-19T04:15:32.5230366Z [cluster2]                                                         },
2021-07-19T04:15:32.5230972Z [cluster2]                                                         "nat_enabled": {
2021-07-19T04:15:32.5231541Z [cluster2]                                                             "type": "boolean"
2021-07-19T04:15:32.5232051Z [cluster2]                                                         },
2021-07-19T04:15:32.5232571Z [cluster2]                                                         "private_ip": {
2021-07-19T04:15:32.5233133Z [cluster2]                                                             "type": "string"
2021-07-19T04:15:32.5233652Z [cluster2]                                                         },
2021-07-19T04:15:32.5234164Z [cluster2]                                                         "public_ip": {
2021-07-19T04:15:32.5234704Z [cluster2]                                                             "type": "string"
2021-07-19T04:15:32.5235222Z [cluster2]                                                         },
2021-07-19T04:15:32.5235725Z [cluster2]                                                         "subnets": {
2021-07-19T04:15:32.5236257Z [cluster2]                                                             "items": {
2021-07-19T04:15:32.5236789Z [cluster2]                                                                 "type": "string"
2021-07-19T04:15:32.5237318Z [cluster2]                                                             },
2021-07-19T04:15:32.5237988Z [cluster2]                                                             "type": "array"
2021-07-19T04:15:32.5238481Z [cluster2]                                                         }
2021-07-19T04:15:32.5238941Z [cluster2]                                                     },
2021-07-19T04:15:32.5239419Z [cluster2]                                                     "required": [
2021-07-19T04:15:32.5239947Z [cluster2]                                                         "backend",
2021-07-19T04:15:32.5240652Z [cluster2]                                                         "cable_name",
2021-07-19T04:15:32.5241210Z [cluster2]                                                         "cluster_id",
2021-07-19T04:15:32.5241906Z [cluster2]                                                         "hostname",
2021-07-19T04:15:32.5242426Z [cluster2]                                                         "nat_enabled",
2021-07-19T04:15:32.5243325Z [cluster2]                                                         "private_ip",
2021-07-19T04:15:32.5244104Z [cluster2]                                                         "public_ip",
2021-07-19T04:15:32.5244642Z [cluster2]                                                         "subnets"
2021-07-19T04:15:32.5256952Z [cluster2]                                                     ],
2021-07-19T04:15:32.5257494Z [cluster2]                                                     "type": "object"
2021-07-19T04:15:32.5258273Z [cluster2]                                                 },
2021-07-19T04:15:32.5258799Z [cluster2]                                                 "statusFailure": {
2021-07-19T04:15:32.5259366Z [cluster2]                                                     "type": "string"
2021-07-19T04:15:32.5259843Z [cluster2]                                                 },
2021-07-19T04:15:32.5260321Z [cluster2]                                                 "version": {
2021-07-19T04:15:32.5260827Z [cluster2]                                                     "type": "string"
2021-07-19T04:15:32.5261321Z [cluster2]                                                 }
2021-07-19T04:15:32.5261748Z [cluster2]                                             },
2021-07-19T04:15:32.5262234Z [cluster2]                                             "required": [
2021-07-19T04:15:32.5262770Z [cluster2]                                                 "connections",
2021-07-19T04:15:32.5263291Z [cluster2]                                                 "haStatus",
2021-07-19T04:15:32.5263841Z [cluster2]                                                 "localEndpoint",
2021-07-19T04:15:32.5264614Z [cluster2]                                                 "statusFailure",
2021-07-19T04:15:32.5265167Z [cluster2]                                                 "version"
2021-07-19T04:15:32.5265623Z [cluster2]                                             ],
2021-07-19T04:15:32.5266114Z [cluster2]                                             "type": "object"
2021-07-19T04:15:32.5266591Z [cluster2]                                         },
2021-07-19T04:15:32.5267054Z [cluster2]                                         "type": "array"
2021-07-19T04:15:32.5267527Z [cluster2]                                     },
2021-07-19T04:15:32.5267992Z [cluster2]                                     "globalCIDR": {
2021-07-19T04:15:32.5268519Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.5268977Z [cluster2]                                     },
2021-07-19T04:15:32.5269599Z [cluster2]                                     "globalnetDaemonSetStatus": {
2021-07-19T04:15:32.5270275Z [cluster2]                                         "properties": {
2021-07-19T04:15:32.5270902Z [cluster2]                                             "lastResourceVersion": {
2021-07-19T04:15:32.5271664Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.5272133Z [cluster2]                                             },
2021-07-19T04:15:32.5272783Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-19T04:15:32.5273469Z [cluster2]                                                 "type": "boolean"
2021-07-19T04:15:32.5273962Z [cluster2]                                             },
2021-07-19T04:15:32.5274562Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-19T04:15:32.5275205Z [cluster2]                                                 "items": {
2021-07-19T04:15:32.5276490Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-19T04:15:32.5278018Z [cluster2]                                                     "properties": {
2021-07-19T04:15:32.5278627Z [cluster2]                                                         "running": {
2021-07-19T04:15:32.5279413Z [cluster2]                                                             "description": "Details about a running container",
2021-07-19T04:15:32.5280262Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5280810Z [cluster2]                                                                 "startedAt": {
2021-07-19T04:15:32.5282057Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-19T04:15:32.5283121Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5283677Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5284283Z [cluster2]                                                                 }
2021-07-19T04:15:32.5284734Z [cluster2]                                                             },
2021-07-19T04:15:32.5285221Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5285715Z [cluster2]                                                         },
2021-07-19T04:15:32.5286189Z [cluster2]                                                         "terminated": {
2021-07-19T04:15:32.5286874Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-19T04:15:32.5287545Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5288289Z [cluster2]                                                                 "containerID": {
2021-07-19T04:15:32.5289356Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-19T04:15:32.5290087Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5290584Z [cluster2]                                                                 },
2021-07-19T04:15:32.5291064Z [cluster2]                                                                 "exitCode": {
2021-07-19T04:15:32.5291796Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-19T04:15:32.5292520Z [cluster2]                                                                     "format": "int32",
2021-07-19T04:15:32.5293047Z [cluster2]                                                                     "type": "integer"
2021-07-19T04:15:32.5293554Z [cluster2]                                                                 },
2021-07-19T04:15:32.5294273Z [cluster2]                                                                 "finishedAt": {
2021-07-19T04:15:32.5294999Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-19T04:15:32.5296277Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5296833Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5297328Z [cluster2]                                                                 },
2021-07-19T04:15:32.5298143Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.5298887Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-19T04:15:32.5299625Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5300282Z [cluster2]                                                                 },
2021-07-19T04:15:32.5300984Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.5302003Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-19T04:15:32.5302969Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5304055Z [cluster2]                                                                 },
2021-07-19T04:15:32.5304741Z [cluster2]                                                                 "signal": {
2021-07-19T04:15:32.5305878Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-19T04:15:32.5306772Z [cluster2]                                                                     "format": "int32",
2021-07-19T04:15:32.5307486Z [cluster2]                                                                     "type": "integer"
2021-07-19T04:15:32.5307973Z [cluster2]                                                                 },
2021-07-19T04:15:32.5308470Z [cluster2]                                                                 "startedAt": {
2021-07-19T04:15:32.5309219Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-19T04:15:32.5310229Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5310802Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5311401Z [cluster2]                                                                 }
2021-07-19T04:15:32.5311851Z [cluster2]                                                             },
2021-07-19T04:15:32.5312322Z [cluster2]                                                             "required": [
2021-07-19T04:15:32.5313070Z [cluster2]                                                                 "exitCode"
2021-07-19T04:15:32.5313736Z [cluster2]                                                             ],
2021-07-19T04:15:32.5314220Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5314705Z [cluster2]                                                         },
2021-07-19T04:15:32.5315161Z [cluster2]                                                         "waiting": {
2021-07-19T04:15:32.5315808Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-19T04:15:32.5316461Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5317004Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.5317727Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-19T04:15:32.5318443Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5318937Z [cluster2]                                                                 },
2021-07-19T04:15:32.5319395Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.5320076Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-19T04:15:32.5321837Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5322441Z [cluster2]                                                                 }
2021-07-19T04:15:32.5322959Z [cluster2]                                                             },
2021-07-19T04:15:32.5324329Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5325055Z [cluster2]                                                         }
2021-07-19T04:15:32.5326019Z [cluster2]                                                     },
2021-07-19T04:15:32.5326684Z [cluster2]                                                     "type": "object"
2021-07-19T04:15:32.5328517Z [cluster2]                                                 },
2021-07-19T04:15:32.5328974Z [cluster2]                                                 "type": "array"
2021-07-19T04:15:32.5329611Z [cluster2]                                             },
2021-07-19T04:15:32.5330227Z [cluster2]                                             "status": {
2021-07-19T04:15:32.5331018Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-19T04:15:32.5331943Z [cluster2]                                                 "properties": {
2021-07-19T04:15:32.5333065Z [cluster2]                                                     "collisionCount": {
2021-07-19T04:15:32.5335132Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-19T04:15:32.5338045Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5338682Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5339153Z [cluster2]                                                     },
2021-07-19T04:15:32.5339779Z [cluster2]                                                     "conditions": {
2021-07-19T04:15:32.5340960Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-19T04:15:32.5341720Z [cluster2]                                                         "items": {
2021-07-19T04:15:32.5342515Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-19T04:15:32.5343559Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5344179Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-19T04:15:32.5345015Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-19T04:15:32.5346006Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5346722Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5347187Z [cluster2]                                                                 },
2021-07-19T04:15:32.5347787Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.5348549Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-19T04:15:32.5349288Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5349763Z [cluster2]                                                                 },
2021-07-19T04:15:32.5350212Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.5351141Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-19T04:15:32.5351828Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5352290Z [cluster2]                                                                 },
2021-07-19T04:15:32.5352854Z [cluster2]                                                                 "status": {
2021-07-19T04:15:32.5353861Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-19T04:15:32.5354802Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5355322Z [cluster2]                                                                 },
2021-07-19T04:15:32.5355797Z [cluster2]                                                                 "type": {
2021-07-19T04:15:32.5356622Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-19T04:15:32.5357245Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5357719Z [cluster2]                                                                 }
2021-07-19T04:15:32.5358151Z [cluster2]                                                             },
2021-07-19T04:15:32.5358810Z [cluster2]                                                             "required": [
2021-07-19T04:15:32.5359291Z [cluster2]                                                                 "status",
2021-07-19T04:15:32.5359742Z [cluster2]                                                                 "type"
2021-07-19T04:15:32.5360181Z [cluster2]                                                             ],
2021-07-19T04:15:32.5360730Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5361189Z [cluster2]                                                         },
2021-07-19T04:15:32.5361626Z [cluster2]                                                         "type": "array"
2021-07-19T04:15:32.5362074Z [cluster2]                                                     },
2021-07-19T04:15:32.5362851Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-19T04:15:32.5364626Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5365827Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5366518Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5367562Z [cluster2]                                                     },
2021-07-19T04:15:32.5368147Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-19T04:15:32.5369448Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5370634Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5371139Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5371613Z [cluster2]                                                     },
2021-07-19T04:15:32.5372125Z [cluster2]                                                     "numberAvailable": {
2021-07-19T04:15:32.5373442Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.5374920Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5378308Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5378972Z [cluster2]                                                     },
2021-07-19T04:15:32.5379528Z [cluster2]                                                     "numberMisscheduled": {
2021-07-19T04:15:32.5381078Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5382199Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5382710Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5383569Z [cluster2]                                                     },
2021-07-19T04:15:32.5384135Z [cluster2]                                                     "numberReady": {
2021-07-19T04:15:32.5385137Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-19T04:15:32.5386119Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5386979Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5387452Z [cluster2]                                                     },
2021-07-19T04:15:32.5387961Z [cluster2]                                                     "numberUnavailable": {
2021-07-19T04:15:32.5389473Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.5390511Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5391024Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5391519Z [cluster2]                                                     },
2021-07-19T04:15:32.5392059Z [cluster2]                                                     "observedGeneration": {
2021-07-19T04:15:32.5392904Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-19T04:15:32.5393951Z [cluster2]                                                         "format": "int64",
2021-07-19T04:15:32.5394493Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5394995Z [cluster2]                                                     },
2021-07-19T04:15:32.5395594Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-19T04:15:32.5396615Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-19T04:15:32.5397335Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5397852Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5398339Z [cluster2]                                                     }
2021-07-19T04:15:32.5398753Z [cluster2]                                                 },
2021-07-19T04:15:32.5399229Z [cluster2]                                                 "required": [
2021-07-19T04:15:32.5399858Z [cluster2]                                                     "currentNumberScheduled",
2021-07-19T04:15:32.5400609Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-19T04:15:32.5401472Z [cluster2]                                                     "numberMisscheduled",
2021-07-19T04:15:32.5402044Z [cluster2]                                                     "numberReady"
2021-07-19T04:15:32.5402778Z [cluster2]                                                 ],
2021-07-19T04:15:32.5403451Z [cluster2]                                                 "type": "object"
2021-07-19T04:15:32.5403953Z [cluster2]                                             }
2021-07-19T04:15:32.5404386Z [cluster2]                                         },
2021-07-19T04:15:32.5404873Z [cluster2]                                         "required": [
2021-07-19T04:15:32.5405569Z [cluster2]                                             "mismatchedContainerImages"
2021-07-19T04:15:32.5406524Z [cluster2]                                         ],
2021-07-19T04:15:32.5406974Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.5407404Z [cluster2]                                     },
2021-07-19T04:15:32.5407916Z [cluster2]                                     "loadBalancerStatus": {
2021-07-19T04:15:32.5408472Z [cluster2]                                         "properties": {
2021-07-19T04:15:32.5408963Z [cluster2]                                             "status": {
2021-07-19T04:15:32.5410057Z [cluster2]                                                 "description": "LoadBalancerStatus represents the status of a load-balancer.",
2021-07-19T04:15:32.5410916Z [cluster2]                                                 "properties": {
2021-07-19T04:15:32.5411418Z [cluster2]                                                     "ingress": {
2021-07-19T04:15:32.5412656Z [cluster2]                                                         "description": "Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points.",
2021-07-19T04:15:32.5414181Z [cluster2]                                                         "items": {
2021-07-19T04:15:32.5415753Z [cluster2]                                                             "description": "LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.",
2021-07-19T04:15:32.5417923Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5418438Z [cluster2]                                                                 "hostname": {
2021-07-19T04:15:32.5419595Z [cluster2]                                                                     "description": "Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers)",
2021-07-19T04:15:32.5420640Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5421129Z [cluster2]                                                                 },
2021-07-19T04:15:32.5421567Z [cluster2]                                                                 "ip": {
2021-07-19T04:15:32.5423682Z [cluster2]                                                                     "description": "IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers)",
2021-07-19T04:15:32.5424721Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5425444Z [cluster2]                                                                 }
2021-07-19T04:15:32.5426425Z [cluster2]                                                             },
2021-07-19T04:15:32.5427036Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5427499Z [cluster2]                                                         },
2021-07-19T04:15:32.5427943Z [cluster2]                                                         "type": "array"
2021-07-19T04:15:32.5428559Z [cluster2]                                                     }
2021-07-19T04:15:32.5428953Z [cluster2]                                                 },
2021-07-19T04:15:32.5429400Z [cluster2]                                                 "type": "object"
2021-07-19T04:15:32.5429969Z [cluster2]                                             }
2021-07-19T04:15:32.5430553Z [cluster2]                                         },
2021-07-19T04:15:32.5431011Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.5431442Z [cluster2]                                     },
2021-07-19T04:15:32.5431896Z [cluster2]                                     "natEnabled": {
2021-07-19T04:15:32.5432379Z [cluster2]                                         "type": "boolean"
2021-07-19T04:15:32.5432839Z [cluster2]                                     },
2021-07-19T04:15:32.5433662Z [cluster2]                                     "networkPlugin": {
2021-07-19T04:15:32.5434648Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.5435137Z [cluster2]                                     },
2021-07-19T04:15:32.5435819Z [cluster2]                                     "routeAgentDaemonSetStatus": {
2021-07-19T04:15:32.5437010Z [cluster2]                                         "properties": {
2021-07-19T04:15:32.5437586Z [cluster2]                                             "lastResourceVersion": {
2021-07-19T04:15:32.5438579Z [cluster2]                                                 "type": "string"
2021-07-19T04:15:32.5439022Z [cluster2]                                             },
2021-07-19T04:15:32.5439627Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-19T04:15:32.5440265Z [cluster2]                                                 "type": "boolean"
2021-07-19T04:15:32.5440849Z [cluster2]                                             },
2021-07-19T04:15:32.5441431Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-19T04:15:32.5442016Z [cluster2]                                                 "items": {
2021-07-19T04:15:32.5443513Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-19T04:15:32.5444889Z [cluster2]                                                     "properties": {
2021-07-19T04:15:32.5445450Z [cluster2]                                                         "running": {
2021-07-19T04:15:32.5446292Z [cluster2]                                                             "description": "Details about a running container",
2021-07-19T04:15:32.5446944Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5448213Z [cluster2]                                                                 "startedAt": {
2021-07-19T04:15:32.5449366Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-19T04:15:32.5450292Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5450870Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5451353Z [cluster2]                                                                 }
2021-07-19T04:15:32.5452261Z [cluster2]                                                             },
2021-07-19T04:15:32.5452746Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5453861Z [cluster2]                                                         },
2021-07-19T04:15:32.5454526Z [cluster2]                                                         "terminated": {
2021-07-19T04:15:32.5455212Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-19T04:15:32.5456006Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5456557Z [cluster2]                                                                 "containerID": {
2021-07-19T04:15:32.5458057Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-19T04:15:32.5458906Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5459915Z [cluster2]                                                                 },
2021-07-19T04:15:32.5460569Z [cluster2]                                                                 "exitCode": {
2021-07-19T04:15:32.5461271Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-19T04:15:32.5461971Z [cluster2]                                                                     "format": "int32",
2021-07-19T04:15:32.5463834Z [cluster2]                                                                     "type": "integer"
2021-07-19T04:15:32.5464398Z [cluster2]                                                                 },
2021-07-19T04:15:32.5465051Z [cluster2]                                                                 "finishedAt": {
2021-07-19T04:15:32.5465757Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-19T04:15:32.5467318Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5468049Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5468547Z [cluster2]                                                                 },
2021-07-19T04:15:32.5469016Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.5470166Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-19T04:15:32.5470941Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5471424Z [cluster2]                                                                 },
2021-07-19T04:15:32.5471904Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.5472609Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-19T04:15:32.5473341Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5473835Z [cluster2]                                                                 },
2021-07-19T04:15:32.5474419Z [cluster2]                                                                 "signal": {
2021-07-19T04:15:32.5475123Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-19T04:15:32.5476010Z [cluster2]                                                                     "format": "int32",
2021-07-19T04:15:32.5476720Z [cluster2]                                                                     "type": "integer"
2021-07-19T04:15:32.5477222Z [cluster2]                                                                 },
2021-07-19T04:15:32.5477707Z [cluster2]                                                                 "startedAt": {
2021-07-19T04:15:32.5478725Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-19T04:15:32.5479759Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5480330Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5480918Z [cluster2]                                                                 }
2021-07-19T04:15:32.5481676Z [cluster2]                                                             },
2021-07-19T04:15:32.5482156Z [cluster2]                                                             "required": [
2021-07-19T04:15:32.5483040Z [cluster2]                                                                 "exitCode"
2021-07-19T04:15:32.5483603Z [cluster2]                                                             ],
2021-07-19T04:15:32.5484067Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5484555Z [cluster2]                                                         },
2021-07-19T04:15:32.5484993Z [cluster2]                                                         "waiting": {
2021-07-19T04:15:32.5485614Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-19T04:15:32.5486267Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5486775Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.5487481Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-19T04:15:32.5488172Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5488945Z [cluster2]                                                                 },
2021-07-19T04:15:32.5489524Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.5490165Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-19T04:15:32.5490835Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5491300Z [cluster2]                                                                 }
2021-07-19T04:15:32.5491740Z [cluster2]                                                             },
2021-07-19T04:15:32.5492202Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5492678Z [cluster2]                                                         }
2021-07-19T04:15:32.5493106Z [cluster2]                                                     },
2021-07-19T04:15:32.5493553Z [cluster2]                                                     "type": "object"
2021-07-19T04:15:32.5494204Z [cluster2]                                                 },
2021-07-19T04:15:32.5494659Z [cluster2]                                                 "type": "array"
2021-07-19T04:15:32.5495122Z [cluster2]                                             },
2021-07-19T04:15:32.5495674Z [cluster2]                                             "status": {
2021-07-19T04:15:32.5496603Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-19T04:15:32.5497356Z [cluster2]                                                 "properties": {
2021-07-19T04:15:32.5498298Z [cluster2]                                                     "collisionCount": {
2021-07-19T04:15:32.5499587Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-19T04:15:32.5500733Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5501254Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5501728Z [cluster2]                                                     },
2021-07-19T04:15:32.5502424Z [cluster2]                                                     "conditions": {
2021-07-19T04:15:32.5503761Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-19T04:15:32.5504525Z [cluster2]                                                         "items": {
2021-07-19T04:15:32.5505309Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-19T04:15:32.5506284Z [cluster2]                                                             "properties": {
2021-07-19T04:15:32.5506845Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-19T04:15:32.5508061Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-19T04:15:32.5509927Z [cluster2]                                                                     "format": "date-time",
2021-07-19T04:15:32.5510594Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5511431Z [cluster2]                                                                 },
2021-07-19T04:15:32.5511919Z [cluster2]                                                                 "message": {
2021-07-19T04:15:32.5513209Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-19T04:15:32.5514535Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5515045Z [cluster2]                                                                 },
2021-07-19T04:15:32.5516207Z [cluster2]                                                                 "reason": {
2021-07-19T04:15:32.5517503Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-19T04:15:32.5518682Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5519504Z [cluster2]                                                                 },
2021-07-19T04:15:32.5520201Z [cluster2]                                                                 "status": {
2021-07-19T04:15:32.5520865Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-19T04:15:32.5521568Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5522048Z [cluster2]                                                                 },
2021-07-19T04:15:32.5522494Z [cluster2]                                                                 "type": {
2021-07-19T04:15:32.5524643Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-19T04:15:32.5525397Z [cluster2]                                                                     "type": "string"
2021-07-19T04:15:32.5525965Z [cluster2]                                                                 }
2021-07-19T04:15:32.5526963Z [cluster2]                                                             },
2021-07-19T04:15:32.5527678Z [cluster2]                                                             "required": [
2021-07-19T04:15:32.5528398Z [cluster2]                                                                 "status",
2021-07-19T04:15:32.5529052Z [cluster2]                                                                 "type"
2021-07-19T04:15:32.5529733Z [cluster2]                                                             ],
2021-07-19T04:15:32.5530212Z [cluster2]                                                             "type": "object"
2021-07-19T04:15:32.5531111Z [cluster2]                                                         },
2021-07-19T04:15:32.5531619Z [cluster2]                                                         "type": "array"
2021-07-19T04:15:32.5532079Z [cluster2]                                                     },
2021-07-19T04:15:32.5532678Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-19T04:15:32.5533960Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5535130Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5535793Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5536276Z [cluster2]                                                     },
2021-07-19T04:15:32.5536883Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-19T04:15:32.5538216Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5539447Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5540081Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5540553Z [cluster2]                                                     },
2021-07-19T04:15:32.5541079Z [cluster2]                                                     "numberAvailable": {
2021-07-19T04:15:32.5542171Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.5543704Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5544421Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5544886Z [cluster2]                                                     },
2021-07-19T04:15:32.5547074Z [cluster2]                                                     "numberMisscheduled": {
2021-07-19T04:15:32.5578381Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-19T04:15:32.5579722Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5580408Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5580849Z [cluster2]                                                     },
2021-07-19T04:15:32.5581305Z [cluster2]                                                     "numberReady": {
2021-07-19T04:15:32.5582097Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-19T04:15:32.5582886Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5583360Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5583812Z [cluster2]                                                     },
2021-07-19T04:15:32.5584305Z [cluster2]                                                     "numberUnavailable": {
2021-07-19T04:15:32.5585443Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-19T04:15:32.5586521Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5587799Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5588281Z [cluster2]                                                     },
2021-07-19T04:15:32.5589005Z [cluster2]                                                     "observedGeneration": {
2021-07-19T04:15:32.5590018Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-19T04:15:32.5591116Z [cluster2]                                                         "format": "int64",
2021-07-19T04:15:32.5591605Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5592069Z [cluster2]                                                     },
2021-07-19T04:15:32.5592633Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-19T04:15:32.5593416Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-19T04:15:32.5594091Z [cluster2]                                                         "format": "int32",
2021-07-19T04:15:32.5594713Z [cluster2]                                                         "type": "integer"
2021-07-19T04:15:32.5595174Z [cluster2]                                                     }
2021-07-19T04:15:32.5595566Z [cluster2]                                                 },
2021-07-19T04:15:32.5596013Z [cluster2]                                                 "required": [
2021-07-19T04:15:32.5596609Z [cluster2]                                                     "currentNumberScheduled",
2021-07-19T04:15:32.5597538Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-19T04:15:32.5598231Z [cluster2]                                                     "numberMisscheduled",
2021-07-19T04:15:32.5598801Z [cluster2]                                                     "numberReady"
2021-07-19T04:15:32.5599273Z [cluster2]                                                 ],
2021-07-19T04:15:32.5599721Z [cluster2]                                                 "type": "object"
2021-07-19T04:15:32.5600347Z [cluster2]                                             }
2021-07-19T04:15:32.5600894Z [cluster2]                                         },
2021-07-19T04:15:32.5601301Z [cluster2]                                         "required": [
2021-07-19T04:15:32.5601902Z [cluster2]                                             "mismatchedContainerImages"
2021-07-19T04:15:32.5602448Z [cluster2]                                         ],
2021-07-19T04:15:32.5602887Z [cluster2]                                         "type": "object"
2021-07-19T04:15:32.5603292Z [cluster2]                                     },
2021-07-19T04:15:32.5603724Z [cluster2]                                     "serviceCIDR": {
2021-07-19T04:15:32.5604184Z [cluster2]                                         "type": "string"
2021-07-19T04:15:32.5604605Z [cluster2]                                     }
2021-07-19T04:15:32.5604986Z [cluster2]                                 },
2021-07-19T04:15:32.5605384Z [cluster2]                                 "required": [
2021-07-19T04:15:32.5605835Z [cluster2]                                     "clusterID",
2021-07-19T04:15:32.5606280Z [cluster2]                                     "natEnabled"
2021-07-19T04:15:32.5606695Z [cluster2]                                 ],
2021-07-19T04:15:34.5541036Z [cluster2]                          cluster1]             "listKind": "SubmarinerList",
2021-07-19T04:15:34.5542413Z [cluster1]             "plural": "submariners",
2021-07-19T04:15:34.5543835Z [cluster1]             "singular": "submariner"
2021-07-19T04:15:34.5544678Z [cluster1]         },
2021-07-19T04:15:34.5545553Z [cluster1]         "conditions": [
2021-07-19T04:15:34.5546057Z [cluster1]             {
2021-07-19T04:15:34.5547525Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:13:50Z",
2021-07-19T04:15:34.5548301Z [cluster1]                 "message": "no conflicts found",
2021-07-19T04:15:34.5548900Z [cluster1]                 "reason": "NoConflicts",
2021-07-19T04:15:34.5549472Z [cluster1]                 "status": "True",
2021-07-19T04:15:34.5550027Z [cluster1]                 "type": "NamesAccepted"
2021-07-19T04:15:34.5550548Z [cluster1]             },
2021-07-19T04:15:34.5550936Z [cluster1]             {
2021-07-19T04:15:34.5551747Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:13:50Z",
2021-07-19T04:15:34.5552487Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-19T04:15:34.5553273Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-19T04:15:34.5553923Z [cluster1]                 "status": "True",
2021-07-19T04:15:34.5554448Z [cluster1]                 "type": "Established"
2021-07-19T04:15:34.5554939Z [cluster1]             }
2021-07-19T04:15:34.5555322Z [cluster1]         ],
2021-07-19T04:15:34.5555810Z [cluster1]         "storedVersions": [
2021-07-19T04:15:34.5556512Z [cluster1]             "v1alpha1"
2021-07-19T04:15:34.5557173Z [cluster1]         ]
2021-07-19T04:15:34.5557810Z [cluster1]     }
2021-07-19T04:15:34.5558196Z [cluster1] }
2021-07-19T04:15:34.5559345Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariners.submariner.io
2021-07-19T04:15:34.5560922Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariners.submariner.io
2021-07-19T04:15:34.5562711Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariners.submariner.io.cluster1.json
2021-07-19T04:15:34.5564579Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-19T04:15:34.5565855Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-19T04:15:34.5567488Z [submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/submariners.submariner.io.cluster1.json
2021-07-19T04:15:34.5569392Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-19T04:15:34.5570602Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-19T04:15:34.5572144Z [submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/submariners.submariner.io.cluster1.json
2021-07-19T04:15:34.5573687Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Submariner
2021-07-19T04:15:34.5574910Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Submariner
2021-07-19T04:15:34.5576842Z [submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/submariners.submariner.io.cluster1.json
2021-07-19T04:15:34.5578276Z [submariner-operator]$ [cluster1] verify_subm_operator
2021-07-19T04:15:34.5579385Z [submariner-operator]$ [cluster1] verify_subm_operator
2021-07-19T04:15:34.5580482Z [submariner-operator]$ [cluster1] kubectl get ns submariner-operator
2021-07-19T04:15:34.5581703Z [submariner-operator]$ [cluster1] kubectl get ns submariner-operator
2021-07-19T04:15:34.5583033Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get ns submariner-operator
2021-07-19T04:15:34.5583897Z [cluster1] NAME                  STATUS   AGE
2021-07-19T04:15:34.5584664Z [cluster1] submariner-operator   Active   102s
2021-07-19T04:15:34.5585770Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-19T04:15:34.5587119Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-19T04:15:34.5588908Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds submariners.submariner.io
2021-07-19T04:15:34.5589834Z [cluster1] NAME                        CREATED AT
2021-07-19T04:15:34.5590694Z [cluster1] submariners.submariner.io   2021-07-19T04:13:50Z
2021-07-19T04:15:34.5591664Z [submariner-operator]$ [cluster1] kubectl api-resources
2021-07-19T04:15:34.5592662Z [submariner-operator]$ [cluster1] kubectl api-resources
2021-07-19T04:15:34.5594373Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 api-resources
2021-07-19T04:15:34.5595571Z [submariner-operator]$ [cluster1] grep submariners
2021-07-19T04:15:34.5596494Z [cluster1] submariners                                    submariner.io/v1alpha1            true         Submariner
2021-07-19T04:15:34.5598278Z [submariner-operator]$ [cluster1] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-19T04:15:34.5617854Z [submariner-operator]$ [cluster1] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-19T04:15:34.5619682Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get sa --namespace=submariner-operator submariner-operator
2021-07-19T04:15:34.5620785Z [cluster1] NAME                  SECRETS   AGE
2021-07-19T04:15:34.5621551Z [cluster1] submariner-operator   1         104s
2021-07-19T04:15:34.5622855Z [submariner-operator]$ [cluster1] kubectl get roles --namespace=submariner-operator submariner-operator
2021-07-19T04:15:34.7736049Z [submariner-operator]$ [cluster1] kubectl get roles --namesp       "type": "object"
2021-07-19T04:15:34.7736829Z [cluster2]                             }
2021-07-19T04:15:34.7737435Z [cluster2]                         },
2021-07-19T04:15:34.7738104Z [cluster2]                         "type": "object"
2021-07-19T04:15:34.7738546Z [cluster2]                     }
2021-07-19T04:15:34.7738949Z [cluster2]                 },
2021-07-19T04:15:34.7739380Z [cluster2]                 "served": true,
2021-07-19T04:15:34.7739895Z [cluster2]                 "storage": true,
2021-07-19T04:15:34.7740400Z [cluster2]                 "subresources": {
2021-07-19T04:15:34.7740894Z [cluster2]                     "status": {}
2021-07-19T04:15:34.7741302Z [cluster2]                 }
2021-07-19T04:15:34.7741887Z [cluster2]             }
2021-07-19T04:15:34.7742282Z [cluster2]         ]
2021-07-19T04:15:34.7742691Z [cluster2]     },
2021-07-19T04:15:34.7743109Z [cluster2]     "status": {
2021-07-19T04:15:34.7743635Z [cluster2]         "acceptedNames": {
2021-07-19T04:15:34.7744228Z [cluster2]             "kind": "Submariner",
2021-07-19T04:15:34.7744988Z [cluster2]             "listKind": "SubmarinerList",
2021-07-19T04:15:34.7745597Z [cluster2]             "plural": "submariners",
2021-07-19T04:15:34.7746148Z [cluster2]             "singular": "submariner"
2021-07-19T04:15:34.7746618Z [cluster2]         },
2021-07-19T04:15:34.7747037Z [cluster2]         "conditions": [
2021-07-19T04:15:34.7747473Z [cluster2]             {
2021-07-19T04:15:34.7748274Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:02Z",
2021-07-19T04:15:34.7748943Z [cluster2]                 "message": "no conflicts found",
2021-07-19T04:15:34.7749698Z [cluster2]                 "reason": "NoConflicts",
2021-07-19T04:15:34.7750752Z [cluster2]                 "status": "True",
2021-07-19T04:15:34.7751321Z [cluster2]                 "type": "NamesAccepted"
2021-07-19T04:15:34.7751817Z [cluster2]             },
2021-07-19T04:15:34.7752229Z [cluster2]             {
2021-07-19T04:15:34.7753072Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:02Z",
2021-07-19T04:15:34.7753834Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-19T04:15:34.7754603Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-19T04:15:34.7755248Z [cluster2]                 "status": "True",
2021-07-19T04:15:34.7755772Z [cluster2]                 "type": "Established"
2021-07-19T04:15:34.7756260Z [cluster2]             }
2021-07-19T04:15:34.7756862Z [cluster2]         ],
2021-07-19T04:15:34.7757387Z [cluster2]         "storedVersions": [
2021-07-19T04:15:34.7757915Z [cluster2]             "v1alpha1"
2021-07-19T04:15:34.7758341Z [cluster2]         ]
2021-07-19T04:15:34.7758736Z [cluster2]     }
2021-07-19T04:15:34.7759108Z [cluster2] }
2021-07-19T04:15:34.7760265Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariners.submariner.io
2021-07-19T04:15:34.7761948Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariners.submariner.io
2021-07-19T04:15:34.7763684Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/submariners.submariner.io.cluster2.json
2021-07-19T04:15:34.7765227Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-19T04:15:34.7766395Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-19T04:15:34.7767936Z [submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/submariners.submariner.io.cluster2.json
2021-07-19T04:15:34.7769448Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-19T04:15:34.7770665Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-19T04:15:34.7772194Z [submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/submariners.submariner.io.cluster2.json
2021-07-19T04:15:34.7773736Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Submariner
2021-07-19T04:15:34.7775190Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Submariner
2021-07-19T04:15:34.7776988Z [submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/submariners.submariner.io.cluster2.json
2021-07-19T04:15:34.7778427Z [submariner-operator]$ [cluster2] verify_subm_operator
2021-07-19T04:15:34.7779393Z [submariner-operator]$ [cluster2] verify_subm_operator
2021-07-19T04:15:34.7780497Z [submariner-operator]$ [cluster2] kubectl get ns submariner-operator
2021-07-19T04:15:34.7782671Z [submariner-operator]$ [cluster2] kubectl get ns submariner-operator
2021-07-19T04:15:34.7784328Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get ns submariner-operator
2021-07-19T04:15:34.7785345Z [cluster2] NAME                  STATUS   AGE
2021-07-19T04:15:34.7786080Z [cluster2] submariner-operator   Active   91s
2021-07-19T04:15:34.7787168Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-19T04:15:34.7788466Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-19T04:15:34.7789891Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds submariners.submariner.io
2021-07-19T04:15:34.7790974Z [cluster2] NAME                        CREATED AT
2021-07-19T04:15:34.7791819Z [cluster2] submariners.submariner.io   2021-07-19T04:14:02Z
2021-07-19T04:15:34.7792839Z [submariner-operator]$ [cluster2] kubectl api-resources
2021-07-19T04:15:34.7793822Z [submariner-operator]$ [cluster2] grep submariners
2021-07-19T04:15:34.7795010Z [submariner-operator]$ [cluster2] kubectl api-resources
2021-07-19T04:15:34.7796203Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 api-resources
2021-07-19T04:15:34.7797328Z [cluster2] submariners                                    submariner.io/v1alpha1            true         Submariner
2021-07-19T04:15:36.5868324Z [submariner-operator]$ [clusace=submariner-operator submariner-operator
2021-07-19T04:15:36.5869874Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get roles --namespace=submariner-operator submariner-operator
2021-07-19T04:15:36.5870953Z [cluster1] NAME                  CREATED AT
2021-07-19T04:15:36.5871881Z [cluster1] submariner-operator   2021-07-19T04:13:50Z
2021-07-19T04:15:36.5873679Z [submariner-operator]$ [cluster1] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-19T04:15:36.5875687Z [submariner-operator]$ [cluster1] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-19T04:15:36.5877353Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get rolebindings --namespace=submariner-operator submariner-operator
2021-07-19T04:15:36.5878480Z [cluster1] NAME                  ROLE                       AGE
2021-07-19T04:15:36.5879310Z [cluster1] submariner-operator   Role/submariner-operator   104s
2021-07-19T04:15:36.5880613Z [submariner-operator]$ [cluster1] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-19T04:15:36.5882909Z [submariner-operator]$ [cluster1] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-19T04:15:36.5884685Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get deployments --namespace=submariner-operator submariner-operator
2021-07-19T04:15:36.5886295Z [cluster1] NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
2021-07-19T04:15:36.5887184Z [cluster1] submariner-operator   1/1     1            1           99s
2021-07-19T04:15:36.5888038Z [submariner-operator]$ [cluster1] verify_subm_op_pod
2021-07-19T04:15:36.5889067Z [submariner-operator]$ [cluster1] verify_subm_op_pod
2021-07-19T04:15:36.5890767Z [submariner-operator]$ [cluster1] subm_operator_pod_name=$(kubectl get pods --namespace=$subm_ns -l name=$operator_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-19T04:15:36.5893665Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:36.5896495Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:36.5898676Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:36.5900770Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator -o json
2021-07-19T04:15:36.5903824Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator -o json
2021-07-19T04:15:36.5906582Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator -o json
2021-07-19T04:15:36.5907699Z [cluster1] {
2021-07-19T04:15:36.5908093Z [cluster1]     "apiVersion": "v1",
2021-07-19T04:15:36.5908529Z [cluster1]     "kind": "Pod",
2021-07-19T04:15:36.5908926Z [cluster1]     "metadata": {
2021-07-19T04:15:36.5909669Z [cluster1]         "creationTimestamp": "2021-07-19T04:13:56Z",
2021-07-19T04:15:36.5910819Z [cluster1]         "generateName": "submariner-operator-5bfdb87494-",
2021-07-19T04:15:36.5911548Z [cluster1]         "labels": {
2021-07-19T04:15:36.5912249Z [cluster1]             "name": "submariner-operator",
2021-07-19T04:15:36.5913036Z [cluster1]             "pod-template-hash": "5bfdb87494"
2021-07-19T04:15:36.5913542Z [cluster1]         },
2021-07-19T04:15:36.5913963Z [cluster1]         "managedFields": [
2021-07-19T04:15:36.5914404Z [cluster1]             {
2021-07-19T04:15:36.5914827Z [cluster1]                 "apiVersion": "v1",
2021-07-19T04:15:36.5915350Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:36.5915838Z [cluster1]                 "fieldsV1": {
2021-07-19T04:15:36.5916296Z [cluster1]                     "f:metadata": {
2021-07-19T04:15:36.5916783Z [cluster1]                         "f:generateName": {},
2021-07-19T04:15:36.5917280Z [cluster1]                         "f:labels": {
2021-07-19T04:15:36.5917908Z [cluster1]                             ".": {},
2021-07-19T04:15:36.5918345Z [cluster1]                             "f:name": {},
2021-07-19T04:15:36.5919087Z [cluster1]                             "f:pod-template-hash": {}
2021-07-19T04:15:36.5919575Z [cluster1]                         },
2021-07-19T04:15:36.5920063Z [cluster1]                         "f:ownerReferences": {
2021-07-19T04:15:36.5920541Z [cluster1]                             ".": {},
2021-07-19T04:15:36.5921325Z [cluster1]                             "k:{\"uid\":\"b0ea7727-0001-4b27-b5e3-13476ffbb6f6\"}": {
2021-07-19T04:15:36.5922512Z [cluster1]                                 ".": {},
2021-07-19T04:15:36.5923032Z [cluster1]                                 "f:apiVersion": {},
2021-07-19T04:15:36.5923684Z [cluster1]                                 "f:blockOwnerDeletion": {},
2021-07-19T04:15:36.5924530Z [cluster1]                                 "f:controller": {},
2021-07-19T04:15:36.5925040Z [cluster1]                                 "f:kind": {},
2021-07-19T04:15:36.5925638Z [cluster1]                                 "f:name": {},
2021-07-19T04:15:36.5960220Z [cluster1]                                 "f:uid": {}
2021-07-19T04:15:36.5960659Z [cluster1]                             }
2021-07-19T04:15:36.5961033Z [cluster1]                         }
2021-07-19T04:15:36.5961418Z [cluster1]                     },
2021-07-19T04:15:36.5961811Z [cluster1]                     "f:spec": {
2021-07-19T04:15:36.5962278Z [cluster1]                         "f:containers": {
2021-07-19T04:15:36.5964159Z [cluster1]                             "k:{\"name\":\"submariner-operator\"}": {
2021-07-19T04:15:36.5964950Z [cluster1]                                 ".": {},
2021-07-19T04:15:36.5965980Z [cluster1]                                 "f:command": {},
2021-07-19T04:15:36.5966435Z [cluster1]                                 "f:env": {
2021-07-19T04:15:36.5966879Z [cluster1]                                     ".": {},
2021-07-19T04:15:36.5967388Z [cluster1]                                     "k:{\"name\":\"OPERATOR_NAME\"}": {
2021-07-19T04:15:36.5967909Z [cluster1]                                         ".": {},
2021-07-19T04:15:36.5968351Z [cluster1]                                         "f:name": {},
2021-07-19T04:15:36.5968984Z [cluster1]                                         "f:value": {}
2021-07-19T04:15:36.5969405Z [cluster1]                                     },
2021-07-19T04:15:36.5969877Z [cluster1]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-19T04:15:36.5970357Z [cluster1]                                         ".": {},
2021-07-19T04:15:36.5970783Z [cluster1]                                         "f:name": {},
2021-07-19T04:15:36.5971450Z [cluster1]                                         "f:valueFrom": {
2021-07-19T04:15:36.5971916Z [cluster1]                                             ".": {},
2021-07-19T04:15:36.5972402Z [cluster1]                                             "f:fieldRef": {
2021-07-19T04:15:36.5972874Z [cluster1]                                                 ".": {},
2021-07-19T04:15:36.5973378Z [cluster1]                                                 "f:apiVersion": {},
2021-07-19T04:15:36.5973929Z [cluster1]                                                 "f:fieldPath": {}
2021-07-19T04:15:36.5974405Z [cluster1]                                             }
2021-07-19T04:15:36.5974834Z [cluster1]                                         }
2021-07-19T04:15:36.5975243Z [cluster1]                                     },
2021-07-19T04:15:36.5975989Z [cluster1]                                     "k:{\"name\":\"WATCH_NAMESPACE\"}": {
2021-07-19T04:15:36.5976499Z [cluster1]                                         ".": {},
2021-07-19T04:15:36.5976959Z [cluster1]                                         "f:name": {},
2021-07-19T04:15:36.5977444Z [cluster1]                                         "f:valueFrom": {
2021-07-19T04:15:36.5977931Z [cluster1]                                             ".": {},
2021-07-19T04:15:36.5978538Z [cluster1]                                             "f:fieldRef": {
2021-07-19T04:15:36.5979027Z [cluster1]                                                 ".": {},
2021-07-19T04:15:36.5979537Z [cluster1]                                                 "f:apiVersion": {},
2021-07-19T04:15:36.5980071Z [cluster1]                                                 "f:fieldPath": {}
2021-07-19T04:15:36.5980559Z [cluster1]                                             }
2021-07-19T04:15:36.5980972Z [cluster1]                                         }
2021-07-19T04:15:36.5981566Z [cluster1]                                     }
2021-07-19T04:15:36.5982210Z [cluster1]                                 },
2021-07-19T04:15:36.5982675Z [cluster1]                                 "f:image": {},
2021-07-19T04:15:36.5983242Z [cluster1]                                 "f:imagePullPolicy": {},
2021-07-19T04:15:36.5983779Z [cluster1]                                 "f:name": {},
2021-07-19T04:15:36.5984290Z [cluster1]                                 "f:resources": {},
2021-07-19T04:15:36.5985251Z [cluster1]                                 "f:terminationMessagePath": {},
2021-07-19T04:15:36.5986017Z [cluster1]                                 "f:terminationMessagePolicy": {}
2021-07-19T04:15:36.5986585Z [cluster1]                             }
2021-07-19T04:15:36.5986980Z [cluster1]                         },
2021-07-19T04:15:36.5987406Z [cluster1]                         "f:dnsPolicy": {},
2021-07-19T04:15:36.5987976Z [cluster1]                         "f:enableServiceLinks": {},
2021-07-19T04:15:36.5988667Z [cluster1]                         "f:restartPolicy": {},
2021-07-19T04:15:36.5989199Z [cluster1]                         "f:schedulerName": {},
2021-07-19T04:15:36.5989763Z [cluster1]                         "f:securityContext": {},
2021-07-19T04:15:36.5990314Z [cluster1]                         "f:serviceAccount": {},
2021-07-19T04:15:36.5990913Z [cluster1]                         "f:serviceAccountName": {},
2021-07-19T04:15:36.5992050Z [cluster1]                         "f:terminationGracePeriodSeconds": {}
2021-07-19T04:15:36.5992921Z [cluster1]                     }
2021-07-19T04:15:36.5993297Z [cluster1]                 },
2021-07-19T04:15:36.5994163Z [cluster1]                 "manager": "kube-controller-manager",
2021-07-19T04:15:36.5994997Z [cluster1]                 "operation": "Update",
2021-07-19T04:15:36.5995641Z [cluster1]                 "time": "2021-07-19T04:13:56Z"
2021-07-19T04:15:36.5996059Z [cluster1]             },
2021-07-19T04:15:36.5996416Z [cluster1]             {
2021-07-19T04:15:36.5996857Z [cluster1]                 "apiVersion": "v1",
2021-07-19T04:15:36.5997369Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:36.5997880Z [cluster1]                 "fieldsV1": {
2021-07-19T04:15:36.5998310Z [cluster1]                     "f:status": {
2021-07-19T04:15:36.5998777Z [cluster1]                         "f:conditions": {
2021-07-19T04:15:36.5999345Z [cluster1]                             "k:{\"type\":\"ContainersReady\"}": {
2021-07-19T04:15:36.6042672Z [cluster1]                                 ".": {},
2021-07-19T04:15:36.6043195Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-19T04:15:36.6043821Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-19T04:15:36.6044516Z [cluster1]                                 "f:status": {},
2021-07-19T04:15:36.6045308Z [cluster1]                                 "f:type": {}
2021-07-19T04:15:36.6045704Z [cluster1]                             },
2021-07-19T04:15:36.6046168Z [cluster1]                             "k:{\"type\":\"Initialized\"}": {
2021-07-19T04:15:36.6046629Z [cluster1]                                 ".": {},
2021-07-19T04:15:36.6047163Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-19T04:15:36.6047736Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-19T04:15:36.6048236Z [cluster1]                                 "f:status": {},
2021-07-19T04:15:36.6048784Z [cluster1]                                 "f:type": {}
2021-07-19T04:15:36.6049183Z [cluster1]                             },
2021-07-19T04:15:36.6049606Z [cluster1]                             "k:{\"type\":\"Ready\"}": {
2021-07-19T04:15:36.6050021Z [cluster1]                                 ".": {},
2021-07-19T04:15:36.6050479Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-19T04:15:36.6051049Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-19T04:15:36.6051760Z [cluster1]                                 "f:status": {},
2021-07-19T04:15:36.6052216Z [cluster1]                                 "f:type": {}
2021-07-19T04:15:36.6052623Z [cluster1]                             }
2021-07-19T04:15:36.6053013Z [cluster1]                         },
2021-07-19T04:15:36.6053502Z [cluster1]                         "f:containerStatuses": {},
2021-07-19T04:15:36.6054037Z [cluster1]                         "f:hostIP": {},
2021-07-19T04:15:36.6054817Z [cluster1]                         "f:phase": {},
2021-07-19T04:15:36.6055694Z [cluster1]                         "f:podIP": {},
2021-07-19T04:15:36.6056132Z [cluster1]                         "f:podIPs": {
2021-07-19T04:15:36.6056524Z [cluster1]                             ".": {},
2021-07-19T04:15:36.6056937Z [cluster1]                             "k:{\"ip\":\"10.0.0.3\"}": {
2021-07-19T04:15:36.6057329Z [cluster1]                                 ".": {},
2021-07-19T04:15:36.6057843Z [cluster1]                                 "f:ip": {}
2021-07-19T04:15:36.6058394Z [cluster1]                             }
2021-07-19T04:15:36.6058778Z [cluster1]                         },
2021-07-19T04:15:36.6059190Z [cluster1]                         "f:startTime": {}
2021-07-19T04:15:36.6059607Z [cluster1]                     }
2021-07-19T04:15:36.6059954Z [cluster1]                 },
2021-07-19T04:15:36.6060541Z [cluster1]                 "manager": "kubelet",
2021-07-19T04:15:36.6061503Z [cluster1]                 "operation": "Update",
2021-07-19T04:15:36.6062775Z [cluster1]                 "time": "2021-07-19T04:14:05Z"
2021-07-19T04:15:36.6063242Z [cluster1]             }
2021-07-19T04:15:36.6063630Z [cluster1]         ],
2021-07-19T04:15:36.6064774Z [cluster1]         "name": "submariner-operator-5bfdb87494-rwjcx",
2021-07-19T04:15:36.6066246Z [cluster1]         "namespace": "submariner-operator",
2021-07-19T04:15:36.6066846Z [cluster1]         "ownerReferences": [
2021-07-19T04:15:36.6067269Z [cluster1]             {
2021-07-19T04:15:36.6067692Z [cluster1]                 "apiVersion": "apps/v1",
2021-07-19T04:15:36.6068237Z [cluster1]                 "blockOwnerDeletion": true,
2021-07-19T04:15:36.6068783Z [cluster1]                 "controller": true,
2021-07-19T04:15:36.6069265Z [cluster1]                 "kind": "ReplicaSet",
2021-07-19T04:15:36.6070097Z [cluster1]                 "name": "submariner-operator-5bfdb87494",
2021-07-19T04:15:36.6071356Z [cluster1]                 "uid": "b0ea7727-0001-4b27-b5e3-13476ffbb6f6"
2021-07-19T04:15:36.6072286Z [cluster1]             }
2021-07-19T04:15:36.6072673Z [cluster1]         ],
2021-07-19T04:15:36.6073152Z [cluster1]         "resourceVersion": "1900",
2021-07-19T04:15:36.6074719Z [cluster1]         "selfLink": "/api/v1/namespaces/submariner-operator/pods/submariner-operator-5bfdb87494-rwjcx",
2021-07-19T04:15:36.6076302Z [cluster1]         "uid": "b7652642-446d-42f9-ba4c-a5efeaf5aa1b"
2021-07-19T04:15:36.6076832Z [cluster1]     },
2021-07-19T04:15:36.6077379Z [cluster1]     "spec": {
2021-07-19T04:15:36.6077779Z [cluster1]         "containers": [
2021-07-19T04:15:36.6078183Z [cluster1]             {
2021-07-19T04:15:36.6078553Z [cluster1]                 "command": [
2021-07-19T04:15:36.6079238Z [cluster1]                     "submariner-operator",
2021-07-19T04:15:36.6079936Z [cluster1]                     "-alsologtostderr",
2021-07-19T04:15:36.6080537Z [cluster1]                     "-v=1"
2021-07-19T04:15:36.6080900Z [cluster1]                 ],
2021-07-19T04:15:36.6081698Z [cluster1]                 "env": [
2021-07-19T04:15:36.6082308Z [cluster1]                     {
2021-07-19T04:15:36.6082816Z [cluster1]                         "name": "WATCH_NAMESPACE",
2021-07-19T04:15:36.6083363Z [cluster1]                         "valueFrom": {
2021-07-19T04:15:36.6083855Z [cluster1]                             "fieldRef": {
2021-07-19T04:15:36.6084389Z [cluster1]                                 "apiVersion": "v1",
2021-07-19T04:15:36.6085335Z [cluster1]                                 "fieldPath": "metadata.namespace"
2021-07-19T04:15:36.6086052Z [cluster1]                             }
2021-07-19T04:15:36.6086414Z [cluster1]                         }
2021-07-19T04:15:36.6086787Z [cluster1]                     },
2021-07-19T04:15:36.6087141Z [cluster1]                     {
2021-07-19T04:15:36.6087619Z [cluster1]                         "name": "POD_NAME",
2021-07-19T04:15:36.6088068Z [cluster1]                         "valueFrom": {
2021-07-19T04:15:36.6088537Z [cluster1]                             "fieldRef": {
2021-07-19T04:15:36.6089023Z [cluster1]                                 "apiVersion": "v1",
2021-07-19T04:15:36.6089558Z [cluster1]                                 "fieldPath": "metadata.name"
2021-07-19T04:15:36.6090050Z [cluster1]                             }
2021-07-19T04:15:36.6090414Z [cluster1]                         }
2021-07-19T04:15:36.6090785Z [cluster1]                     },
2021-07-19T04:15:36.6091137Z [cluster1]                     {
2021-07-19T04:15:36.6092100Z [cluster1]                         "name": "OPERATOR_NAME",
2021-07-19T04:15:36.6092975Z [cluster1]                         "value": "submariner-operator"
2021-07-19T04:15:36.6093550Z [cluster1]                     }
2021-07-19T04:15:36.6093933Z [cluster1]                 ],
2021-07-19T04:15:36.6094759Z [cluster1]                 "image": "localhost:5000/submariner-operator:local",
2021-07-19T04:15:36.6096055Z [cluster1]                 "imagePullPolicy": "IfNotPresent",
2021-07-19T04:15:36.6096894Z [cluster1]                 "name": "submariner-operator",
2021-07-19T04:15:36.6097444Z [cluster1]                 "resources": {},
2021-07-19T04:15:36.6098262Z [cluster1]                 "terminationMessagePath": "/dev/termination-log",
2021-07-19T04:15:36.6099086Z [cluster1]                 "terminationMessagePolicy": "File",
2021-07-19T04:15:36.6099695Z [cluster1]                 "volumeMounts": [
2021-07-19T04:15:36.6100125Z [cluster1]                     {
2021-07-19T04:15:36.6100726Z [cluster1]                         "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
2021-07-19T04:15:36.6102017Z [cluster1]                         "name": "submariner-operator-token-r96pp",
2021-07-19T04:15:36.6102777Z [cluster1]                         "readOnly": true
2021-07-19T04:15:36.6103398Z [cluster1]                     }
2021-07-19T04:15:36.6103800Z [cluster1]                 ]
2021-07-19T04:15:36.6104181Z [cluster1]             }
2021-07-19T04:15:36.6104574Z [cluster1]         ],
2021-07-19T04:15:36.6106347Z [cluster1]         "dnsPolicy": "ClusterFirst",
2021-07-19T04:15:36.6107064Z [cluster1]         "enableServiceLinks": true,
2021-07-19T04:15:36.6108056Z [cluster1]         "nodeName": "cluster1-worker",
2021-07-19T04:15:36.6109020Z [cluster1]         "preemptionPolicy": "PreemptLowerPriority",
2021-07-19T04:15:36.6109637Z [cluster1]         "priority": 0,
2021-07-19T04:15:36.6110095Z [cluster1]         "restartPolicy": "Always",
2021-07-19T04:15:36.6110863Z [cluster1]         "schedulerName": "default-scheduler",
2021-07-19T04:15:36.6112018Z [cluster1]         "securityContext": {},
2021-07-19T04:15:36.6112896Z [cluster1]         "serviceAccount": "submariner-operator",
2021-07-19T04:15:36.6113922Z [cluster1]         "serviceAccountName": "submariner-operator",
2021-07-19T04:15:36.6115177Z [cluster1]         "terminationGracePeriodSeconds": 30,
2021-07-19T04:15:36.6116178Z [cluster1]         "tolerations": [
2021-07-19T04:15:36.6116978Z [cluster1]             {
2021-07-19T04:15:36.6117564Z [cluster1]                 "effect": "NoExecute",
2021-07-19T04:15:36.6118411Z [cluster1]                 "key": "node.kubernetes.io/not-ready",
2021-07-19T04:15:36.6119202Z [cluster1]                 "operator": "Exists",
2021-07-19T04:15:36.6120141Z [cluster1]                 "tolerationSeconds": 300
2021-07-19T04:15:36.6120638Z [cluster1]             },
2021-07-19T04:15:36.6121000Z [cluster1]             {
2021-07-19T04:15:36.6121864Z [cluster1]                 "effect": "NoExecute",
2021-07-19T04:15:36.6122582Z [cluster1]                 "key": "node.kubernetes.io/unreachable",
2021-07-19T04:15:36.6123448Z [cluster1]                 "operator": "Exists",
2021-07-19T04:15:36.6124017Z [cluster1]                 "tolerationSeconds": 300
2021-07-19T04:15:36.6124682Z [cluster1]             }
2021-07-19T04:15:36.6125357Z [cluster1]         ],
2021-07-19T04:15:36.6125744Z [cluster1]         "volumes": [
2021-07-19T04:15:36.6126130Z [cluster1]             {
2021-07-19T04:15:36.6126952Z [cluster1]                 "name": "submariner-operator-token-r96pp",
2021-07-19T04:15:36.6127625Z [cluster1]                 "secret": {
2021-07-19T04:15:36.6128063Z [cluster1]                     "defaultMode": 420,
2021-07-19T04:15:36.6128974Z [cluster1]                     "secretName": "submariner-operator-token-r96pp"
2021-07-19T04:15:36.6129632Z [cluster1]                 }
2021-07-19T04:15:36.6141786Z [cluster1]             }
2021-07-19T04:15:36.6142722Z [cluster1]         ]
2021-07-19T04:15:36.6143116Z [cluster1]     },
2021-07-19T04:15:36.6143884Z [cluster1]     "status": {
2021-07-19T04:15:36.6144334Z [cluster1]         "conditions": [
2021-07-19T04:15:36.6144902Z [cluster1]             {
2021-07-19T04:15:36.6145686Z [cluster1]                 "lastProbeTime": null,
2021-07-19T04:15:36.6146576Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:13:56Z",
2021-07-19T04:15:36.6154670Z [cluster1]                 "status": "True",
2021-07-19T04:15:36.6155781Z [cluster1]                 "type": "Initialized"
2021-07-19T04:15:36.6161994Z [cluster1]             },
2021-07-19T04:15:36.6162376Z [cluster1]             {
2021-07-19T04:15:36.6162816Z [cluster1]                 "lastProbeTime": null,
2021-07-19T04:15:36.6163945Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:14:05Z",
2021-07-19T04:15:36.6164498Z [cluster1]                 "status": "True",
2021-07-19T04:15:36.6165112Z [cluster1]                 "type": "Ready"
2021-07-19T04:15:36.6165494Z [cluster1]             },
2021-07-19T04:15:36.6165863Z [cluster1]             {
2021-07-19T04:15:36.6166311Z [cluster1]                 "lastProbeTime": null,
2021-07-19T04:15:36.6167078Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:14:05Z",
2021-07-19T04:15:36.6167629Z [cluster1]                 "status": "True",
2021-07-19T04:15:36.6168301Z [cluster1]                 "type": "ContainersReady"
2021-07-19T04:15:36.6169697Z [cluster1]             },
2021-07-19T04:15:36.6170230Z [cluster1]             {
2021-07-19T04:15:36.6170687Z [cluster1]                 "lastProbeTime": null,
2021-07-19T04:15:36.6172170Z [cluster1]                 "lastTransitionTime": "2021-07-19T04:13:56Z",
2021-07-19T04:15:36.6172982Z [cluster1]                 "status": "True",
2021-07-19T04:15:36.6173546Z [cluster1]                 "type": "PodScheduled"
2021-07-19T04:15:36.6174033Z [cluster1]             }
2021-07-19T04:15:36.6174440Z [cluster1]         ],
2021-07-19T04:15:36.6174942Z [cluster1]         "containerStatuses": [
2021-07-19T04:15:36.6176284Z [cluster1]             {
2021-07-19T04:15:36.6177663Z [cluster1]                 "containerID": "containerd://7947f42525043563daac17b2138aaa5720da139a6d2009dc22384c31781e72ba",
2021-07-19T04:15:36.6179476Z [cluster1]                 "image": "localhost:5000/submariner-operator:local",
2021-07-19T04:15:37.2222883Z [cluster1]                 "imageID": "localhost:5000/submariner-opeter2] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2224561Z [submariner-operator]$ [cluster2] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2226089Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get sa --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2226999Z [cluster2] NAME                  SECRETS   AGE
2021-07-19T04:15:37.2227657Z [cluster2] submariner-operator   1         93s
2021-07-19T04:15:37.2228716Z [submariner-operator]$ [cluster2] kubectl get roles --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2230315Z [submariner-operator]$ [cluster2] kubectl get roles --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2231796Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get roles --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2232702Z [cluster2] NAME                  CREATED AT
2021-07-19T04:15:37.2233367Z [cluster2] submariner-operator   2021-07-19T04:14:02Z
2021-07-19T04:15:37.2234604Z [submariner-operator]$ [cluster2] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2236171Z [submariner-operator]$ [cluster2] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2237719Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get rolebindings --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2239099Z [cluster2] NAME                  ROLE                       AGE
2021-07-19T04:15:37.2240152Z [cluster2] submariner-operator   Role/submariner-operator   92s
2021-07-19T04:15:37.2241551Z [submariner-operator]$ [cluster2] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2243934Z [submariner-operator]$ [cluster2] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2245527Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get deployments --namespace=submariner-operator submariner-operator
2021-07-19T04:15:37.2246864Z [cluster2] NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
2021-07-19T04:15:37.2247600Z [cluster2] submariner-operator   1/1     1            1           88s
2021-07-19T04:15:37.2248383Z [submariner-operator]$ [cluster2] verify_subm_op_pod
2021-07-19T04:15:37.2249591Z [submariner-operator]$ [cluster2] verify_subm_op_pod
2021-07-19T04:15:37.2251038Z [submariner-operator]$ [cluster2] subm_operator_pod_name=$(kubectl get pods --namespace=$subm_ns -l name=$operator_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-19T04:15:37.2253587Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:37.2255843Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:37.2257737Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:37.2260039Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator -o json
2021-07-19T04:15:37.2262195Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator -o json
2021-07-19T04:15:37.2265094Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pod submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator -o json
2021-07-19T04:15:37.2269083Z [cluster2] {
2021-07-19T04:15:37.2269561Z [cluster2]     "apiVersion": "v1",
2021-07-19T04:15:37.2270238Z [cluster2]     "kind": "Pod",
2021-07-19T04:15:37.2270772Z [cluster2]     "metadata": {
2021-07-19T04:15:37.2271981Z [cluster2]         "creationTimestamp": "2021-07-19T04:14:08Z",
2021-07-19T04:15:37.2273571Z [cluster2]         "generateName": "submariner-operator-5bfdb87494-",
2021-07-19T04:15:37.2274615Z [cluster2]         "labels": {
2021-07-19T04:15:37.2275271Z [cluster2]             "name": "submariner-operator",
2021-07-19T04:15:37.2276023Z [cluster2]             "pod-template-hash": "5bfdb87494"
2021-07-19T04:15:37.2276501Z [cluster2]         },
2021-07-19T04:15:37.2276899Z [cluster2]         "managedFields": [
2021-07-19T04:15:37.2277325Z [cluster2]             {
2021-07-19T04:15:37.2277715Z [cluster2]                 "apiVersion": "v1",
2021-07-19T04:15:37.2278212Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:37.2278669Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:37.2279291Z [cluster2]                     "f:metadata": {
2021-07-19T04:15:37.2280161Z [cluster2]                         "f:generateName": {},
2021-07-19T04:15:37.2280852Z [cluster2]                         "f:labels": {
2021-07-19T04:15:37.2281318Z [cluster2]                             ".": {},
2021-07-19T04:15:37.2281753Z [cluster2]                             "f:name": {},
2021-07-19T04:15:37.2283204Z [cluster2]                             "f:pod-template-hash": {}
2021-07-19T04:15:37.2283691Z [cluster2]                         },
2021-07-19T04:15:37.2284182Z [cluster2]                         "f:ownerReferences": {
2021-07-19T04:15:37.2284662Z [cluster2]                             ".": {},
2021-07-19T04:15:37.2285767Z [cluster2]                             "k:{\"uid\":\"68f544c5-8726-4aa1-adbc-c11fb2fd6a8a\"}": {
2021-07-19T04:15:37.2286325Z [cluster2]                                 ".": {},
2021-07-19T04:15:37.2287292Z [cluster2]                                 "f:apiVersion": {},
2021-07-19T04:15:37.2287843Z [cluster2]                                 "f:blockOwnerDeletion": {},
2021-07-19T04:15:37.2288371Z [cluster2]                                 "f:controller": {},
2021-07-19T04:15:37.2288825Z [cluster2]                                 "f:kind": {},
2021-07-19T04:15:37.2289628Z [cluster2]                                 "f:name": {},
2021-07-19T04:15:37.2290099Z [cluster2]                                 "f:uid": {}
2021-07-19T04:15:37.2290671Z [cluster2]                             }
2021-07-19T04:15:37.2291243Z [cluster2]                         }
2021-07-19T04:15:37.2291640Z [cluster2]                     },
2021-07-19T04:15:37.2292079Z [cluster2]                     "f:spec": {
2021-07-19T04:15:37.2292895Z [cluster2]                         "f:containers": {
2021-07-19T04:15:37.2293833Z [cluster2]                             "k:{\"name\":\"submariner-operator\"}": {
2021-07-19T04:15:37.2294581Z [cluster2]                                 ".": {},
2021-07-19T04:15:37.2295027Z [cluster2]                                 "f:command": {},
2021-07-19T04:15:37.2335639Z [cluster2]                                 "f:env": {
2021-07-19T04:15:37.2336073Z [cluster2]                                     ".": {},
2021-07-19T04:15:37.2336553Z [cluster2]                                     "k:{\"name\":\"OPERATOR_NAME\"}": {
2021-07-19T04:15:37.2337004Z [cluster2]                                         ".": {},
2021-07-19T04:15:37.2337425Z [cluster2]                                         "f:name": {},
2021-07-19T04:15:37.2337862Z [cluster2]                                         "f:value": {}
2021-07-19T04:15:37.2338406Z [cluster2]                                     },
2021-07-19T04:15:37.2339014Z [cluster2]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-19T04:15:37.2339693Z [cluster2]                                         ".": {},
2021-07-19T04:15:37.2340151Z [cluster2]                                         "f:name": {},
2021-07-19T04:15:37.2340630Z [cluster2]                                         "f:valueFrom": {
2021-07-19T04:15:37.2341116Z [cluster2]                                             ".": {},
2021-07-19T04:15:37.2341580Z [cluster2]                                             "f:fieldRef": {
2021-07-19T04:15:37.2342201Z [cluster2]                                                 ".": {},
2021-07-19T04:15:37.2343023Z [cluster2]                                                 "f:apiVersion": {},
2021-07-19T04:15:37.2343510Z [cluster2]                                                 "f:fieldPath": {}
2021-07-19T04:15:37.2343939Z [cluster2]                                             }
2021-07-19T04:15:37.2344301Z [cluster2]                                         }
2021-07-19T04:15:37.2344724Z [cluster2]                                     },
2021-07-19T04:15:37.2345201Z [cluster2]                                     "k:{\"name\":\"WATCH_NAMESPACE\"}": {
2021-07-19T04:15:37.2345727Z [cluster2]                                         ".": {},
2021-07-19T04:15:37.2346147Z [cluster2]                                         "f:name": {},
2021-07-19T04:15:37.2346748Z [cluster2]                                         "f:valueFrom": {
2021-07-19T04:15:37.2347195Z [cluster2]                                             ".": {},
2021-07-19T04:15:37.2347680Z [cluster2]                                             "f:fieldRef": {
2021-07-19T04:15:37.2348331Z [cluster2]                                                 ".": {},
2021-07-19T04:15:37.2348806Z [cluster2]                                                 "f:apiVersion": {},
2021-07-19T04:15:37.2350015Z [cluster2]                                                 "f:fieldPath": {}
2021-07-19T04:15:37.2350530Z [cluster2]                                             }
2021-07-19T04:15:37.2351131Z [cluster2]                                         }
2021-07-19T04:15:37.2351736Z [cluster2]                                     }
2021-07-19T04:15:37.2352212Z [cluster2]                                 },
2021-07-19T04:15:37.2352864Z [cluster2]                                 "f:image": {},
2021-07-19T04:15:37.2353736Z [cluster2]                                 "f:imagePullPolicy": {},
2021-07-19T04:15:37.2354272Z [cluster2]                                 "f:name": {},
2021-07-19T04:15:37.2354722Z [cluster2]                                 "f:resources": {},
2021-07-19T04:15:37.2355352Z [cluster2]                                 "f:terminationMessagePath": {},
2021-07-19T04:15:37.2356079Z [cluster2]                                 "f:terminationMessagePolicy": {}
2021-07-19T04:15:37.2356684Z [cluster2]                             }
2021-07-19T04:15:37.2357055Z [cluster2]                         },
2021-07-19T04:15:37.2357508Z [cluster2]                         "f:dnsPolicy": {},
2021-07-19T04:15:37.2358048Z [cluster2]                         "f:enableServiceLinks": {},
2021-07-19T04:15:37.2358673Z [cluster2]                         "f:restartPolicy": {},
2021-07-19T04:15:37.2359628Z [cluster2]                         "f:schedulerName": {},
2021-07-19T04:15:37.2360314Z [cluster2]                         "f:securityContext": {},
2021-07-19T04:15:37.2360967Z [cluster2]                         "f:serviceAccount": {},
2021-07-19T04:15:37.2361617Z [cluster2]                         "f:serviceAccountName": {},
2021-07-19T04:15:37.2362509Z [cluster2]                         "f:terminationGracePeriodSeconds": {}
2021-07-19T04:15:37.2566908Z [cluster2]                     }
2021-07-19T04:15:37.2567381Z [cluster2]                 },
2021-07-19T04:15:37.2568380Z [cluster2]                 "manager": "kube-controller-manager",
2021-07-19T04:15:37.2569180Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:37.2570896Z [cluster2]                 "time": "2021-07-19T04:14:08Z"
2021-07-19T04:15:37.2571386Z [cluster2]             },
2021-07-19T04:15:37.2571814Z [cluster2]             {
2021-07-19T04:15:37.2572510Z [cluster2]                 "apiVersion": "v1",
2021-07-19T04:15:37.2573293Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:37.2573796Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:37.2574223Z [cluster2]                     "f:status": {
2021-07-19T04:15:37.2574689Z [cluster2]                         "f:conditions": {
2021-07-19T04:15:37.2575442Z [cluster2]                             "k:{\"type\":\"ContainersReady\"}": {
2021-07-19T04:15:37.2576243Z [cluster2]                                 ".": {},
2021-07-19T04:15:37.2576719Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-19T04:15:37.2577322Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-19T04:15:37.2577870Z [cluster2]                                 "f:status": {},
2021-07-19T04:15:37.2578300Z [cluster2]                                 "f:type": {}
2021-07-19T04:15:37.2578724Z [cluster2]                             },
2021-07-19T04:15:37.2579294Z [cluster2]                             "k:{\"type\":\"Initialized\"}": {
2021-07-19T04:15:37.2579782Z [cluster2]                                 ".": {},
2021-07-19T04:15:37.2580440Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-19T04:15:37.2581059Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-19T04:15:37.2581622Z [cluster2]                                 "f:status": {},
2021-07-19T04:15:37.2582094Z [cluster2]                                 "f:type": {}
2021-07-19T04:15:37.2583023Z [cluster2]                             },
2021-07-19T04:15:37.2583460Z [cluster2]                             "k:{\"type\":\"Ready\"}": {
2021-07-19T04:15:37.2583907Z [cluster2]                                 ".": {},
2021-07-19T04:15:37.2584377Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-19T04:15:37.2585262Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-19T04:15:37.2585799Z [cluster2]                                 "f:status": {},
2021-07-19T04:15:37.2586257Z [cluster2]                                 "f:type": {}
2021-07-19T04:15:37.2586659Z [cluster2]                             }
2021-07-19T04:15:37.2587050Z [cluster2]                         },
2021-07-19T04:15:37.2587554Z [cluster2]                         "f:containerStatuses": {},
2021-07-19T04:15:37.2588082Z [cluster2]                         "f:hostIP": {},
2021-07-19T04:15:37.2588536Z [cluster2]                         "f:phase": {},
2021-07-19T04:15:37.2588958Z [cluster2]                         "f:podIP": {},
2021-07-19T04:15:37.2589411Z [cluster2]                         "f:podIPs": {
2021-07-19T04:15:37.2589824Z [cluster2]                             ".": {},
2021-07-19T04:15:37.2590360Z [cluster2]                             "k:{\"ip\":\"10.0.0.3\"}": {
2021-07-19T04:15:37.2590780Z [cluster2]                                 ".": {},
2021-07-19T04:15:37.2591209Z [cluster2]                                 "f:ip": {}
2021-07-19T04:15:37.2591618Z [cluster2]                             }
2021-07-19T04:15:37.2591992Z [cluster2]                         },
2021-07-19T04:15:37.2592426Z [cluster2]                         "f:startTime": {}
2021-07-19T04:15:37.2592838Z [cluster2]                     }
2021-07-19T04:15:37.2593213Z [cluster2]                 },
2021-07-19T04:15:37.2593638Z [cluster2]                 "manager": "kubelet",
2021-07-19T04:15:37.2594156Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:37.2594900Z [cluster2]                 "time": "2021-07-19T04:14:25Z"
2021-07-19T04:15:37.2595311Z [cluster2]             }
2021-07-19T04:15:37.2595660Z [cluster2]         ],
2021-07-19T04:15:37.2596542Z [cluster2]         "name": "submariner-operator-5bfdb87494-jzjck",
2021-07-19T04:15:37.2597615Z [cluster2]         "namespace": "submariner-operator",
2021-07-19T04:15:37.2598245Z [cluster2]         "ownerReferences": [
2021-07-19T04:15:37.2598709Z [cluster2]             {
2021-07-19T04:15:37.2599505Z [cluster2]                 "apiVersion": "apps/v1",
2021-07-19T04:15:37.2600092Z [cluster2]                 "blockOwnerDeletion": true,
2021-07-19T04:15:37.2600651Z [cluster2]                 "controller": true,
2021-07-19T04:15:37.2601155Z [cluster2]                 "kind": "ReplicaSet",
2021-07-19T04:15:37.2602039Z [cluster2]                 "name": "submariner-operator-5bfdb87494",
2021-07-19T04:15:37.2603298Z [cluster2]                 "uid": "68f544c5-8726-4aa1-adbc-c11fb2fd6a8a"
2021-07-19T04:15:37.2603822Z [cluster2]             }
2021-07-19T04:15:37.2604348Z [cluster2]         ],
2021-07-19T04:15:37.2604811Z [cluster2]         "resourceVersion": "1907",
2021-07-19T04:15:37.2606238Z [cluster2]         "selfLink": "/api/v1/namespaces/submariner-operator/pods/submariner-operator-5bfdb87494-jzjck",
2021-07-19T04:15:37.2607451Z [cluster2]         "uid": "3e6023c3-0957-4b9a-ba34-0768b4d6a831"
2021-07-19T04:15:37.2607928Z [cluster2]     },
2021-07-19T04:15:37.2608294Z [cluster2]     "spec": {
2021-07-19T04:15:37.2608694Z [cluster2]         "containers": [
2021-07-19T04:15:37.2609093Z [cluster2]             {
2021-07-19T04:15:37.2609666Z [cluster2]                 "command": [
2021-07-19T04:15:37.2610364Z [cluster2]                     "submariner-operator",
2021-07-19T04:15:37.2611087Z [cluster2]                     "-alsologtostderr",
2021-07-19T04:15:37.2611699Z [cluster2]                     "-v=1"
2021-07-19T04:15:37.2612097Z [cluster2]                 ],
2021-07-19T04:15:37.2612475Z [cluster2]                 "env": [
2021-07-19T04:15:37.2613022Z [cluster2]                     {
2021-07-19T04:15:37.2613457Z [cluster2]                         "name": "WATCH_NAMESPACE",
2021-07-19T04:15:37.2613945Z [cluster2]                         "valueFrom": {
2021-07-19T04:15:37.2614384Z [cluster2]                             "fieldRef": {
2021-07-19T04:15:37.2614864Z [cluster2]                                 "apiVersion": "v1",
2021-07-19T04:15:37.2654286Z [cluster2]                                 "fieldPath": "metadata.namespace"
2021-07-19T04:15:37.2654885Z [cluster2]                             }
2021-07-19T04:15:37.2655239Z [cluster2]                         }
2021-07-19T04:15:37.2655711Z [cluster2]                     },
2021-07-19T04:15:37.2656073Z [cluster2]                     {
2021-07-19T04:15:37.2656472Z [cluster2]                         "name": "POD_NAME",
2021-07-19T04:15:37.2656940Z [cluster2]                         "valueFrom": {
2021-07-19T04:15:37.2657373Z [cluster2]                             "fieldRef": {
2021-07-19T04:15:37.2657928Z [cluster2]                                 "apiVersion": "v1",
2021-07-19T04:15:37.2658457Z [cluster2]                                 "fieldPath": "metadata.name"
2021-07-19T04:15:37.2658935Z [cluster2]                             }
2021-07-19T04:15:37.2659472Z [cluster2]                         }
2021-07-19T04:15:37.2660028Z [cluster2]                     },
2021-07-19T04:15:37.2660414Z [cluster2]                     {
2021-07-19T04:15:37.2660858Z [cluster2]                         "name": "OPERATOR_NAME",
2021-07-19T04:15:37.2661778Z [cluster2]                         "value": "submariner-operator"
2021-07-19T04:15:37.2662293Z [cluster2]                     }
2021-07-19T04:15:37.2662860Z [cluster2]                 ],
2021-07-19T04:15:37.2663573Z [cluster2]                 "image": "localhost:5000/submariner-operator:local",
2021-07-19T04:15:37.2664284Z [cluster2]                 "imagePullPolicy": "IfNotPresent",
2021-07-19T04:15:37.2665062Z [cluster2]                 "name": "submariner-operator",
2021-07-19T04:15:37.2665596Z [cluster2]                 "resources": {},
2021-07-19T04:15:37.2666387Z [cluster2]                 "terminationMessagePath": "/dev/termination-log",
2021-07-19T04:15:37.2667185Z [cluster2]                 "terminationMessagePolicy": "File",
2021-07-19T04:15:37.2667787Z [cluster2]                 "volumeMounts": [
2021-07-19T04:15:37.2668189Z [cluster2]                     {
2021-07-19T04:15:37.2668786Z [cluster2]                         "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
2021-07-19T04:15:37.2673403Z [cluster2]                         "name": "submariner-operator-token-klpkf",
2021-07-19T04:15:37.2674267Z [cluster2]                         "readOnly": true
2021-07-19T04:15:37.2675048Z [cluster2]                     }
2021-07-19T04:15:37.2675404Z [cluster2]                 ]
2021-07-19T04:15:37.2675741Z [cluster2]             }
2021-07-19T04:15:37.2676261Z [cluster2]         ],
2021-07-19T04:15:37.2676741Z [cluster2]         "dnsPolicy": "ClusterFirst",
2021-07-19T04:15:37.2677308Z [cluster2]         "enableServiceLinks": true,
2021-07-19T04:15:37.2678108Z [cluster2]         "nodeName": "cluster2-worker2",
2021-07-19T04:15:37.2678970Z [cluster2]         "preemptionPolicy": "PreemptLowerPriority",
2021-07-19T04:15:37.2679594Z [cluster2]         "priority": 0,
2021-07-19T04:15:37.2680049Z [cluster2]         "restartPolicy": "Always",
2021-07-19T04:15:37.2680849Z [cluster2]         "schedulerName": "default-scheduler",
2021-07-19T04:15:37.2681433Z [cluster2]         "securityContext": {},
2021-07-19T04:15:37.2682201Z [cluster2]         "serviceAccount": "submariner-operator",
2021-07-19T04:15:37.2683104Z [cluster2]         "serviceAccountName": "submariner-operator",
2021-07-19T04:15:37.2683927Z [cluster2]         "terminationGracePeriodSeconds": 30,
2021-07-19T04:15:37.2684561Z [cluster2]         "tolerations": [
2021-07-19T04:15:37.2684950Z [cluster2]             {
2021-07-19T04:15:37.2685376Z [cluster2]                 "effect": "NoExecute",
2021-07-19T04:15:37.2686103Z [cluster2]                 "key": "node.kubernetes.io/not-ready",
2021-07-19T04:15:37.2686675Z [cluster2]                 "operator": "Exists",
2021-07-19T04:15:37.2687214Z [cluster2]                 "tolerationSeconds": 300
2021-07-19T04:15:37.2687653Z [cluster2]             },
2021-07-19T04:15:37.2688005Z [cluster2]             {
2021-07-19T04:15:37.2688428Z [cluster2]                 "effect": "NoExecute",
2021-07-19T04:15:37.2689326Z [cluster2]                 "key": "node.kubernetes.io/unreachable",
2021-07-19T04:15:37.2689928Z [cluster2]                 "operator": "Exists",
2021-07-19T04:15:37.2690601Z [cluster2]                 "tolerationSeconds": 300
2021-07-19T04:15:37.2691053Z [cluster2]             }
2021-07-19T04:15:37.2691382Z [cluster2]         ],
2021-07-19T04:15:37.2691755Z [cluster2]         "volumes": [
2021-07-19T04:15:37.2692111Z [cluster2]             {
2021-07-19T04:15:37.2692943Z [cluster2]                 "name": "submariner-operator-token-klpkf",
2021-07-19T04:15:37.2693761Z [cluster2]                 "secret": {
2021-07-19T04:15:37.2694215Z [cluster2]                     "defaultMode": 420,
2021-07-19T04:15:37.2695133Z [cluster2]                     "secretName": "submariner-operator-token-klpkf"
2021-07-19T04:15:37.2695938Z [cluster2]                 }
2021-07-19T04:15:37.2696460Z [cluster2]             }
2021-07-19T04:15:37.2696973Z [cluster2]         ]
2021-07-19T04:15:37.2697365Z [cluster2]     },
2021-07-19T04:15:37.2697719Z [cluster2]     "status": {
2021-07-19T04:15:37.2698121Z [cluster2]         "conditions": [
2021-07-19T04:15:37.2698497Z [cluster2]             {
2021-07-19T04:15:37.2699129Z [cluster2]                 "lastProbeTime": null,
2021-07-19T04:15:37.2700223Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:08Z",
2021-07-19T04:15:37.2700754Z [cluster2]                 "status": "True",
2021-07-19T04:15:37.2701209Z [cluster2]                 "type": "Initialized"
2021-07-19T04:15:37.2701633Z [cluster2]             },
2021-07-19T04:15:37.2702140Z [cluster2]             {
2021-07-19T04:15:37.2702550Z [cluster2]                 "lastProbeTime": null,
2021-07-19T04:15:37.2703275Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:25Z",
2021-07-19T04:15:37.2703773Z [cluster2]                 "status": "True",
2021-07-19T04:15:37.2704194Z [cluster2]                 "type": "Ready"
2021-07-19T04:15:37.2704558Z [cluster2]             },
2021-07-19T04:15:37.2704895Z [cluster2]             {
2021-07-19T04:15:37.2705304Z [cluster2]                 "lastProbeTime": null,
2021-07-19T04:15:37.2706204Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:25Z",
2021-07-19T04:15:37.2706889Z [cluster2]                 "status": "True",
2021-07-19T04:15:37.2707405Z [cluster2]                 "type": "ContainersReady"
2021-07-19T04:15:37.2707872Z [cluster2]             },
2021-07-19T04:15:37.2708331Z [cluster2]             {
2021-07-19T04:15:37.4762168Z [clusterator@sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b",
2021-07-19T04:15:37.4763029Z [cluster1]                 "lastState": {
2021-07-19T04:15:37.4763664Z [cluster1]                     "terminated": {
2021-07-19T04:15:37.4765801Z [cluster1]                         "containerID": "containerd://5333bfaaee6d3682ef3737bdeb327aaa170d457d9bd28ae278764720bf661660",
2021-07-19T04:15:37.4767764Z [cluster1]                         "exitCode": 1,
2021-07-19T04:15:37.4768926Z [cluster1]                         "finishedAt": "2021-07-19T04:14:03Z",
2021-07-19T04:15:37.4769764Z [cluster1]                         "reason": "Error",
2021-07-19T04:15:37.4770607Z [cluster1]                         "startedAt": "2021-07-19T04:13:58Z"
2021-07-19T04:15:37.4771235Z [cluster1]                     }
2021-07-19T04:15:37.4771628Z [cluster1]                 },
2021-07-19T04:15:37.4772467Z [cluster1]                 "name": "submariner-operator",
2021-07-19T04:15:37.4773234Z [cluster1]                 "ready": true,
2021-07-19T04:15:37.4774120Z [cluster1]                 "restartCount": 1,
2021-07-19T04:15:37.4774567Z [cluster1]                 "started": true,
2021-07-19T04:15:37.4774989Z [cluster1]                 "state": {
2021-07-19T04:15:37.4775378Z [cluster1]                     "running": {
2021-07-19T04:15:37.4776258Z [cluster1]                         "startedAt": "2021-07-19T04:14:05Z"
2021-07-19T04:15:37.4776685Z [cluster1]                     }
2021-07-19T04:15:37.4777248Z [cluster1]                 }
2021-07-19T04:15:37.4777584Z [cluster1]             }
2021-07-19T04:15:37.4777932Z [cluster1]         ],
2021-07-19T04:15:37.4778299Z [cluster1]         "hostIP": "172.18.0.8",
2021-07-19T04:15:37.4778734Z [cluster1]         "phase": "Running",
2021-07-19T04:15:37.4779147Z [cluster1]         "podIP": "10.0.0.3",
2021-07-19T04:15:37.4779556Z [cluster1]         "podIPs": [
2021-07-19T04:15:37.4779929Z [cluster1]             {
2021-07-19T04:15:37.4780505Z [cluster1]                 "ip": "10.0.0.3"
2021-07-19T04:15:37.4780885Z [cluster1]             }
2021-07-19T04:15:37.4781225Z [cluster1]         ],
2021-07-19T04:15:37.4781662Z [cluster1]         "qosClass": "BestEffort",
2021-07-19T04:15:37.4782384Z [cluster1]         "startTime": "2021-07-19T04:13:56Z"
2021-07-19T04:15:37.4782803Z [cluster1]     }
2021-07-19T04:15:37.4783128Z [cluster1] }
2021-07-19T04:15:37.4784872Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-19T04:15:37.4787062Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-19T04:15:37.4793965Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-19T04:15:37.4795635Z [submariner-operator]$ [cluster1] grep Running
2021-07-19T04:15:37.4796138Z [cluster1] Running
2021-07-19T04:15:37.4797316Z [submariner-operator]$ [cluster1] kubectl logs submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator
2021-07-19T04:15:37.4832900Z [submariner-operator]$ [cluster1] kubectl logs submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator
2021-07-19T04:15:37.4842094Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 logs submariner-operator-5bfdb87494-rwjcx --namespace=submariner-operator
2021-07-19T04:15:37.4846822Z [cluster1] I0719 04:14:05.228295       1 main.go:77] cmd "msg"="Go Version: go1.16.5"  
2021-07-19T04:15:37.4847381Z [cluster1] I0719 04:14:05.228353       1 main.go:78] cmd "msg"="Go OS/Arch: linux/amd64"  
2021-07-19T04:15:37.4862265Z [cluster1] I0719 04:14:05.228360       1 main.go:79] cmd "msg"="Version of operator-sdk: v0.19.0"  
2021-07-19T04:15:37.4864216Z [cluster1] I0719 04:14:05.228366       1 main.go:80] cmd "msg"="Submariner operator version: devel-0105e4d"  
2021-07-19T04:15:37.4864976Z [cluster1] I0719 04:14:05.228523       1 leader.go:89] leader "msg"="Trying to become the leader."  
2021-07-19T04:15:37.4865734Z [cluster1] I0719 04:14:05.735314       1 leader.go:122] leader "msg"="Found existing lock with my name. I was likely restarted."  
2021-07-19T04:15:37.4878588Z [cluster1] I0719 04:14:05.735346       1 leader.go:123] leader "msg"="Continuing as the leader."  
2021-07-19T04:15:37.4905833Z [cluster1] I0719 04:14:06.242938       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"="0.0.0.0:8383"
2021-07-19T04:15:37.4906883Z [cluster1] I0719 04:14:06.244025       1 main.go:129] cmd "msg"="Registering Components."  
2021-07-19T04:15:37.4919892Z [cluster1] I0719 04:14:06.244227       1 main.go:137] cmd "msg"="Creating the Lighthouse CRDs"  
2021-07-19T04:15:37.4921213Z [cluster1] I0719 04:14:09.436730       1 main.go:182] cmd "msg"="Could not create ServiceMonitor object"  "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.4923265Z [cluster1] I0719 04:14:09.436780       1 main.go:186] cmd "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects"  "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.4924693Z [cluster1] I0719 04:14:09.436843       1 main.go:203] cmd "msg"="Starting the Cmd."  
2021-07-19T04:15:37.4925702Z [cluster1] I0719 04:14:09.437151       1 internal.go:406] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
2021-07-19T04:15:37.4929040Z [cluster1] I0719 04:14:09.437456       1 controller.go:158] controller-runtime/manager/controller/servicediscovery "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-19T04:15:37.4937783Z [cluster1] I0719 04:14:09.437619       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-19T04:15:37.4942799Z [cluster1] I0719 04:14:09.437763       1 controller.go:158] controller-runtime/manager/controller/broker "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Broker" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{}}}
2021-07-19T04:15:37.4948283Z [cluster1] I0719 04:14:09.437794       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-19T04:15:37.4954733Z [cluster1] I0719 04:14:09.438249       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-19T04:15:37.4958268Z [cluster1] I0719 04:14:09.540331       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-19T04:15:37.4961624Z [cluster1] I0719 04:14:09.540509       1 controller.go:165] controller-runtime/manager/controller/servicediscovery "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-19T04:15:37.4964041Z [cluster1] I0719 04:14:09.540584       1 controller.go:165] controller-runtime/manager/controller/broker "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-19T04:15:37.4967227Z [cluster1] I0719 04:14:09.540742       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"updateStrategy":{}},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}}
2021-07-19T04:15:37.4971991Z [cluster1] I0719 04:14:09.540864       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-19T04:15:37.4975038Z [cluster1] I0719 04:14:09.641016       1 controller.go:165] controller-runtime/manager/controller/submariner "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Submariner" 
2021-07-19T04:15:37.4977044Z [cluster1] I0719 04:14:09.641209       1 controller.go:192] controller-runtime/manager/controller/submariner "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "worker count"=1
2021-07-19T04:15:37.4979674Z [cluster1] I0719 04:14:09.641080       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"status":{"version":"","haStatus":"","localEndpoint":{"cluster_id":"","cable_name":"","hostname":"","subnets":null,"private_ip":"","public_ip":"","nat_enabled":false,"backend":""},"statusFailure":"","connections":null}}}
2021-07-19T04:15:37.4982622Z [cluster1] I0719 04:14:09.641595       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.4985001Z [cluster1] I0719 04:14:09.641129       1 controller.go:192] controller-runtime/manager/controller/broker "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Broker" "worker count"=1
2021-07-19T04:15:37.4986643Z [cluster1] I0719 04:14:09.641157       1 controller.go:165] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting Controller"  
2021-07-19T04:15:37.4988457Z [cluster1] I0719 04:14:09.641171       1 controller.go:192] controller-runtime/manager/controller/servicediscovery "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "worker count"=1
2021-07-19T04:15:37.4989850Z [cluster1] I0719 04:14:09.727442       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"  
2021-07-19T04:15:37.4991810Z [cluster1] I0719 04:14:09.727494       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.0.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.0.0.0/16"]
2021-07-19T04:15:37.4993322Z [cluster1] I0719 04:14:09.727535       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.4994849Z [cluster1] I0719 04:14:09.727577       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.4997041Z [cluster1] I0719 04:14:09.736127       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.4999157Z [cluster1] I0719 04:14:09.742499       1 controller.go:192] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting workers"  "worker count"=1
2021-07-19T04:15:37.5000704Z [cluster1] I0719 04:14:09.742540       1 controller.go:165] controller-runtime/manager/controller/submariner-controller "msg"="Starting Controller"  
2021-07-19T04:15:37.5002543Z [cluster1] I0719 04:14:09.742564       1 controller.go:192] controller-runtime/manager/controller/submariner-controller "msg"="Starting workers"  "worker count"=1
2021-07-19T04:15:37.5004837Z [cluster1] I0719 04:14:09.742653       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5006230Z [cluster1] I0719 04:14:09.742707       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5007313Z [cluster1] I0719 04:14:09.742725       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5009269Z [cluster1] I0719 04:14:09.776179       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5012987Z [cluster1] I0719 04:14:09.828004       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5081732Z [cluster1] I0719 04:14:09.965859       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5085258Z [cluster1] I0719 04:14:10.333388       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5087798Z [cluster1] I0719 04:14:10.351258       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5090656Z [cluster1] I0719 04:14:10.362683       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5093596Z [cluster1] I0719 04:14:10.382606       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5096356Z [cluster1] I0719 04:14:10.469179       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5098881Z [cluster1] I0719 04:14:11.038889       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5101177Z [cluster1] I0719 04:14:11.103892       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5103330Z [cluster1] I0719 04:14:11.104894       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5105909Z [cluster1] I0719 04:14:11.207575       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5109638Z [cluster1] E0719 04:14:11.207667       1 controller.go:267] controller-runtime/manager/controller/servicediscovery "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: Deployment.apps \"submariner-lighthouse-agent\" not found" "name"="service-discovery" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-19T04:15:37.5113192Z [cluster1] I0719 04:14:11.218065       1 servicediscovery_resources.go:66] controller_submariner "msg"="Created Service Discovery CR" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Name"="service-discovery" "Namespace"="submariner-operator"
2021-07-19T04:15:37.5115835Z [cluster1] I0719 04:14:11.252589       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5119251Z [cluster1] E0719 04:14:11.334669       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: deployments.apps \"submariner-lighthouse-agent\" already exists" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-19T04:15:37.5123162Z [cluster1] I0719 04:14:11.334715       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5126574Z [cluster1] I0719 04:14:11.570479       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5129578Z [cluster1] I0719 04:14:11.581945       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5132462Z [cluster1] I0719 04:14:11.694972       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5137629Z [cluster1] I0719 04:14:11.730401       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5139611Z [cluster1] I0719 04:14:11.730466       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5140915Z [cluster1] I0719 04:14:11.730477       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5143684Z [cluster1] I0719 04:14:11.990901       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5147327Z [cluster1] I0719 04:14:12.185041       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5150035Z [cluster1] I0719 04:14:12.510360       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5152753Z [cluster1] I0719 04:14:12.724673       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5156595Z [cluster1] I0719 04:14:12.725956       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5159346Z [cluster1] I0719 04:14:13.015368       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5162196Z [cluster1] E0719 04:14:13.207340       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5165181Z [cluster1] I0719 04:14:13.207582       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5167146Z [cluster1] I0719 04:14:13.207795       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5168269Z [cluster1] I0719 04:14:13.207922       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5170235Z [cluster1] I0719 04:14:13.228407       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5173159Z [cluster1] I0719 04:14:13.232847       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5206418Z [cluster1] I0719 04:14:13.393624       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5209236Z [cluster1] I0719 04:14:13.394702       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5212019Z [cluster1] I0719 04:14:13.421986       1 helpers.go:175] controller_servicediscovery "msg"="Created a new ConfigMap" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "ConfigMap.Name"="submariner-lighthouse-coredns" "ConfigMap.Namespace"="submariner-operator"
2021-07-19T04:15:37.5217186Z [cluster1] E0719 04:14:13.422147       1 servicediscovery_controller.go:151] controller_servicediscovery "msg"="Error creating the lighthouseCoreDNS configMap" "error"="error creating or updating ConfigMap submariner-operator/submariner-lighthouse-coredns: ConfigMap \"submariner-lighthouse-coredns\" not found"  
2021-07-19T04:15:37.5221894Z [cluster1] E0719 04:14:13.422300       1 controller.go:267] controller-runtime/manager/controller/servicediscovery "msg"="Reconciler error" "error"="error creating or updating ConfigMap submariner-operator/submariner-lighthouse-coredns: ConfigMap \"submariner-lighthouse-coredns\" not found" "name"="service-discovery" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-19T04:15:37.5226595Z [cluster1] I0719 04:14:13.433100       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5229246Z [cluster1] I0719 04:14:13.481045       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5231912Z [cluster1] I0719 04:14:13.604062       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5235763Z [cluster1] I0719 04:14:14.342921       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5240196Z [cluster1] E0719 04:14:14.343196       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-agent-metrics: services \"submariner-lighthouse-agent-metrics\" already exists" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-19T04:15:37.5243886Z [cluster1] I0719 04:14:14.343225       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5246912Z [cluster1] I0719 04:14:14.390476       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5259628Z [cluster1] I0719 04:14:14.880669       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5262997Z [cluster1] I0719 04:14:14.957864       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5266171Z [cluster1] I0719 04:14:15.036360       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5268925Z [cluster1] I0719 04:14:15.054040       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5271268Z [cluster1] I0719 04:14:15.084785       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5272599Z [cluster1] I0719 04:14:15.084836       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5273991Z [cluster1] I0719 04:14:15.085235       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5283688Z [cluster1] I0719 04:14:15.112766       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5287732Z [cluster1] I0719 04:14:15.165802       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5290665Z [cluster1] I0719 04:14:15.226862       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5294127Z [cluster1] I0719 04:14:15.352266       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5300011Z [cluster1] I0719 04:14:15.675748       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5304282Z [cluster1] I0719 04:14:15.855745       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5308681Z [cluster1] I0719 04:14:15.931084       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5312329Z [cluster1] I0719 04:14:15.996473       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5323083Z [cluster1] I0719 04:14:15.997064       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5341565Z [cluster1] I0719 04:14:16.584431       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5345697Z [cluster1] I0719 04:14:16.591491       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5350707Z [cluster1] E0719 04:14:16.600409       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5354496Z [cluster1] I0719 04:14:16.600525       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5356278Z [cluster1] I0719 04:14:16.600623       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5357825Z [cluster1] I0719 04:14:16.600639       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5360176Z [cluster1] I0719 04:14:16.893126       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5364502Z [cluster1] I0719 04:14:17.012371       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5368097Z [cluster1] I0719 04:14:17.264478       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5370783Z [cluster1] I0719 04:14:17.265188       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5390982Z [cluster1] I0719 04:14:17.545927       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5394783Z [cluster1] I0719 04:14:17.612096       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5400193Z [cluster1] I0719 04:14:17.615234       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5404030Z [cluster1] I0719 04:14:18.162397       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5407736Z [cluster1] I0719 04:14:18.174703       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5411543Z [cluster1] I0719 04:14:18.516860       1 servicediscovery_controller.go:487] controller_servicediscovery "msg"="coredns configmap does not have lighthouse configuration hence creating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5417492Z [cluster1] I0719 04:14:18.516999       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.229.9\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n"  
2021-07-19T04:15:37.5422229Z [cluster1] I0719 04:14:18.589952       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5426924Z [cluster1] I0719 04:14:18.736779       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5430228Z [cluster1] I0719 04:14:18.758041       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5434246Z [cluster1] I0719 04:14:18.758054       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5440397Z [cluster1] I0719 04:14:18.857495       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5446654Z [cluster1] E0719 04:14:18.901697       1 controller.go:267] controller-runtime/manager/controller/servicediscovery "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-coredns-metrics: services \"submariner-lighthouse-coredns-metrics\" already exists" "name"="service-discovery" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-19T04:15:37.5458544Z [cluster1] I0719 04:14:18.906550       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5462307Z [cluster1] I0719 04:14:18.923334       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5465789Z [cluster1] I0719 04:14:19.102799       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5470391Z [cluster1] I0719 04:14:19.106942       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5474043Z [cluster1] I0719 04:14:19.216143       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5477093Z [cluster1] I0719 04:14:19.322646       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5479819Z [cluster1] I0719 04:14:19.418533       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5489063Z [cluster1] I0719 04:14:19.687238       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5492536Z [cluster1] I0719 04:14:19.702824       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5497235Z [cluster1] I0719 04:14:19.722606       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5501866Z [cluster1] I0719 04:14:19.779805       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5505455Z [cluster1] I0719 04:14:19.792384       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5516365Z [cluster1] I0719 04:14:19.869318       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5524145Z [cluster1] I0719 04:14:20.242238       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5527603Z [cluster1] I0719 04:14:20.281988       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5531469Z [cluster1] I0719 04:14:20.282049       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.229.9\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n"  
2021-07-19T04:15:37.5535412Z [cluster1] I0719 04:14:20.377960       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5609425Z [cluster1] I0719 04:14:20.400635       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5613551Z [cluster1] I0719 04:14:20.400857       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.229.9\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n"  
2021-07-19T04:15:37.5617317Z [cluster1] I0719 04:14:30.398369       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5618842Z [cluster1] I0719 04:14:30.398494       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5620091Z [cluster1] I0719 04:14:30.398602       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5622339Z [cluster1] I0719 04:14:30.538658       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5625373Z [cluster1] I0719 04:14:30.588952       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5628174Z [cluster1] I0719 04:14:31.221289       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5630950Z [cluster1] I0719 04:14:31.334311       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5633761Z [cluster1] I0719 04:14:31.463989       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5637075Z [cluster1] I0719 04:14:31.646666       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5641781Z [cluster1] I0719 04:14:32.167860       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5644366Z [cluster1] I0719 04:14:32.220174       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5645967Z [cluster1] I0719 04:14:32.220391       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5647371Z [cluster1] I0719 04:14:32.220492       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5649108Z [cluster1] I0719 04:14:32.254537       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5650543Z [cluster1] I0719 04:14:32.254580       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5652210Z [cluster1] I0719 04:14:32.254591       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5655667Z [cluster1] I0719 04:14:32.284272       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5658893Z [cluster1] I0719 04:14:32.399416       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5661704Z [cluster1] I0719 04:14:32.518254       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5665143Z [cluster1] I0719 04:14:32.754440       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5668353Z [cluster1] I0719 04:14:33.032584       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5671124Z [cluster1] I0719 04:14:33.127095       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5674141Z [cluster1] I0719 04:14:33.290571       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5704420Z [cluster1] I0719 04:14:33.290675       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5707718Z [cluster1] I0719 04:14:33.349423       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5710499Z [cluster1] I0719 04:14:33.731502       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5713979Z [cluster1] I0719 04:14:33.732031       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5717318Z [cluster1] I0719 04:14:33.799680       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5720170Z [cluster1] I0719 04:14:33.815947       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5723755Z [cluster1] I0719 04:14:33.990740       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5727326Z [cluster1] I0719 04:14:34.017947       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5740964Z [cluster1] I0719 04:14:34.362934       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5744524Z [cluster1] I0719 04:14:34.622522       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5748235Z [cluster1] I0719 04:14:34.622843       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5752019Z [cluster1] E0719 04:14:34.825067       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5755039Z [cluster1] I0719 04:14:34.825129       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5762975Z [cluster1] I0719 04:14:34.825197       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5765589Z [cluster1] I0719 04:14:34.825208       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5770427Z [cluster1] I0719 04:14:35.148380       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5774566Z [cluster1] I0719 04:14:35.285984       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5779108Z [cluster1] I0719 04:14:35.455715       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5782424Z [cluster1] I0719 04:14:36.045357       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5785585Z [cluster1] I0719 04:14:36.047445       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5789699Z [cluster1] I0719 04:14:36.276545       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5795517Z [cluster1] I0719 04:14:36.276591       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.229.9\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n"  
2021-07-19T04:15:37.5800118Z [cluster1] I0719 04:14:36.303626       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5804554Z [cluster1] I0719 04:14:36.361867       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5809100Z [cluster1] I0719 04:14:36.488224       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5812611Z [cluster1] I0719 04:14:36.593670       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5815892Z [cluster1] I0719 04:14:36.847449       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5818991Z [cluster1] I0719 04:14:36.997801       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5821909Z [cluster1] I0719 04:14:37.032037       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5825536Z [cluster1] I0719 04:14:37.061977       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.5828876Z [cluster1] I0719 04:14:37.113040       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5832891Z [cluster1] I0719 04:14:37.228693       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5836302Z [cluster1] I0719 04:14:37.615255       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5839497Z [cluster1] I0719 04:14:37.657727       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5842455Z [cluster1] I0719 04:14:37.657885       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5844645Z [cluster1] I0719 04:14:37.657898       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5846746Z [cluster1] I0719 04:14:37.735376       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5848221Z [cluster1] I0719 04:14:37.735470       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5849441Z [cluster1] I0719 04:14:37.735488       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5851971Z [cluster1] I0719 04:14:37.739909       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5855822Z [cluster1] I0719 04:14:37.770588       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5860416Z [cluster1] I0719 04:14:37.770646       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.229.9\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n"  
2021-07-19T04:15:37.5866851Z [cluster1] I0719 04:14:37.780288       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5870446Z [cluster1] I0719 04:14:37.819781       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5874852Z [cluster1] I0719 04:14:37.839308       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5877953Z [cluster1] I0719 04:14:37.920559       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5880775Z [cluster1] I0719 04:14:38.325721       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5885392Z [cluster1] I0719 04:14:38.343616       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5888918Z [cluster1] I0719 04:14:38.364533       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5892599Z [cluster1] I0719 04:14:38.403095       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5895906Z [cluster1] I0719 04:14:38.423644       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5898765Z [cluster1] I0719 04:14:38.474408       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5901570Z [cluster1] I0719 04:14:38.538575       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5905103Z [cluster1] I0719 04:14:38.574618       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5908908Z [cluster1] I0719 04:14:38.908894       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5911986Z [cluster1] I0719 04:14:39.080638       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5915216Z [cluster1] I0719 04:14:42.265933       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5916878Z [cluster1] I0719 04:14:42.265991       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5918442Z [cluster1] I0719 04:14:42.266022       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5920615Z [cluster1] I0719 04:14:42.457358       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5923615Z [cluster1] I0719 04:14:42.617886       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5927015Z [cluster1] I0719 04:14:43.125261       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5929687Z [cluster1] I0719 04:14:43.141169       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5932381Z [cluster1] I0719 04:14:43.155559       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5936621Z [cluster1] I0719 04:14:43.172335       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5939492Z [cluster1] I0719 04:14:43.675161       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5943555Z [cluster1] I0719 04:14:43.745989       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5945576Z [cluster1] I0719 04:14:43.746546       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5948238Z [cluster1] I0719 04:14:43.746568       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5951033Z [cluster1] I0719 04:14:43.748525       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.5952456Z [cluster1] I0719 04:14:43.748565       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.5954240Z [cluster1] I0719 04:14:43.748601       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.5957202Z [cluster1] I0719 04:14:43.763810       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5960794Z [cluster1] I0719 04:14:43.768031       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5964763Z [cluster1] I0719 04:14:43.802644       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5969747Z [cluster1] I0719 04:14:43.802949       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5972708Z [cluster1] I0719 04:14:44.318750       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5977090Z [cluster1] I0719 04:14:44.319712       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.5980917Z [cluster1] I0719 04:14:44.381694       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5984065Z [cluster1] I0719 04:14:44.381732       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5988124Z [cluster1] I0719 04:14:44.439320       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5991764Z [cluster1] I0719 04:14:44.466120       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.5995499Z [cluster1] I0719 04:14:44.471473       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.5999334Z [cluster1] I0719 04:14:44.504925       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6003500Z [cluster1] I0719 04:14:44.972169       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6007425Z [cluster1] I0719 04:14:45.009141       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6010651Z [cluster1] I0719 04:14:45.010804       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6012406Z [cluster1] I0719 04:14:45.010846       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6014063Z [cluster1] I0719 04:14:45.010880       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6016803Z [cluster1] I0719 04:14:45.250506       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6020581Z [cluster1] I0719 04:14:45.412763       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6024290Z [cluster1] I0719 04:14:45.926875       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6028359Z [cluster1] I0719 04:14:45.943535       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6032211Z [cluster1] I0719 04:14:45.975561       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6035538Z [cluster1] I0719 04:14:46.090247       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6038651Z [cluster1] I0719 04:14:46.611712       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6041020Z [cluster1] I0719 04:14:46.710388       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6043972Z [cluster1] I0719 04:14:46.710435       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6045886Z [cluster1] I0719 04:14:46.710446       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6047801Z [cluster1] I0719 04:14:46.738782       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6049312Z [cluster1] I0719 04:14:46.738837       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6050732Z [cluster1] I0719 04:14:46.738851       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6053010Z [cluster1] I0719 04:14:46.946369       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6056504Z [cluster1] I0719 04:14:46.947754       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6059480Z [cluster1] I0719 04:14:47.001769       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6062749Z [cluster1] I0719 04:14:47.001780       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6066942Z [cluster1] I0719 04:14:47.520454       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6070079Z [cluster1] I0719 04:14:47.520600       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6074466Z [cluster1] I0719 04:14:47.599037       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6077313Z [cluster1] I0719 04:14:47.609798       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6080502Z [cluster1] I0719 04:14:47.616312       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6083354Z [cluster1] I0719 04:14:47.658203       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6087789Z [cluster1] I0719 04:14:47.695972       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6090887Z [cluster1] I0719 04:14:47.773401       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6093862Z [cluster1] I0719 04:14:48.201040       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6096842Z [cluster1] I0719 04:14:48.326289       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6099428Z [cluster1] I0719 04:14:48.386779       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6101672Z [cluster1] I0719 04:14:48.490644       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6103340Z [cluster1] I0719 04:14:48.490842       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6104892Z [cluster1] I0719 04:14:48.490928       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6107625Z [cluster1] I0719 04:14:48.611267       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.6110441Z [cluster1] I0719 04:14:48.613441       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6125043Z [cluster1] I0719 04:14:48.711798       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6128807Z [cluster1] I0719 04:14:48.782257       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6131784Z [cluster1] I0719 04:14:49.303136       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6134734Z [cluster1] I0719 04:14:49.329417       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:37.6162748Z [cluster1] I0719 04:14:49.414042       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6166589Z [cluster1] I0719 04:14:49.493058       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6169547Z [cluster1] I0719 04:14:49.574805       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6173205Z [cluster1] I0719 04:14:49.651470       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6176825Z [cluster1] I0719 04:14:49.676595       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6179508Z [cluster1] I0719 04:14:50.006963       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6182538Z [cluster1] I0719 04:14:50.023703       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6201645Z [cluster1] I0719 04:14:50.023753       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.229.9\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n"  
2021-07-19T04:15:37.6206733Z [cluster1] I0719 04:14:50.178946       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6209246Z [cluster1] I0719 04:14:50.216196       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6210622Z [cluster1] I0719 04:14:50.216370       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6212306Z [cluster1] I0719 04:14:50.216393       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6214510Z [cluster1] I0719 04:14:50.217925       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6216496Z [cluster1] I0719 04:14:50.217977       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6218024Z [cluster1] I0719 04:14:50.217988       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6220508Z [cluster1] I0719 04:14:50.239960       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6223873Z [cluster1] I0719 04:14:50.267788       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6227196Z [cluster1] I0719 04:14:50.285832       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6229883Z [cluster1] I0719 04:14:50.336588       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6232826Z [cluster1] I0719 04:14:50.770431       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6236368Z [cluster1] I0719 04:14:50.783168       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6239217Z [cluster1] I0719 04:14:50.819212       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6242704Z [cluster1] I0719 04:14:50.841373       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6247337Z [cluster1] I0719 04:14:50.847551       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6250844Z [cluster1] I0719 04:14:50.891124       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6254835Z [cluster1] I0719 04:14:50.944306       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6300428Z [cluster1] I0719 04:14:50.956855       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6303579Z [cluster1] I0719 04:14:51.351729       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6306760Z [cluster1] I0719 04:14:51.464545       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6309112Z [cluster1] I0719 04:15:02.474746       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6310536Z [cluster1] I0719 04:15:02.474942       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6311703Z [cluster1] I0719 04:15:02.475074       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6314474Z [cluster1] I0719 04:15:02.495853       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6320894Z [cluster1] I0719 04:15:02.529335       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6324130Z [cluster1] I0719 04:15:03.033177       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6327245Z [cluster1] I0719 04:15:03.043752       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6333562Z [cluster1] I0719 04:15:03.058957       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6337352Z [cluster1] I0719 04:15:03.066310       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6366297Z [cluster1] I0719 04:15:03.577103       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6368872Z [cluster1] I0719 04:15:03.604071       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6370490Z [cluster1] I0719 04:15:03.604245       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6371911Z [cluster1] I0719 04:15:03.604273       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6374476Z [cluster1] I0719 04:15:03.620675       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6376339Z [cluster1] I0719 04:15:03.620784       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6378194Z [cluster1] I0719 04:15:03.620860       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6380469Z [cluster1] I0719 04:15:03.645237       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6383305Z [cluster1] I0719 04:15:03.645504       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6386474Z [cluster1] I0719 04:15:03.673225       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6389376Z [cluster1] I0719 04:15:03.673820       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6392168Z [cluster1] I0719 04:15:04.174869       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6394887Z [cluster1] I0719 04:15:04.175683       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6397656Z [cluster1] I0719 04:15:04.183447       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6403631Z [cluster1] I0719 04:15:04.185831       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6407005Z [cluster1] I0719 04:15:04.190362       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6409940Z [cluster1] I0719 04:15:04.190916       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6413279Z [cluster1] I0719 04:15:04.198866       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6416723Z [cluster1] I0719 04:15:04.199889       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6419655Z [cluster1] I0719 04:15:04.700521       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6422511Z [cluster1] I0719 04:15:04.701627       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6424806Z [cluster1] I0719 04:15:07.526246       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6426253Z [cluster1] I0719 04:15:07.526337       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6427479Z [cluster1] I0719 04:15:07.526350       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6429755Z [cluster1] I0719 04:15:07.539215       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6433293Z [cluster1] I0719 04:15:07.548795       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6437247Z [cluster1] I0719 04:15:08.051284       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6441161Z [cluster1] I0719 04:15:08.061131       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6485298Z [cluster1] I0719 04:15:08.067817       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6488385Z [cluster1] I0719 04:15:08.075048       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6491492Z [cluster1] I0719 04:15:08.578153       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6494254Z [cluster1] I0719 04:15:08.600833       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6496377Z [cluster1] I0719 04:15:08.601107       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6497606Z [cluster1] I0719 04:15:08.601201       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6501178Z [cluster1] I0719 04:15:08.640377       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6504282Z [cluster1] I0719 04:15:08.648172       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6506108Z [cluster1] I0719 04:15:08.648343       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6507332Z [cluster1] I0719 04:15:08.648455       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6509532Z [cluster1] I0719 04:15:08.650019       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6512928Z [cluster1] I0719 04:15:08.680205       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6516243Z [cluster1] I0719 04:15:08.689293       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6519017Z [cluster1] I0719 04:15:09.151772       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6522245Z [cluster1] I0719 04:15:09.168196       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6525485Z [cluster1] I0719 04:15:09.174162       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6555582Z [cluster1] I0719 04:15:09.178574       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6559197Z [cluster1] I0719 04:15:09.191039       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6562316Z [cluster1] I0719 04:15:09.198222       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6566114Z [cluster1] I0719 04:15:09.202612       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6568931Z [cluster1] I0719 04:15:09.209776       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6571762Z [cluster1] I0719 04:15:09.680115       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6574876Z [cluster1] I0719 04:15:09.711236       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6577724Z [cluster1] I0719 04:15:12.541917       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6579242Z [cluster1] I0719 04:15:12.541977       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6581114Z [cluster1] I0719 04:15:12.541988       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6584135Z [cluster1] I0719 04:15:12.551991       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6587963Z [cluster1] I0719 04:15:12.560114       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6591478Z [cluster1] I0719 04:15:13.062782       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6594717Z [cluster1] I0719 04:15:13.076409       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6597596Z [cluster1] I0719 04:15:13.088295       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6600524Z [cluster1] I0719 04:15:13.099447       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6604711Z [cluster1] I0719 04:15:13.603330       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6608345Z [cluster1] I0719 04:15:13.620490       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6609865Z [cluster1] I0719 04:15:13.620676       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6611130Z [cluster1] I0719 04:15:13.620699       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6613794Z [cluster1] I0719 04:15:13.639084       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6616803Z [cluster1] I0719 04:15:13.647437       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6618537Z [cluster1] I0719 04:15:13.647474       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6619818Z [cluster1] I0719 04:15:13.647484       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6622973Z [cluster1] I0719 04:15:13.666528       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6626420Z [cluster1] I0719 04:15:13.674434       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6629539Z [cluster1] I0719 04:15:13.701226       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6632596Z [cluster1] I0719 04:15:14.168405       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6635558Z [cluster1] I0719 04:15:14.181625       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6638564Z [cluster1] I0719 04:15:14.196393       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6641502Z [cluster1] I0719 04:15:14.209451       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6646009Z [cluster1] I0719 04:15:14.218598       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6650252Z [cluster1] I0719 04:15:14.277582       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6655952Z [cluster1] I0719 04:15:14.286572       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6660157Z [cluster1] I0719 04:15:14.326958       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6664054Z [cluster1] I0719 04:15:14.723585       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6667279Z [cluster1] I0719 04:15:14.832124       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6670215Z [cluster1] I0719 04:15:17.641900       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6672115Z [cluster1] I0719 04:15:17.641977       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6673425Z [cluster1] I0719 04:15:17.641990       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6675888Z [cluster1] I0719 04:15:17.700815       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6678960Z [cluster1] I0719 04:15:17.705493       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6682238Z [cluster1] I0719 04:15:18.208002       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6685294Z [cluster1] I0719 04:15:18.218173       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6688502Z [cluster1] I0719 04:15:18.233496       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6691528Z [cluster1] I0719 04:15:18.239124       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6694667Z [cluster1] I0719 04:15:18.742363       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6697701Z [cluster1] I0719 04:15:18.754326       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6699265Z [cluster1] I0719 04:15:18.754371       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6700555Z [cluster1] I0719 04:15:18.754383       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6702417Z [cluster1] I0719 04:15:18.756098       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6703971Z [cluster1] I0719 04:15:18.756135       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6705252Z [cluster1] I0719 04:15:18.756146       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6709271Z [cluster1] I0719 04:15:18.780561       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6712656Z [cluster1] I0719 04:15:18.781013       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6715541Z [cluster1] I0719 04:15:18.788431       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6718807Z [cluster1] I0719 04:15:18.788824       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6722450Z [cluster1] I0719 04:15:19.294348       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6725485Z [cluster1] I0719 04:15:19.300373       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6728430Z [cluster1] I0719 04:15:19.319757       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6732526Z [cluster1] I0719 04:15:19.325239       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6735918Z [cluster1] I0719 04:15:19.347151       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6738828Z [cluster1] I0719 04:15:19.382099       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6741769Z [cluster1] I0719 04:15:19.394669       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6749686Z [cluster1] I0719 04:15:19.458174       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6754159Z [cluster1] I0719 04:15:19.905156       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6760186Z [cluster1] I0719 04:15:19.960825       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6776014Z [cluster1] I0719 04:15:22.708722       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6777583Z [cluster1] I0719 04:15:22.708902       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6778981Z [cluster1] I0719 04:15:22.709033       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6781380Z [cluster1] I0719 04:15:22.717884       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6784412Z [cluster1] I0719 04:15:22.722137       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6787515Z [cluster1] I0719 04:15:23.224295       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6804288Z [cluster1] I0719 04:15:23.236041       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6807716Z [cluster1] I0719 04:15:23.242350       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6810841Z [cluster1] I0719 04:15:23.247084       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6813705Z [cluster1] I0719 04:15:23.749078       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6816301Z [cluster1] I0719 04:15:23.768592       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6818051Z [cluster1] I0719 04:15:23.768807       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6819540Z [cluster1] I0719 04:15:23.768969       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6821198Z [cluster1] I0719 04:15:23.774857       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6822543Z [cluster1] I0719 04:15:23.775564       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6823688Z [cluster1] I0719 04:15:23.775591       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6825964Z [cluster1] I0719 04:15:23.789052       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6833653Z [cluster1] I0719 04:15:23.796558       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6837432Z [cluster1] I0719 04:15:23.838325       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6840830Z [cluster1] I0719 04:15:23.842871       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6844416Z [cluster1] I0719 04:15:24.297855       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6847919Z [cluster1] I0719 04:15:24.309938       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6852281Z [cluster1] I0719 04:15:24.314564       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6856139Z [cluster1] I0719 04:15:24.324079       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6858828Z [cluster1] I0719 04:15:24.344495       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6861646Z [cluster1] I0719 04:15:24.351643       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6868799Z [cluster1] I0719 04:15:24.356910       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6872082Z [cluster1] I0719 04:15:24.374246       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6874970Z [cluster1] I0719 04:15:24.827964       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6878667Z [cluster1] I0719 04:15:24.877339       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6881147Z [cluster1] I0719 04:15:27.734409       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6882563Z [cluster1] I0719 04:15:27.734507       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6883733Z [cluster1] I0719 04:15:27.734524       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6886240Z [cluster1] I0719 04:15:27.807015       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6889466Z [cluster1] I0719 04:15:27.814346       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6892078Z [cluster1] I0719 04:15:28.316475       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6894879Z [cluster1] I0719 04:15:28.323489       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6899074Z [cluster1] I0719 04:15:28.328779       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6902488Z [cluster1] I0719 04:15:28.334481       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6906351Z [cluster1] I0719 04:15:28.836476       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6908830Z [cluster1] I0719 04:15:28.856638       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6910237Z [cluster1] I0719 04:15:28.856778       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6911402Z [cluster1] I0719 04:15:28.856790       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6913087Z [cluster1] I0719 04:15:28.863524       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6914474Z [cluster1] I0719 04:15:28.863668       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6915891Z [cluster1] I0719 04:15:28.863809       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6918303Z [cluster1] I0719 04:15:28.918384       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6921181Z [cluster1] I0719 04:15:28.919093       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6923896Z [cluster1] I0719 04:15:28.930142       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6926996Z [cluster1] I0719 04:15:28.930528       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6930216Z [cluster1] I0719 04:15:29.431770       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6947806Z [cluster1] I0719 04:15:29.432795       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6951700Z [cluster1] I0719 04:15:29.443808       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6956214Z [cluster1] I0719 04:15:29.449850       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6959935Z [cluster1] I0719 04:15:29.452873       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6962942Z [cluster1] I0719 04:15:29.461629       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6966086Z [cluster1] I0719 04:15:29.480129       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6969782Z [cluster1] I0719 04:15:29.485317       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6972499Z [cluster1] I0719 04:15:29.963763       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6975145Z [cluster1] I0719 04:15:29.987498       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6978195Z [cluster1] I0719 04:15:32.803290       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.6979799Z [cluster1] I0719 04:15:32.803361       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.6980967Z [cluster1] I0719 04:15:32.803379       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.6983178Z [cluster1] I0719 04:15:32.822138       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6986154Z [cluster1] I0719 04:15:32.865672       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.6989370Z [cluster1] I0719 04:15:33.367972       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.6991981Z [cluster1] I0719 04:15:33.374169       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6994605Z [cluster1] I0719 04:15:33.379181       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.6997813Z [cluster1] I0719 04:15:33.385396       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:37.7001500Z [cluster1] I0719 04:15:33.887278       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:37.7003799Z [cluster1] I0719 04:15:33.912458       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.7005658Z [cluster1] I0719 04:15:33.912502       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.7007094Z [cluster1] I0719 04:15:33.912512       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.7009212Z [cluster1] I0719 04:15:33.915092       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:37.7010714Z [cluster1] I0719 04:15:33.915132       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:37.7011857Z [cluster1] I0719 04:15:33.915150       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:37.7014072Z [cluster1] I0719 04:15:33.942004       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.7017815Z [cluster1] I0719 04:15:33.942491       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:37.7020943Z [cluster1] I0719 04:15:33.951821       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3385385Z [cluster1] I0719 04:15:33.952532       1 helpers.go:233] controller_submariner "msg"="Updated exisr2]                 "lastProbeTime": null,
2021-07-19T04:15:38.3386987Z [cluster2]                 "lastTransitionTime": "2021-07-19T04:14:08Z",
2021-07-19T04:15:38.3387589Z [cluster2]                 "status": "True",
2021-07-19T04:15:38.3388102Z [cluster2]                 "type": "PodScheduled"
2021-07-19T04:15:38.3388934Z [cluster2]             }
2021-07-19T04:15:38.3389585Z [cluster2]         ],
2021-07-19T04:15:38.3391752Z [cluster2]         "containerStatuses": [
2021-07-19T04:15:38.3392281Z [cluster2]             {
2021-07-19T04:15:38.3394589Z [cluster2]                 "containerID": "containerd://2f146176bb7dce30821b24d001cd6a621c46dffaa4027602fb57717f0b92d96b",
2021-07-19T04:15:38.3401778Z [cluster2]                 "image": "localhost:5000/submariner-operator:local",
2021-07-19T04:15:38.3403642Z [cluster2]                 "imageID": "localhost:5000/submariner-operator@sha256:4fc688208e3986e77644027577c7d3896e83b320cc64da4dcdbe1f94adcbda8b",
2021-07-19T04:15:38.3404658Z [cluster2]                 "lastState": {
2021-07-19T04:15:38.3405124Z [cluster2]                     "terminated": {
2021-07-19T04:15:38.3407700Z [cluster2]                         "containerID": "containerd://e16ba792fb5ddd391a6430f03c6ef69679233cd378d57be23915cc834bf8d21c",
2021-07-19T04:15:38.3409258Z [cluster2]                         "exitCode": 1,
2021-07-19T04:15:38.3410096Z [cluster2]                         "finishedAt": "2021-07-19T04:14:21Z",
2021-07-19T04:15:38.3410912Z [cluster2]                         "reason": "Error",
2021-07-19T04:15:38.3411632Z [cluster2]                         "startedAt": "2021-07-19T04:14:09Z"
2021-07-19T04:15:38.3412087Z [cluster2]                     }
2021-07-19T04:15:38.3412448Z [cluster2]                 },
2021-07-19T04:15:38.3413255Z [cluster2]                 "name": "submariner-operator",
2021-07-19T04:15:38.3413802Z [cluster2]                 "ready": true,
2021-07-19T04:15:38.3414287Z [cluster2]                 "restartCount": 1,
2021-07-19T04:15:38.3414786Z [cluster2]                 "started": true,
2021-07-19T04:15:38.3415215Z [cluster2]                 "state": {
2021-07-19T04:15:38.3416387Z [cluster2]                     "running": {
2021-07-19T04:15:38.3417130Z [cluster2]                         "startedAt": "2021-07-19T04:14:24Z"
2021-07-19T04:15:38.3417591Z [cluster2]                     }
2021-07-19T04:15:38.3417953Z [cluster2]                 }
2021-07-19T04:15:38.3418327Z [cluster2]             }
2021-07-19T04:15:38.3418686Z [cluster2]         ],
2021-07-19T04:15:38.3419091Z [cluster2]         "hostIP": "172.18.0.4",
2021-07-19T04:15:38.3419531Z [cluster2]         "phase": "Running",
2021-07-19T04:15:38.3420036Z [cluster2]         "podIP": "10.0.0.3",
2021-07-19T04:15:38.3420597Z [cluster2]         "podIPs": [
2021-07-19T04:15:38.3420983Z [cluster2]             {
2021-07-19T04:15:38.3421378Z [cluster2]                 "ip": "10.0.0.3"
2021-07-19T04:15:38.3421750Z [cluster2]             }
2021-07-19T04:15:38.3422124Z [cluster2]         ],
2021-07-19T04:15:38.3422560Z [cluster2]         "qosClass": "BestEffort",
2021-07-19T04:15:38.3423285Z [cluster2]         "startTime": "2021-07-19T04:14:08Z"
2021-07-19T04:15:38.3423698Z [cluster2]     }
2021-07-19T04:15:38.3424053Z [cluster2] }
2021-07-19T04:15:38.3425428Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-19T04:15:38.3426899Z [submariner-operator]$ [cluster2] grep Running
2021-07-19T04:15:38.3428460Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-19T04:15:38.3430703Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pod submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-19T04:15:38.3431959Z [cluster2] Running
2021-07-19T04:15:38.3433929Z [submariner-operator]$ [cluster2] kubectl logs submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator
2021-07-19T04:15:38.3436031Z [submariner-operator]$ [cluster2] kubectl logs submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator
2021-07-19T04:15:38.3440410Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 logs submariner-operator-5bfdb87494-jzjck --namespace=submariner-operator
2021-07-19T04:15:38.3442117Z [cluster2] I0719 04:14:25.308248       1 main.go:77] cmd "msg"="Go Version: go1.16.5"  
2021-07-19T04:15:38.3442769Z [cluster2] I0719 04:14:25.313676       1 main.go:78] cmd "msg"="Go OS/Arch: linux/amd64"  
2021-07-19T04:15:38.3444020Z [cluster2] I0719 04:14:25.313757       1 main.go:79] cmd "msg"="Version of operator-sdk: v0.19.0"  
2021-07-19T04:15:38.3444997Z [cluster2] I0719 04:14:25.313823       1 main.go:80] cmd "msg"="Submariner operator version: devel-0105e4d"  
2021-07-19T04:15:38.3445809Z [cluster2] I0719 04:14:25.314072       1 leader.go:89] leader "msg"="Trying to become the leader."  
2021-07-19T04:15:38.3446642Z [cluster2] I0719 04:14:25.876266       1 leader.go:122] leader "msg"="Found existing lock with my name. I was likely restarted."  
2021-07-19T04:15:38.3447481Z [cluster2] I0719 04:14:25.876382       1 leader.go:123] leader "msg"="Continuing as the leader."  
2021-07-19T04:15:38.3448667Z [cluster2] I0719 04:14:26.388538       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"="0.0.0.0:8383"
2021-07-19T04:15:38.3449752Z [cluster2] I0719 04:14:26.388794       1 main.go:129] cmd "msg"="Registering Components."  
2021-07-19T04:15:38.3450480Z [cluster2] I0719 04:14:26.388904       1 main.go:137] cmd "msg"="Creating the Lighthouse CRDs"  
2021-07-19T04:15:38.3451521Z [cluster2] I0719 04:14:30.091695       1 main.go:182] cmd "msg"="Could not create ServiceMonitor object"  "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3453583Z [cluster2] I0719 04:14:30.091734       1 main.go:186] cmd "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects"  "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3454669Z [cluster2] I0719 04:14:30.091793       1 main.go:203] cmd "msg"="Starting the Cmd."  
2021-07-19T04:15:38.3482882Z [cluster2] I0719 04:14:30.092111       1 internal.go:406] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
2021-07-19T04:15:38.3496522Z [cluster2] I0719 04:14:30.097869       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-19T04:15:38.3513982Z [cluster2] I0719 04:14:30.101099       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-19T04:15:38.3520281Z [cluster2] I0719 04:14:30.101685       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-19T04:15:38.3523741Z [cluster2] I0719 04:14:30.119572       1 controller.go:158] controller-runtime/manager/controller/broker "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Broker" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{}}}
2021-07-19T04:15:38.3527873Z [cluster2] I0719 04:14:30.122532       1 controller.go:158] controller-runtime/manager/controller/servicediscovery "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-19T04:15:38.3531662Z [cluster2] I0719 04:14:30.203112       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"updateStrategy":{}},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}}
2021-07-19T04:15:38.3553708Z [cluster2] I0719 04:14:30.205779       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-19T04:15:38.3567413Z [cluster2] I0719 04:14:30.230088       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-19T04:15:38.3574600Z [cluster2] I0719 04:14:30.230266       1 controller.go:165] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting Controller"  
2021-07-19T04:15:38.3596015Z [cluster2] I0719 04:14:30.230389       1 controller.go:192] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting workers"  "worker count"=1
2021-07-19T04:15:38.3598772Z [cluster2] I0719 04:14:30.235692       1 controller.go:165] controller-runtime/manager/controller/broker "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-19T04:15:38.3609574Z [cluster2] I0719 04:14:30.235840       1 controller.go:192] controller-runtime/manager/controller/broker "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Broker" "worker count"=1
2021-07-19T04:15:38.3611530Z [cluster2] I0719 04:14:30.241991       1 controller.go:165] controller-runtime/manager/controller/servicediscovery "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-19T04:15:38.3614008Z [cluster2] I0719 04:14:30.242137       1 controller.go:192] controller-runtime/manager/controller/servicediscovery "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "worker count"=1
2021-07-19T04:15:38.3618172Z [cluster2] I0719 04:14:30.303630       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"status":{"version":"","haStatus":"","localEndpoint":{"cluster_id":"","cable_name":"","hostname":"","subnets":null,"private_ip":"","public_ip":"","nat_enabled":false,"backend":""},"statusFailure":"","connections":null}}}
2021-07-19T04:15:38.3621422Z [cluster2] I0719 04:14:30.306224       1 controller.go:165] controller-runtime/manager/controller/submariner "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Submariner" 
2021-07-19T04:15:38.3623777Z [cluster2] I0719 04:14:30.404500       1 controller.go:165] controller-runtime/manager/controller/submariner-controller "msg"="Starting Controller"  
2021-07-19T04:15:38.3626678Z [cluster2] I0719 04:14:30.404727       1 controller.go:192] controller-runtime/manager/controller/submariner-controller "msg"="Starting workers"  "worker count"=1
2021-07-19T04:15:38.3629260Z [cluster2] I0719 04:14:30.404931       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3631588Z [cluster2] I0719 04:14:30.410350       1 controller.go:192] controller-runtime/manager/controller/submariner "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "worker count"=1
2021-07-19T04:15:38.3633951Z [cluster2] I0719 04:14:30.410534       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3637737Z [cluster2] I0719 04:14:30.731807       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"  
2021-07-19T04:15:38.3642063Z [cluster2] I0719 04:14:30.731912       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.0.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.0.0.0/16"]
2021-07-19T04:15:38.3643898Z [cluster2] I0719 04:14:30.731971       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.3645305Z [cluster2] I0719 04:14:30.732069       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.3647549Z [cluster2] I0719 04:14:30.765435       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3649176Z [cluster2] I0719 04:14:31.050809       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"  
2021-07-19T04:15:38.3650567Z [cluster2] I0719 04:14:31.051441       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.0.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.0.0.0/16"]
2021-07-19T04:15:38.3654272Z [cluster2] I0719 04:14:31.051669       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.3656066Z [cluster2] I0719 04:14:31.052513       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.3659074Z [cluster2] I0719 04:14:31.217043       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3666151Z [cluster2] E0719 04:14:31.218308       1 controller.go:267] controller-runtime/manager/controller/submariner-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-gateway-metrics: Service \"submariner-gateway-metrics\" not found" "name"="submariner" "namespace"="submariner-operator" 
2021-07-19T04:15:38.3670674Z [cluster2] I0719 04:14:31.218986       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3672276Z [cluster2] I0719 04:14:31.219499       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.3674189Z [cluster2] I0719 04:14:31.219990       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.3676721Z [cluster2] I0719 04:14:31.301460       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3679969Z [cluster2] I0719 04:14:31.312532       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3682957Z [cluster2] I0719 04:14:31.394589       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3685942Z [cluster2] I0719 04:14:31.398881       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3690555Z [cluster2] I0719 04:14:31.897598       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3694926Z [cluster2] I0719 04:14:31.902794       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3699041Z [cluster2] E0719 04:14:31.902848       1 controller.go:267] controller-runtime/manager/controller/submariner-controller "msg"="Reconciler error" "error"="error creating or updating DaemonSet submariner-operator/submariner-routeagent: DaemonSet.apps \"submariner-routeagent\" not found" "name"="submariner" "namespace"="submariner-operator" 
2021-07-19T04:15:38.3701575Z [cluster2] I0719 04:14:31.902913       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3707992Z [cluster2] I0719 04:14:31.903044       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.3709596Z [cluster2] I0719 04:14:31.903066       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.3714752Z [cluster2] I0719 04:14:31.920456       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3719577Z [cluster2] I0719 04:14:31.946109       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3722972Z [cluster2] I0719 04:14:31.953446       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3730933Z [cluster2] I0719 04:14:32.057732       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3734915Z [cluster2] I0719 04:14:32.105944       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3738803Z [cluster2] I0719 04:14:32.304793       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3745012Z [cluster2] E0719 04:14:32.304916       1 controller.go:267] controller-runtime/manager/controller/submariner "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-globalnet-metrics: Service \"submariner-globalnet-metrics\" not found" "name"="submariner" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Submariner" 
2021-07-19T04:15:38.3749334Z [cluster2] I0719 04:14:32.310589       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3751547Z [cluster2] I0719 04:14:32.310636       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.3752706Z [cluster2] I0719 04:14:32.310647       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.3755468Z [cluster2] I0719 04:14:32.608342       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3759819Z [cluster2] I0719 04:14:33.040962       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3762969Z [cluster2] I0719 04:14:33.548273       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3766856Z [cluster2] I0719 04:14:33.601293       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3769681Z [cluster2] I0719 04:14:33.765162       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3772659Z [cluster2] I0719 04:14:33.874501       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3778272Z [cluster2] I0719 04:14:34.110484       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3781424Z [cluster2] I0719 04:14:34.215227       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3785626Z [cluster2] I0719 04:14:34.280718       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3789536Z [cluster2] I0719 04:14:34.319816       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3799883Z [cluster2] I0719 04:14:34.426711       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3804230Z [cluster2] I0719 04:14:34.528069       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3807486Z [cluster2] I0719 04:14:34.528663       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3811210Z [cluster2] I0719 04:14:34.559804       1 servicediscovery_resources.go:66] controller_submariner "msg"="Created Service Discovery CR" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Name"="service-discovery" "Namespace"="submariner-operator"
2021-07-19T04:15:38.3815313Z [cluster2] I0719 04:14:34.885746       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3822780Z [cluster2] I0719 04:14:35.122382       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.3827239Z [cluster2] E0719 04:14:35.122541       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: Deployment.apps \"submariner-lighthouse-agent\" not found" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-19T04:15:38.3830803Z [cluster2] E0719 04:14:35.122788       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3834356Z [cluster2] I0719 04:14:35.122927       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3836390Z [cluster2] I0719 04:14:35.122968       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.3837681Z [cluster2] I0719 04:14:35.123111       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.3839670Z [cluster2] I0719 04:14:35.123172       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3844014Z [cluster2] I0719 04:14:35.144283       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3845718Z [cluster2] I0719 04:14:35.144331       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.3847599Z [cluster2] I0719 04:14:35.144342       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.3850595Z [cluster2] E0719 04:14:35.331770       1 controller.go:267] controller-runtime/manager/controller/servicediscovery "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: deployments.apps \"submariner-lighthouse-agent\" already exists" "name"="service-discovery" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-19T04:15:38.3854265Z [cluster2] I0719 04:14:35.337025       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.3858085Z [cluster2] I0719 04:14:35.463233       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3860695Z [cluster2] I0719 04:14:35.467450       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3865942Z [cluster2] I0719 04:14:35.556244       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3870227Z [cluster2] I0719 04:14:35.701922       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3878106Z [cluster2] I0719 04:14:35.737222       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.3881794Z [cluster2] I0719 04:14:35.737652       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.3885469Z [cluster2] I0719 04:14:36.058123       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3888772Z [cluster2] I0719 04:14:36.216153       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3899427Z [cluster2] I0719 04:14:36.599728       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3904961Z [cluster2] I0719 04:14:36.743103       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3908964Z [cluster2] I0719 04:14:36.886262       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3911678Z [cluster2] I0719 04:14:36.892440       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3914924Z [cluster2] I0719 04:14:37.254727       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.3918423Z [cluster2] I0719 04:14:37.360336       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.3922537Z [cluster2] I0719 04:14:37.480989       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3927753Z [cluster2] I0719 04:14:37.487001       1 helpers.go:175] controller_servicediscovery "msg"="Created a new ConfigMap" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "ConfigMap.Name"="submariner-lighthouse-coredns" "ConfigMap.Namespace"="submariner-operator"
2021-07-19T04:15:38.3979308Z [cluster2] I0719 04:14:37.606665       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.3982484Z [cluster2] I0719 04:14:37.970763       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.3986079Z [cluster2] I0719 04:14:37.997027       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4049994Z [cluster2] E0719 04:14:38.143998       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-agent-metrics: services \"submariner-lighthouse-agent-metrics\" already exists" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-19T04:15:38.4054976Z [cluster2] I0719 04:14:38.144040       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4059643Z [cluster2] I0719 04:14:38.145032       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4063522Z [cluster2] I0719 04:14:38.388678       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4067404Z [cluster2] I0719 04:14:38.438042       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4070648Z [cluster2] I0719 04:14:38.915835       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4074161Z [cluster2] I0719 04:14:39.246773       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4077847Z [cluster2] I0719 04:14:39.431612       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4081182Z [cluster2] I0719 04:14:39.752188       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4114371Z [cluster2] I0719 04:14:40.060877       1 servicediscovery_controller.go:487] controller_servicediscovery "msg"="coredns configmap does not have lighthouse configuration hence creating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4118763Z [cluster2] I0719 04:14:40.060933       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.15.237\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n"  
2021-07-19T04:15:38.4122529Z [cluster2] I0719 04:14:40.068842       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4126632Z [cluster2] I0719 04:14:40.368715       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4129876Z [cluster2] I0719 04:14:40.874108       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4133036Z [cluster2] I0719 04:14:40.898056       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4151284Z [cluster2] I0719 04:14:40.898123       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.15.237\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n"  
2021-07-19T04:15:38.4155278Z [cluster2] I0719 04:14:40.917282       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4175038Z [cluster2] I0719 04:14:40.958699       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4180804Z [cluster2] I0719 04:14:41.015590       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4204745Z [cluster2] I0719 04:14:41.523967       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4208346Z [cluster2] I0719 04:14:41.719929       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4211865Z [cluster2] I0719 04:14:41.823935       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4215646Z [cluster2] I0719 04:14:42.343525       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4218721Z [cluster2] I0719 04:14:42.405337       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4274984Z [cluster2] I0719 04:14:42.405407       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.15.237\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n"  
2021-07-19T04:15:38.4279025Z [cluster2] I0719 04:14:56.486249       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4280622Z [cluster2] I0719 04:14:56.486300       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4282083Z [cluster2] I0719 04:14:56.486326       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4285014Z [cluster2] I0719 04:14:56.720365       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4289423Z [cluster2] I0719 04:14:56.917543       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4292526Z [cluster2] I0719 04:14:57.433474       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4295791Z [cluster2] I0719 04:14:57.498807       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4299091Z [cluster2] I0719 04:14:57.517767       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4302151Z [cluster2] I0719 04:14:57.534483       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4305099Z [cluster2] I0719 04:14:57.836268       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4309492Z [cluster2] I0719 04:14:57.962437       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4313493Z [cluster2] I0719 04:14:58.011711       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4316742Z [cluster2] I0719 04:14:58.073658       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4319203Z [cluster2] I0719 04:14:58.114300       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4320747Z [cluster2] I0719 04:14:58.114349       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4322262Z [cluster2] I0719 04:14:58.114360       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4324325Z [cluster2] I0719 04:14:58.123737       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4325815Z [cluster2] I0719 04:14:58.123778       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4327072Z [cluster2] I0719 04:14:58.123796       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4329295Z [cluster2] I0719 04:14:58.202773       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4393957Z [cluster2] I0719 04:14:58.203486       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4397164Z [cluster2] I0719 04:14:58.275959       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4400697Z [cluster2] I0719 04:14:58.276844       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4404349Z [cluster2] I0719 04:14:58.516815       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4407714Z [cluster2] I0719 04:14:58.545693       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4411599Z [cluster2] I0719 04:14:58.780112       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4414892Z [cluster2] I0719 04:14:58.781239       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4418544Z [cluster2] I0719 04:14:58.818215       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4421983Z [cluster2] I0719 04:14:58.881474       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4425107Z [cluster2] I0719 04:14:58.910304       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4431192Z [cluster2] I0719 04:14:58.947671       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4434459Z [cluster2] I0719 04:14:58.956703       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4437613Z [cluster2] I0719 04:14:58.976327       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4441710Z [cluster2] I0719 04:14:58.979101       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4444857Z [cluster2] I0719 04:14:59.321476       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4462736Z [cluster2] I0719 04:14:59.337877       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4467108Z [cluster2] I0719 04:14:59.337935       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.15.237\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n"  
2021-07-19T04:15:38.4471989Z [cluster2] I0719 04:14:59.488873       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4475763Z [cluster2] I0719 04:14:59.489615       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4481122Z [cluster2] E0719 04:14:59.523252       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4484193Z [cluster2] I0719 04:14:59.523435       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4496964Z [cluster2] I0719 04:14:59.523483       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4498314Z [cluster2] I0719 04:14:59.523604       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4501063Z [cluster2] I0719 04:14:59.560565       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4504171Z [cluster2] I0719 04:14:59.604944       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4506974Z [cluster2] I0719 04:14:59.802625       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4510324Z [cluster2] I0719 04:14:59.835698       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4513957Z [cluster2] I0719 04:14:59.877266       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4517090Z [cluster2] I0719 04:15:00.147770       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4520383Z [cluster2] I0719 04:15:00.166972       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4523267Z [cluster2] I0719 04:15:00.193428       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4526391Z [cluster2] I0719 04:15:00.221817       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4529955Z [cluster2] I0719 04:15:00.380386       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4533384Z [cluster2] I0719 04:15:00.404458       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4537251Z [cluster2] I0719 04:15:00.423117       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4540470Z [cluster2] I0719 04:15:00.727693       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4543539Z [cluster2] I0719 04:15:00.927122       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4546535Z [cluster2] I0719 04:15:00.930398       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4551219Z [cluster2] I0719 04:15:00.930632       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.15.237\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n"  
2021-07-19T04:15:38.4555216Z [cluster2] I0719 04:15:02.269215       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4557399Z [cluster2] I0719 04:15:02.269272       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4558851Z [cluster2] I0719 04:15:02.269284       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4561893Z [cluster2] I0719 04:15:02.339087       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4564935Z [cluster2] I0719 04:15:02.394525       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4567717Z [cluster2] I0719 04:15:02.896348       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4570927Z [cluster2] I0719 04:15:02.918523       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4574269Z [cluster2] I0719 04:15:02.950590       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4577548Z [cluster2] I0719 04:15:02.964067       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4580311Z [cluster2] I0719 04:15:03.267242       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4583709Z [cluster2] I0719 04:15:03.302823       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4587239Z [cluster2] I0719 04:15:03.315137       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4590640Z [cluster2] I0719 04:15:03.468231       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4593200Z [cluster2] I0719 04:15:03.498681       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4594710Z [cluster2] I0719 04:15:03.498924       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4595957Z [cluster2] I0719 04:15:03.499245       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4597747Z [cluster2] I0719 04:15:03.507436       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4599249Z [cluster2] I0719 04:15:03.507606       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4600798Z [cluster2] I0719 04:15:03.507698       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4603136Z [cluster2] I0719 04:15:03.615399       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4606576Z [cluster2] I0719 04:15:03.626554       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4609416Z [cluster2] I0719 04:15:03.692815       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4612718Z [cluster2] I0719 04:15:03.717841       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4656442Z [cluster2] I0719 04:15:03.816690       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4660940Z [cluster2] I0719 04:15:03.823979       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-19T04:15:38.4665030Z [cluster2] I0719 04:15:03.840778       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4669488Z [cluster2] I0719 04:15:04.128024       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4672924Z [cluster2] I0719 04:15:04.134028       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4676117Z [cluster2] I0719 04:15:04.138696       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4679244Z [cluster2] I0719 04:15:04.142993       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4682753Z [cluster2] I0719 04:15:04.218905       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4686000Z [cluster2] I0719 04:15:04.225676       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4689533Z [cluster2] I0719 04:15:04.235603       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4693278Z [cluster2] I0719 04:15:04.246351       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4696648Z [cluster2] I0719 04:15:04.342145       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4699539Z [cluster2] I0719 04:15:04.344835       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4704406Z [cluster2] I0719 04:15:04.344934       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.0.15.237\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf {\n       max_concurrent 1000\n    }\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n"  
2021-07-19T04:15:38.4708114Z [cluster2] I0719 04:15:04.645155       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4711262Z [cluster2] I0719 04:15:04.748164       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4713933Z [cluster2] I0719 04:15:07.411364       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4715443Z [cluster2] I0719 04:15:07.411437       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4716691Z [cluster2] I0719 04:15:07.411449       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4718927Z [cluster2] I0719 04:15:07.432995       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4722047Z [cluster2] I0719 04:15:07.448891       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4725130Z [cluster2] I0719 04:15:07.951956       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4728165Z [cluster2] I0719 04:15:07.960645       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4731889Z [cluster2] I0719 04:15:07.967092       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4735474Z [cluster2] I0719 04:15:07.972822       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4763817Z [cluster2] I0719 04:15:08.482098       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4766544Z [cluster2] I0719 04:15:08.512011       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4768214Z [cluster2] I0719 04:15:08.512181       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4769459Z [cluster2] I0719 04:15:08.512224       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4771971Z [cluster2] I0719 04:15:08.512892       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4774434Z [cluster2] I0719 04:15:08.513081       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4802300Z [cluster2] I0719 04:15:08.513100       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4804998Z [cluster2] I0719 04:15:08.545890       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4808366Z [cluster2] I0719 04:15:08.620447       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4811263Z [cluster2] I0719 04:15:08.705878       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4814206Z [cluster2] I0719 04:15:08.705980       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4817380Z [cluster2] I0719 04:15:09.209153       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4820291Z [cluster2] I0719 04:15:09.216642       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4824231Z [cluster2] I0719 04:15:09.220160       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4827058Z [cluster2] I0719 04:15:09.225367       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4830060Z [cluster2] I0719 04:15:09.231850       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4833285Z [cluster2] I0719 04:15:09.238788       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4837016Z [cluster2] I0719 04:15:09.240482       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4840298Z [cluster2] I0719 04:15:09.246310       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4843871Z [cluster2] I0719 04:15:09.744509       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4846870Z [cluster2] I0719 04:15:09.748428       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4849214Z [cluster2] I0719 04:15:12.449285       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4851119Z [cluster2] I0719 04:15:12.449355       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4852461Z [cluster2] I0719 04:15:12.449365       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4856992Z [cluster2] I0719 04:15:12.463662       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4860355Z [cluster2] I0719 04:15:12.466983       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4864170Z [cluster2] I0719 04:15:12.968538       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4867744Z [cluster2] I0719 04:15:12.976671       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4871139Z [cluster2] I0719 04:15:12.982060       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4874391Z [cluster2] I0719 04:15:12.990798       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4877466Z [cluster2] I0719 04:15:13.493736       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4879923Z [cluster2] I0719 04:15:13.507397       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4882045Z [cluster2] I0719 04:15:13.507457       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4883432Z [cluster2] I0719 04:15:13.507468       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4885619Z [cluster2] I0719 04:15:13.514698       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4887353Z [cluster2] I0719 04:15:13.514746       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4888698Z [cluster2] I0719 04:15:13.514757       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4891515Z [cluster2] I0719 04:15:13.576555       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4938618Z [cluster2] I0719 04:15:13.576799       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4941786Z [cluster2] I0719 04:15:13.590896       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4944944Z [cluster2] I0719 04:15:13.593892       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4948029Z [cluster2] I0719 04:15:14.093713       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4951036Z [cluster2] I0719 04:15:14.097675       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4954364Z [cluster2] I0719 04:15:14.110064       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4959226Z [cluster2] I0719 04:15:14.114860       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4962613Z [cluster2] I0719 04:15:14.131898       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4965946Z [cluster2] I0719 04:15:14.135585       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.4969100Z [cluster2] I0719 04:15:14.170336       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4972297Z [cluster2] I0719 04:15:14.207709       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.4975462Z [cluster2] I0719 04:15:14.672040       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4991480Z [cluster2] I0719 04:15:14.710143       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.4994217Z [cluster2] I0719 04:15:17.500756       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.4995934Z [cluster2] I0719 04:15:17.500834       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.4997197Z [cluster2] I0719 04:15:17.500952       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.4999450Z [cluster2] I0719 04:15:17.529469       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5002931Z [cluster2] I0719 04:15:17.537122       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5006049Z [cluster2] I0719 04:15:18.039385       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5009438Z [cluster2] I0719 04:15:18.050262       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5012428Z [cluster2] I0719 04:15:18.057773       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5015673Z [cluster2] I0719 04:15:18.069387       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5018735Z [cluster2] I0719 04:15:18.571146       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5021326Z [cluster2] I0719 04:15:18.591166       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5022812Z [cluster2] I0719 04:15:18.591222       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5024073Z [cluster2] I0719 04:15:18.591233       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5025855Z [cluster2] I0719 04:15:18.593873       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5027350Z [cluster2] I0719 04:15:18.593914       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5028601Z [cluster2] I0719 04:15:18.593931       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5030921Z [cluster2] I0719 04:15:18.607115       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5033865Z [cluster2] I0719 04:15:18.617793       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5036861Z [cluster2] I0719 04:15:18.619251       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5039664Z [cluster2] I0719 04:15:18.634200       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5042440Z [cluster2] I0719 04:15:19.119986       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5045757Z [cluster2] I0719 04:15:19.134112       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5049042Z [cluster2] I0719 04:15:19.141770       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5051777Z [cluster2] I0719 04:15:19.157393       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5054571Z [cluster2] I0719 04:15:19.157673       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5098138Z [cluster2] I0719 04:15:19.163060       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5100858Z [cluster2] I0719 04:15:19.175330       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5103523Z [cluster2] I0719 04:15:19.183283       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5107145Z [cluster2] I0719 04:15:19.664846       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5110506Z [cluster2] I0719 04:15:19.685367       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5112781Z [cluster2] I0719 04:15:22.539060       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5114199Z [cluster2] I0719 04:15:22.539136       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5115597Z [cluster2] I0719 04:15:22.539148       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5118187Z [cluster2] I0719 04:15:22.548454       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5120826Z [cluster2] I0719 04:15:22.553164       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5123447Z [cluster2] I0719 04:15:23.055241       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5126738Z [cluster2] I0719 04:15:23.061686       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5129604Z [cluster2] I0719 04:15:23.071506       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5132270Z [cluster2] I0719 04:15:23.080380       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5135322Z [cluster2] I0719 04:15:23.583773       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5138731Z [cluster2] I0719 04:15:23.596937       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5140438Z [cluster2] I0719 04:15:23.597032       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5141765Z [cluster2] I0719 04:15:23.597069       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5143917Z [cluster2] I0719 04:15:23.612861       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5145901Z [cluster2] I0719 04:15:23.612915       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5147352Z [cluster2] I0719 04:15:23.612989       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5150191Z [cluster2] I0719 04:15:23.623347       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5153677Z [cluster2] I0719 04:15:23.628926       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5157652Z [cluster2] I0719 04:15:23.660808       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5162164Z [cluster2] I0719 04:15:23.721990       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5166482Z [cluster2] I0719 04:15:24.162048       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5169578Z [cluster2] I0719 04:15:24.169981       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5173174Z [cluster2] I0719 04:15:24.177614       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5177184Z [cluster2] I0719 04:15:24.182664       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5180123Z [cluster2] I0719 04:15:24.223751       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5183290Z [cluster2] I0719 04:15:24.235389       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5256709Z [cluster2] I0719 04:15:24.246174       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5263912Z [cluster2] I0719 04:15:24.255869       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5267115Z [cluster2] I0719 04:15:24.685085       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5270382Z [cluster2] I0719 04:15:24.758743       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5272830Z [cluster2] I0719 04:15:27.568962       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5274677Z [cluster2] I0719 04:15:27.569046       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5276333Z [cluster2] I0719 04:15:27.569060       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5278982Z [cluster2] I0719 04:15:27.583256       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5281920Z [cluster2] I0719 04:15:27.589172       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5285000Z [cluster2] I0719 04:15:28.091304       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5288627Z [cluster2] I0719 04:15:28.099445       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5291670Z [cluster2] I0719 04:15:28.108490       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5294880Z [cluster2] I0719 04:15:28.129519       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5298794Z [cluster2] I0719 04:15:28.631204       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5301467Z [cluster2] I0719 04:15:28.644101       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5302932Z [cluster2] I0719 04:15:28.644650       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5304496Z [cluster2] I0719 04:15:28.644809       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5307203Z [cluster2] I0719 04:15:28.648806       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5309300Z [cluster2] I0719 04:15:28.648850       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5310536Z [cluster2] I0719 04:15:28.648860       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5313556Z [cluster2] I0719 04:15:28.669439       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5316841Z [cluster2] I0719 04:15:28.676327       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5319702Z [cluster2] I0719 04:15:28.678261       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5322587Z [cluster2] I0719 04:15:28.696292       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5325459Z [cluster2] I0719 04:15:29.178441       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5328296Z [cluster2] I0719 04:15:29.202229       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5354732Z [cluster2] I0719 04:15:29.212784       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5358225Z [cluster2] I0719 04:15:29.425999       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5362629Z [cluster2] I0719 04:15:29.428247       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5366406Z [cluster2] I0719 04:15:29.536833       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5370130Z [cluster2] I0719 04:15:29.545233       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5383067Z [cluster2] I0719 04:15:29.582853       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5386319Z [cluster2] I0719 04:15:30.047008       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5389489Z [cluster2] I0719 04:15:30.085684       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5391977Z [cluster2] I0719 04:15:32.618417       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5393574Z [cluster2] I0719 04:15:32.618472       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5394776Z [cluster2] I0719 04:15:32.618482       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5397263Z [cluster2] I0719 04:15:32.634987       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5401105Z [cluster2] I0719 04:15:32.640817       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5444523Z [cluster2] I0719 04:15:33.143306       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5448131Z [cluster2] I0719 04:15:33.160152       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5451301Z [cluster2] I0719 04:15:33.178650       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5454681Z [cluster2] I0719 04:15:33.187054       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5458431Z [cluster2] I0719 04:15:33.689406       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5461419Z [cluster2] I0719 04:15:33.706971       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5465586Z [cluster2] I0719 04:15:33.707148       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5466982Z [cluster2] I0719 04:15:33.707282       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5469863Z [cluster2] I0719 04:15:33.719034       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5472558Z [cluster2] I0719 04:15:33.723345       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5474030Z [cluster2] I0719 04:15:33.723480       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5475381Z [cluster2] I0719 04:15:33.723532       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5477987Z [cluster2] I0719 04:15:33.743474       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5481424Z [cluster2] I0719 04:15:33.749027       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5484124Z [cluster2] I0719 04:15:33.781521       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5488158Z [cluster2] I0719 04:15:34.250922       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5491203Z [cluster2] I0719 04:15:34.268128       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5494070Z [cluster2] I0719 04:15:34.276454       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5545516Z [cluster2] I0719 04:15:34.282971       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5549376Z [cluster2] I0719 04:15:34.292429       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5552701Z [cluster2] I0719 04:15:34.315119       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5555996Z [cluster2] I0719 04:15:34.320496       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5560147Z [cluster2] I0719 04:15:34.324382       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5563986Z [cluster2] I0719 04:15:34.794102       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5567592Z [cluster2] I0719 04:15:34.826467       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.5570309Z [cluster2] I0719 04:15:37.682104       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-19T04:15:38.5571824Z [cluster2] I0719 04:15:37.685942       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.0.0.0/16" "type"="Cluster"
2021-07-19T04:15:38.5573084Z [cluster2] I0719 04:15:37.685971       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.0.0.0/16" "type"="Service"
2021-07-19T04:15:38.5575365Z [cluster2] I0719 04:15:37.710846       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.5579150Z [cluster2] I0719 04:15:37.717478       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.5582016Z [cluster2] I0719 04:15:38.219495       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.7437119Z [cluster2] I0719 04:15ting Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.7439943Z [cluster1] I0719 04:15:34.453998       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.7443131Z [cluster1] I0719 04:15:34.455057       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.7446455Z [cluster1] I0719 04:15:34.465097       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.7449154Z [cluster1] I0719 04:15:34.467360       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.7452362Z [cluster1] I0719 04:15:34.471237       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.7455869Z [cluster1] I0719 04:15:34.474183       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:38.7458588Z [cluster1] I0719 04:15:34.482946       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.7462198Z [cluster1] I0719 04:15:34.510550       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:38.7465357Z [cluster1] I0719 04:15:34.983963       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.7467760Z [cluster1] I0719 04:15:35.012406       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-19T04:15:38.7469251Z [submariner-operator]$ [cluster1] verify_subm_cr
2021-07-19T04:15:38.7470016Z [submariner-operator]$ [cluster1] verify_subm_cr
2021-07-19T04:15:38.7471159Z [submariner-operator]$ [cluster1] kubectl get submariner --namespace=submariner-operator
2021-07-19T04:15:38.7472436Z [submariner-operator]$ [cluster1] grep submariner
2021-07-19T04:15:38.7473560Z [submariner-operator]$ [cluster1] kubectl get submariner --namespace=submariner-operator
2021-07-19T04:15:38.7475444Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner --namespace=submariner-operator
2021-07-19T04:15:38.7476485Z [cluster1] submariner   89s
2021-07-19T04:15:38.7477696Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner.cluster1.json
2021-07-19T04:15:38.7479371Z [submariner-operator]$ [cluster1] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:38.7483676Z [submariner-operator]$ [cluster1] tee /tmp/submariner.cluster1.json
2021-07-19T04:15:38.7486270Z [submariner-operator]$ [cluster1] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:38.7487807Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:38.7488669Z [cluster1] {
2021-07-19T04:15:38.7489167Z [cluster1]     "apiVersion": "submariner.io/v1alpha1",
2021-07-19T04:15:38.7489712Z [cluster1]     "kind": "Submariner",
2021-07-19T04:15:38.7490160Z [cluster1]     "metadata": {
2021-07-19T04:15:38.7491737Z [cluster1]         "creationTimestamp": "2021-07-19T04:14:09Z",
2021-07-19T04:15:38.7492327Z [cluster1]         "generation": 1,
2021-07-19T04:15:38.7492847Z [cluster1]         "managedFields": [
2021-07-19T04:15:38.7493302Z [cluster1]             {
2021-07-19T04:15:38.7494083Z [cluster1]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-19T04:15:38.7495285Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:38.7495897Z [cluster1]                 "fieldsV1": {
2021-07-19T04:15:38.7496297Z [cluster1]                     "f:spec": {
2021-07-19T04:15:38.7496846Z [cluster1]                         ".": {},
2021-07-19T04:15:38.7497441Z [cluster1]                         "f:broker": {},
2021-07-19T04:15:38.7498215Z [cluster1]                         "f:brokerK8sApiServer": {},
2021-07-19T04:15:38.7498870Z [cluster1]                         "f:brokerK8sApiServerToken": {},
2021-07-19T04:15:38.7499629Z [cluster1]                         "f:brokerK8sCA": {},
2021-07-19T04:15:38.7500232Z [cluster1]                         "f:brokerK8sRemoteNamespace": {},
2021-07-19T04:15:38.7501147Z [cluster1]                         "f:cableDriver": {},
2021-07-19T04:15:38.7502031Z [cluster1]                         "f:ceIPSecDebug": {},
2021-07-19T04:15:38.7502579Z [cluster1]                         "f:ceIPSecIKEPort": {},
2021-07-19T04:15:38.7503345Z [cluster1]                         "f:ceIPSecNATTPort": {},
2021-07-19T04:15:38.7504072Z [cluster1]                         "f:ceIPSecPSK": {},
2021-07-19T04:15:38.7504618Z [cluster1]                         "f:clusterCIDR": {},
2021-07-19T04:15:38.7505673Z [cluster1]                         "f:clusterID": {},
2021-07-19T04:15:38.7506484Z [cluster1]                         "f:colorCodes": {},
2021-07-19T04:15:38.7507431Z [cluster1]                         "f:connectionHealthCheck": {
2021-07-19T04:15:38.7508095Z [cluster1]                             ".": {},
2021-07-19T04:15:38.7508635Z [cluster1]                             "f:enabled": {},
2021-07-19T04:15:38.7509121Z [cluster1]                             "f:intervalSeconds": {},
2021-07-19T04:15:38.7509850Z [cluster1]                             "f:maxPacketLossCount": {}
2021-07-19T04:15:38.7510306Z [cluster1]                         },
2021-07-19T04:15:38.7510693Z [cluster1]                         "f:debug": {},
2021-07-19T04:15:38.7511444Z [cluster1]                         "f:globalCIDR": {},
2021-07-19T04:15:38.7512111Z [cluster1]                         "f:namespace": {},
2021-07-19T04:15:38.7512603Z [cluster1]                         "f:natEnabled": {},
2021-07-19T04:15:38.7513081Z [cluster1]                         "f:repository": {},
2021-07-19T04:15:38.7513775Z [cluster1]                         "f:serviceCIDR": {},
2021-07-19T04:15:38.7514414Z [cluster1]                         "f:serviceDiscoveryEnabled": {},
2021-07-19T04:15:38.7515520Z [cluster1]                         "f:version": {}
2021-07-19T04:15:38.7515886Z [cluster1]                     }
2021-07-19T04:15:38.7516230Z [cluster1]                 },
2021-07-19T04:15:38.7516606Z [cluster1]                 "manager": "subctl",
2021-07-19T04:15:38.7517066Z [cluster1]                 "operation": "Update",
2021-07-19T04:15:38.7517751Z [cluster1]                 "time": "2021-07-19T04:14:09Z"
2021-07-19T04:15:38.7518241Z [cluster1]             },
2021-07-19T04:15:38.7518578Z [cluster1]             {
2021-07-19T04:15:38.7519047Z [cluster1]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-19T04:15:38.7519614Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:38.7525118Z [cluster1]                 "fieldsV1": {
2021-07-19T04:15:38.7525839Z [cluster1]                     "f:status": {
2021-07-19T04:15:38.7526220Z [cluster1]                         ".": {},
2021-07-19T04:15:38.7526639Z [cluster1]                         "f:clusterCIDR": {},
2021-07-19T04:15:38.7527084Z [cluster1]                         "f:clusterID": {},
2021-07-19T04:15:38.7527536Z [cluster1]                         "f:colorCodes": {},
2021-07-19T04:15:38.7528110Z [cluster1]                         "f:deploymentInfo": {},
2021-07-19T04:15:38.7528692Z [cluster1]                         "f:gatewayDaemonSetStatus": {
2021-07-19T04:15:38.7529236Z [cluster1]                             ".": {},
2021-07-19T04:15:38.7529722Z [cluster1]                             "f:lastResourceVersion": {},
2021-07-19T04:15:38.7530394Z [cluster1]                             "f:mismatchedContainerImages": {},
2021-07-19T04:15:38.7531584Z [cluster1]                             "f:nonReadyContainerStates": {},
2021-07-19T04:15:38.7532208Z [cluster1]                             "f:status": {
2021-07-19T04:15:38.7532640Z [cluster1]                                 ".": {},
2021-07-19T04:15:38.7533417Z [cluster1]                                 "f:currentNumberScheduled": {},
2021-07-19T04:15:38.7534210Z [cluster1]                                 "f:desiredNumberScheduled": {},
2021-07-19T04:15:38.7535422Z [cluster1]                                 "f:numberAvailable": {},
2021-07-19T04:15:38.7564041Z [cluster1]                                 "f:numberMisscheduled": {},
2021-07-19T04:15:38.7564958Z [cluster1]                                 "f:numberReady": {},
2021-07-19T04:15:38.7565526Z [cluster1]                                 "f:observedGeneration": {},
2021-07-19T04:15:38.7566165Z [cluster1]                                 "f:updatedNumberScheduled": {}
2021-07-19T04:15:38.7566697Z [cluster1]                             }
2021-07-19T04:15:38.7567066Z [cluster1]                         },
2021-07-19T04:15:38.7567460Z [cluster1]                         "f:gateways": {},
2021-07-19T04:15:38.7567920Z [cluster1]                         "f:globalCIDR": {},
2021-07-19T04:15:38.7568663Z [cluster1]                         "f:globalnetDaemonSetStatus": {
2021-07-19T04:15:38.7569240Z [cluster1]                             ".": {},
2021-07-19T04:15:38.7569723Z [cluster1]                             "f:lastResourceVersion": {},
2021-07-19T04:15:38.7570410Z [cluster1]                             "f:mismatchedContainerImages": {},
2021-07-19T04:15:38.7571123Z [cluster1]                             "f:nonReadyContainerStates": {},
2021-07-19T04:15:38.7572065Z [cluster1]                             "f:status": {
2021-07-19T04:15:38.7572672Z [cluster1]                                 ".": {},
2021-07-19T04:15:38.7573391Z [cluster1]                                 "f:currentNumberScheduled": {},
2021-07-19T04:15:38.7574101Z [cluster1]                                 "f:desiredNumberScheduled": {},
2021-07-19T04:15:38.7574899Z [cluster1]                                 "f:numberAvailable": {},
2021-07-19T04:15:38.7575480Z [cluster1]                                 "f:numberMisscheduled": {},
2021-07-19T04:15:38.7576622Z [cluster1]                                 "f:numberReady": {},
2021-07-19T04:15:38.7577181Z [cluster1]                                 "f:observedGeneration": {},
2021-07-19T04:15:38.7577818Z [cluster1]                                 "f:updatedNumberScheduled": {}
2021-07-19T04:15:38.7578354Z [cluster1]                             }
2021-07-19T04:15:38.7578723Z [cluster1]                         },
2021-07-19T04:15:38.7579192Z [cluster1]                         "f:loadBalancerStatus": {},
2021-07-19T04:15:38.7579841Z [cluster1]                         "f:natEnabled": {},
2021-07-19T04:15:38.7580322Z [cluster1]                         "f:networkPlugin": {},
2021-07-19T04:15:38.7581131Z [cluster1]                         "f:routeAgentDaemonSetStatus": {
2021-07-19T04:15:38.7582966Z [cluster1]                             ".": {},
2021-07-19T04:15:38.7583594Z [cluster1]                             "f:lastResourceVersion": {},
2021-07-19T04:15:38.7584423Z [cluster1]                             "f:mismatchedContainerImages": {},
2021-07-19T04:15:38.7585316Z [cluster1]                             "f:nonReadyContainerStates": {},
2021-07-19T04:15:38.7586287Z [cluster1]                             "f:status": {
2021-07-19T04:15:38.7586689Z [cluster1]                                 ".": {},
2021-07-19T04:15:38.7587244Z [cluster1]                                 "f:currentNumberScheduled": {},
2021-07-19T04:15:38.7587942Z [cluster1]                                 "f:desiredNumberScheduled": {},
2021-07-19T04:15:38.7588590Z [cluster1]                                 "f:numberAvailable": {},
2021-07-19T04:15:38.7589173Z [cluster1]                                 "f:numberMisscheduled": {},
2021-07-19T04:15:38.7589737Z [cluster1]                                 "f:numberReady": {},
2021-07-19T04:15:38.7590307Z [cluster1]                                 "f:observedGeneration": {},
2021-07-19T04:15:38.7590958Z [cluster1]                                 "f:updatedNumberScheduled": {}
2021-07-19T04:15:38.7591704Z [cluster1]                             }
2021-07-19T04:15:38.7592082Z [cluster1]                         },
2021-07-19T04:15:38.7592535Z [cluster1]                         "f:serviceCIDR": {}
2021-07-19T04:15:38.7592962Z [cluster1]                     }
2021-07-19T04:15:38.7593741Z [cluster1]                 },
2021-07-19T04:15:38.7594853Z [cluster1]                 "manager": "submariner-operator",
2021-07-19T04:15:38.7595449Z [cluster1]                 "operation": "Update",
2021-07-19T04:15:38.7596088Z [cluster1]                 "time": "2021-07-19T04:14:50Z"
2021-07-19T04:15:38.7596495Z [cluster1]             }
2021-07-19T04:15:38.7596849Z [cluster1]         ],
2021-07-19T04:15:38.7597247Z [cluster1]         "name": "submariner",
2021-07-19T04:15:38.7597989Z [cluster1]         "namespace": "submariner-operator",
2021-07-19T04:15:38.7598585Z [cluster1]         "resourceVersion": "2492",
2021-07-19T04:15:38.7599791Z [cluster1]         "selfLink": "/apis/submariner.io/v1alpha1/namespaces/submariner-operator/submariners/submariner",
2021-07-19T04:15:38.7636160Z [cluster1]         "uid": "93ce6b08-23f2-43d3-ac42-a68670d832f4"
2021-07-19T04:15:38.7636681Z [cluster1]     },
2021-07-19T04:15:38.7637027Z [cluster1]     "spec": {
2021-07-19T04:15:38.7637471Z [cluster1]         "broker": "k8s",
2021-07-19T04:15:38.7637982Z [cluster1]         "brokerK8sApiServer": "172.18.0.7:6443",
2021-07-19T04:15:38.7799702Z [cluster1]         "brokerK8sApiServerToken": "***",
2021-07-19T04:15:38.8426464Z [cluster1]         "brokerK8sCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==",
2021-07-19T04:15:38.9042922Z [cluster1]         "brokerK8sRemoteNamespace": "submariner-k8s-broker",
2021-07-19T04:15:38.9043792Z [cluster1]         "cableDriver": "libreswan",
2021-07-19T04:15:38.9044369Z [cluster1]         "ceIPSecDebug": false,
2021-07-19T04:15:38.9045144Z [cluster1]         "ceIPSecIKEPort": 500,
2021-07-19T04:15:38.9045708Z [cluster1]         "ceIPSecNATTPort": 4500,
2021-07-19T04:15:38.9046792Z [cluster1]         "ceIPSecPSK": "/lcX3t1f1LkWM2NxjfGKcDYjF5h8RcIAsg0peNU7/wCAZuvDKZBlCQBKlFRsYd+/",
2021-07-19T04:15:38.9048171Z [cluster1]         "clusterCIDR": "",
2021-07-19T04:15:38.9048707Z [cluster1]         "clusterID": "cluster1",
2021-07-19T04:15:38.9049215Z [cluster1]         "colorCodes": "blue",
2021-07-19T04:15:38.9049818Z [cluster1]         "connectionHealthCheck": {
2021-07-19T04:15:38.9050390Z [cluster1]             "enabled": true,
2021-07-19T04:15:38.9050916Z [cluster1]             "intervalSeconds": 1,
2021-07-19T04:15:38.9051499Z [cluster1]             "maxPacketLossCount": 5
2021-07-19T04:15:38.9051998Z [cluster1]         },
2021-07-19T04:15:38.9052581Z [cluster1]         "debug": false,
2021-07-19T04:15:38.9053037Z [cluster1]         "globalCIDR": "242.254.1.0/24",
2021-07-19T04:15:38.9053857Z [cluster1]         "namespace": "submariner-operator",
2021-07-19T04:15:38.9082971Z [cluster1]         "natEnabled": false,
2021-07-19T04:15:38.9083624Z [cluster1]         "repository": "localhost:5000",
2021-07-19T04:15:38.9084149Z [cluster1]         "serviceCIDR": "",
2021-07-19T04:15:38.9085432Z [cluster1]         "serviceDiscoveryEnabled": true,
2021-07-19T04:15:38.9086285Z [cluster1]         "version": "local"
2021-07-19T04:15:38.9086736Z [cluster1]     },
2021-07-19T04:15:38.9087138Z [cluster1]     "status": {
2021-07-19T04:15:38.9087641Z [cluster1]         "clusterCIDR": "10.0.0.0/16",
2021-07-19T04:15:38.9088186Z [cluster1]         "clusterID": "cluster1",
2021-07-19T04:15:38.9088894Z [cluster1]         "colorCodes": "blue",
2021-07-19T04:15:38.9089442Z [cluster1]         "deploymentInfo": {},
2021-07-19T04:15:38.9090081Z [cluster1]         "gatewayDaemonSetStatus": {
2021-07-19T04:15:38.9091079Z [cluster1]             "lastResourceVersion": "2117",
2021-07-19T04:15:38.9091859Z [cluster1]             "mismatchedContainerImages": false,
2021-07-19T04:15:38.9092669Z [cluster1]             "nonReadyContainerStates": [],
2021-07-19T04:15:38.9093261Z [cluster1]             "status": {
2021-07-19T04:15:38.9093862Z [cluster1]                 "currentNumberScheduled": 1,
2021-07-19T04:15:38.9094597Z [cluster1]                 "desiredNumberScheduled": 1,
2021-07-19T04:15:38.9095280Z [cluster1]                 "numberAvailable": 1,
2021-07-19T04:15:38.9097313Z [cluster1]                 "numberMisscheduled": 0,
2021-07-19T04:15:38.9097994Z [cluster1]                 "numberReady": 1,
2021-07-19T04:15:38.9098628Z [cluster1]                 "observedGeneration": 1,
2021-07-19T04:15:38.9099370Z [cluster1]                 "updatedNumberScheduled": 1
2021-07-19T04:15:38.9100124Z [cluster1]             }
2021-07-19T04:15:38.9100491Z [cluster1]         },
2021-07-19T04:15:38.9100912Z [cluster1]         "gateways": [
2021-07-19T04:15:38.9101314Z [cluster1]             {
2021-07-19T04:15:38.9101752Z [cluster1]                 "connections": [
2021-07-19T04:15:38.9102179Z [cluster1]                     {
2021-07-19T04:15:38.9102631Z [cluster1]                         "endpoint": {
2021-07-19T04:15:38.9103155Z [cluster1]                             "backend": "libreswan",
2021-07-19T04:15:38.9103683Z [cluster1]                             "backend_config": {
2021-07-19T04:15:38.9104637Z [cluster1]                                 "natt-discovery-port": "4490",
2021-07-19T04:15:38.9106477Z [cluster1]                                 "preferred-server": "false"
2021-07-19T04:15:38.9107082Z [cluster1]                             },
2021-07-19T04:15:38.9108156Z [cluster1]                             "cable_name": "submariner-cable-cluster2-172-18-0-9",
2021-07-19T04:15:38.9109116Z [cluster1]                             "cluster_id": "cluster2",
2021-07-19T04:15:38.9110084Z [cluster1]                             "hostname": "cluster2-worker",
2021-07-19T04:15:38.9110714Z [cluster1]                             "nat_enabled": false,
2021-07-19T04:15:38.9111258Z [cluster1]                             "private_ip": "172.18.0.9",
2021-07-19T04:15:38.9111772Z [cluster1]                             "public_ip": "40.124.48.108",
2021-07-19T04:15:38.9112275Z [cluster1]                             "subnets": [
2021-07-19T04:15:38.9112728Z [cluster1]                                 "242.254.2.0/24"
2021-07-19T04:15:39.1134303Z [cluster1]                             ]
2021-07-19T04:15:39.1134766Z [cluster1]                         },
2021-07-19T04:15:39.1135299Z [cluster1]                         "latencyRTT": {
2021-07-19T04:15:39.1143951Z [cluster1]                             "average": "245.985µs",
2021-07-19T04:15:39.1144861Z [cluster1]                             "last": "209.602µs",
2021-07-19T04:15:39.1145644Z [cluster1]                             "max": "466.705µs",
2021-07-19T04:15:39.1146308Z [cluster1]                             "min": "143.802µs",
2021-07-19T04:15:39.1146949Z [cluster1]                             "stdDev": "69.33µs"
2021-07-19T04:15:39.1147373Z [cluster1]                         },
2021-07-19T04:15:39.1148023Z [cluster1]                         "status": "connected",
2021-07-19T04:15:39.1148886Z [cluster1]                         "statusMessage": "",
2021-07-19T04:15:39.1149378Z [cluster1]                         "usingIP": "172.18.0.9"
2021-07-19T04:15:39.1149772Z [cluster1]                     }
2021-07-19T04:15:39.1150151Z [cluster1]                 ],
2021-07-19T04:15:39.1150566Z [cluster1]                 "haStatus": "active",
2021-07-19T04:15:39.1151231Z [cluster1]                 "localEndpoint": {
2021-07-19T04:15:39.1151754Z [cluster1]                     "backend": "libreswan",
2021-07-19T04:15:39.1152252Z [cluster1]                     "backend_config": {
2021-07-19T04:15:39.1153013Z [cluster1]                         "natt-discovery-port": "4490",
2021-07-19T04:15:39.1154003Z [cluster1]                         "preferred-server": "false"
2021-07-19T04:15:39.1154542Z [cluster1]                     },
2021-07-19T04:15:39.1155912Z [cluster1]                     "cable_name": "submariner-cable-cluster1-172-18-0-8",
2021-07-19T04:15:39.1156817Z [cluster1]                     "cluster_id": "cluster1",
2021-07-19T04:15:39.1157628Z [cluster1]                     "hostname": "cluster1-worker",
2021-07-19T04:15:39.1158242Z [cluster1]                     "nat_enabled": false,
2021-07-19T04:15:39.1159109Z [cluster1]                     "private_ip": "172.18.0.8",
2021-07-19T04:15:39.1159572Z [cluster1]                     "public_ip": "40.124.48.108",
2021-07-19T04:15:39.1160031Z [cluster1]                     "subnets": [
2021-07-19T04:15:39.1160441Z [cluster1]                         "242.254.1.0/24"
2021-07-19T04:15:39.1160833Z [cluster1]                     ]
2021-07-19T04:15:39.1161190Z [cluster1]                 },
2021-07-19T04:15:39.1161640Z [cluster1]                 "statusFailure": "",
2021-07-19T04:15:39.1162355Z [cluster1]                 "version": "devel-1441a80"
2021-07-19T04:15:39.1162814Z [cluster1]             }
2021-07-19T04:15:39.1163180Z [cluster1]         ],
2021-07-19T04:15:39.1163591Z [cluster1]         "globalCIDR": "242.254.1.0/24",
2021-07-19T04:15:39.1164206Z [cluster1]         "globalnetDaemonSetStatus": {
2021-07-19T04:15:39.1164996Z [cluster1]             "lastResourceVersion": "2248",
2021-07-19T04:15:39.1165952Z [cluster1]             "mismatchedContainerImages": false,
2021-07-19T04:15:39.1166868Z [cluster1]             "nonReadyContainerStates": [],
2021-07-19T04:15:39.1167608Z [cluster1]             "status": {
2021-07-19T04:15:39.1168332Z [cluster1]                 "currentNumberScheduled": 1,
2021-07-19T04:15:39.1169195Z [cluster1]                 "desiredNumberScheduled": 1,
2021-07-19T04:15:39.1169807Z [cluster1]                 "numberAvailable": 1,
2021-07-19T04:15:39.1170396Z [cluster1]                 "numberMisscheduled": 0,
2021-07-19T04:15:39.1170952Z [cluster1]                 "numberReady": 1,
2021-07-19T04:15:39.1171491Z [cluster1]                 "observedGeneration": 1,
2021-07-19T04:15:39.1172151Z [cluster1]                 "updatedNumberScheduled": 1
2021-07-19T04:15:39.1172670Z [cluster1]             }
2021-07-19T04:15:39.1173034Z [cluster1]         },
2021-07-19T04:15:39.1173500Z [cluster1]         "loadBalancerStatus": {},
2021-07-19T04:15:39.1174046Z [cluster1]         "natEnabled": false,
2021-07-19T04:15:39.1174807Z [cluster1]         "networkPlugin": "weave-net",
2021-07-19T04:15:39.1176577Z [cluster1]         "routeAgentDaemonSetStatus": {
2021-07-19T04:15:39.1177290Z [cluster1]             "lastResourceVersion": "2218",
2021-07-19T04:15:39.1178036Z [cluster1]             "mismatchedContainerImages": false,
2021-07-19T04:15:39.1179007Z [cluster1]             "nonReadyContainerStates": [],
2021-07-19T04:15:39.1179560Z [cluster1]             "status": {
2021-07-19T04:15:39.1180121Z [cluster1]                 "currentNumberScheduled": 3,
2021-07-19T04:15:39.1180810Z [cluster1]                 "desiredNumberScheduled": 3,
2021-07-19T04:15:39.1181440Z [cluster1]                 "numberAvailable": 3,
2021-07-19T04:15:39.1182008Z [cluster1]                 "numberMisscheduled": 0,
2021-07-19T04:15:39.1182561Z [cluster1]                 "numberReady": 3,
2021-07-19T04:15:39.1183102Z [cluster1]                 "observedGeneration": 1,
2021-07-19T04:15:39.1183761Z [cluster1]                 "updatedNumberScheduled": 3
2021-07-19T04:15:39.1184294Z [cluster1]             }
2021-07-19T04:15:39.1184649Z [cluster1]         },
2021-07-19T04:15:39.1185084Z [cluster1]         "serviceCIDR": "100.0.0.0/16"
2021-07-19T04:15:39.1185494Z [cluster1]     }
2021-07-19T04:15:39.1186163Z [cluster1] }
2021-07-19T04:15:39.1187327Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-operator
2021-07-19T04:15:39.1189324Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-operator
2021-07-19T04:15:39.1191083Z [submariner-operator]$ [cluster1] jq -r -M .metadata.namespace /tmp/submariner.cluster1.json
2021-07-19T04:15:39.1192691Z [submariner-operator]$ [cluster1] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-19T04:15:39.1194053Z [submariner-operator]$ [cluster1] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-19T04:15:39.1195376Z [submariner-operator]$ [cluster1] jq -r -M .apiVersion /tmp/submariner.cluster1.json
2021-07-19T04:15:39.1196578Z [submariner-operator]$ [cluster1] validate_equals .kind Submariner
2021-07-19T04:15:39.1197629Z [submariner-operator]$ [cluster1] validate_equals .kind Submariner
2021-07-19T04:15:39.4513666Z [submariner-operator]$ [cluster1] jq -r -M .kind /tmp/submariner.cluster1.json
2021-07-19T04:15:39.4514917Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner
2021-07-19T04:15:39.4516060Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner
2021-07-19T04:15:39.4519019Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariner.cluster1.json
2021-07-19T04:15:39.4520976Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sApiServer 172.18.0.7:6443
2021-07-19T04:15:39.4523031Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sApiServer 172.18.0.7:6443
2021-07-19T04:15:39.4525221Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sApiServer /tmp/submariner.cluster1.json
2021-07-19T04:15:39.4692707Z [submariner-operator]$ [cluster1] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-19T04:15:39.4890750Z [submariner-operator]$ [cluster1] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-19T04:15:39.4892636Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sApiServerToken /tmp/submariner.cluster1.json
2021-07-19T04:15:39.6477052Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2:38.230682       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:39.6843914Z [cluster2] I0719 04:15:38.239855       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-globalnet" "DaemonSet.Namespace"="submariner-operator"
2021-07-19T04:15:39.6847063Z [cluster2] I0719 04:15:38.247583       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-globalnet-metrics" "Service.Namespace"="submariner-operator"
2021-07-19T04:15:39.6848995Z [submariner-operator]$ [cluster2] verify_subm_cr
2021-07-19T04:15:39.6850172Z [submariner-operator]$ [cluster2] verify_subm_cr
2021-07-19T04:15:39.6851454Z [submariner-operator]$ [cluster2] kubectl get submariner --namespace=submariner-operator
2021-07-19T04:15:39.6852614Z [submariner-operator]$ [cluster2] grep submariner
2021-07-19T04:15:39.6853987Z [submariner-operator]$ [cluster2] kubectl get submariner --namespace=submariner-operator
2021-07-19T04:15:39.6855469Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner --namespace=submariner-operator
2021-07-19T04:15:39.6864594Z [cluster2] submariner   82s
2021-07-19T04:15:39.6865737Z [submariner-operator]$ [cluster2] json_file=/tmp/submariner.cluster2.json
2021-07-19T04:15:39.6867170Z [submariner-operator]$ [cluster2] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:39.6868669Z [submariner-operator]$ [cluster2] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:39.6870313Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:39.6871751Z [submariner-operator]$ [cluster2] tee /tmp/submariner.cluster2.json
2021-07-19T04:15:39.6872460Z [cluster2] {
2021-07-19T04:15:39.6872974Z [cluster2]     "apiVersion": "submariner.io/v1alpha1",
2021-07-19T04:15:39.6873978Z [cluster2]     "kind": "Submariner",
2021-07-19T04:15:39.6874619Z [cluster2]     "metadata": {
2021-07-19T04:15:39.6875650Z [cluster2]         "creationTimestamp": "2021-07-19T04:14:17Z",
2021-07-19T04:15:39.6876740Z [cluster2]         "generation": 1,
2021-07-19T04:15:39.6877375Z [cluster2]         "managedFields": [
2021-07-19T04:15:39.6879562Z [cluster2]             {
2021-07-19T04:15:39.6880130Z [cluster2]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-19T04:15:39.6880954Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:39.6881458Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:39.6881924Z [cluster2]                     "f:spec": {
2021-07-19T04:15:39.6882855Z [cluster2]                         ".": {},
2021-07-19T04:15:39.6883513Z [cluster2]                         "f:broker": {},
2021-07-19T04:15:39.6884069Z [cluster2]                         "f:brokerK8sApiServer": {},
2021-07-19T04:15:39.6885586Z [cluster2]                         "f:brokerK8sApiServerToken": {},
2021-07-19T04:15:39.6887807Z [cluster2]                         "f:brokerK8sCA": {},
2021-07-19T04:15:39.6888606Z [cluster2]                         "f:brokerK8sRemoteNamespace": {},
2021-07-19T04:15:39.6889346Z [cluster2]                         "f:cableDriver": {},
2021-07-19T04:15:39.6889937Z [cluster2]                         "f:ceIPSecDebug": {},
2021-07-19T04:15:39.6890873Z [cluster2]                         "f:ceIPSecIKEPort": {},
2021-07-19T04:15:39.6891845Z [cluster2]                         "f:ceIPSecNATTPort": {},
2021-07-19T04:15:39.6892453Z [cluster2]                         "f:ceIPSecPSK": {},
2021-07-19T04:15:39.6893330Z [cluster2]                         "f:clusterCIDR": {},
2021-07-19T04:15:39.6893833Z [cluster2]                         "f:clusterID": {},
2021-07-19T04:15:39.6894345Z [cluster2]                         "f:colorCodes": {},
2021-07-19T04:15:39.6894946Z [cluster2]                         "f:connectionHealthCheck": {
2021-07-19T04:15:39.6895904Z [cluster2]                             ".": {},
2021-07-19T04:15:39.6896371Z [cluster2]                             "f:enabled": {},
2021-07-19T04:15:39.6896924Z [cluster2]                             "f:intervalSeconds": {},
2021-07-19T04:15:39.6897540Z [cluster2]                             "f:maxPacketLossCount": {}
2021-07-19T04:15:39.6898068Z [cluster2]                         },
2021-07-19T04:15:39.6898504Z [cluster2]                         "f:debug": {},
2021-07-19T04:15:39.6898976Z [cluster2]                         "f:globalCIDR": {},
2021-07-19T04:15:39.6899485Z [cluster2]                         "f:namespace": {},
2021-07-19T04:15:39.6900149Z [cluster2]                         "f:natEnabled": {},
2021-07-19T04:15:39.6900690Z [cluster2]                         "f:repository": {},
2021-07-19T04:15:39.6901199Z [cluster2]                         "f:serviceCIDR": {},
2021-07-19T04:15:39.6901847Z [cluster2]                         "f:serviceDiscoveryEnabled": {},
2021-07-19T04:15:39.6902458Z [cluster2]                         "f:version": {}
2021-07-19T04:15:39.6902891Z [cluster2]                     }
2021-07-19T04:15:39.6903263Z [cluster2]                 },
2021-07-19T04:15:39.6903726Z [cluster2]                 "manager": "subctl",
2021-07-19T04:15:39.6904302Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:39.6905187Z [cluster2]                 "time": "2021-07-19T04:14:17Z"
2021-07-19T04:15:39.6905624Z [cluster2]             },
2021-07-19T04:15:39.6905990Z [cluster2]             {
2021-07-19T04:15:39.6906538Z [cluster2]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-19T04:15:39.6907170Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-19T04:15:39.6907692Z [cluster2]                 "fieldsV1": {
2021-07-19T04:15:39.6908131Z [cluster2]                     "f:status": {
2021-07-19T04:15:39.6908562Z [cluster2]                         ".": {},
2021-07-19T04:15:39.6909014Z [cluster2]                         "f:clusterCIDR": {},
2021-07-19T04:15:39.6909533Z [cluster2]                         "f:clusterID": {},
2021-07-19T04:15:39.6910041Z [cluster2]                         "f:colorCodes": {},
2021-07-19T04:15:39.6910694Z [cluster2]                         "f:deploymentInfo": {},
2021-07-19T04:15:39.6911999Z [cluster2]                         "f:gatewayDaemonSetStatus": {
2021-07-19T04:15:39.6912608Z [cluster2]                             ".": {},
2021-07-19T04:15:39.6913198Z [cluster2]                             "f:lastResourceVersion": {},
2021-07-19T04:15:39.6913992Z [cluster2]                             "f:mismatchedContainerImages": {},
2021-07-19T04:15:39.6915160Z [cluster2]                             "f:nonReadyContainerStates": {},
2021-07-19T04:15:39.6915764Z [cluster2]                             "f:status": {
2021-07-19T04:15:39.6916178Z [cluster2]                                 ".": {},
2021-07-19T04:15:39.6917010Z [cluster2]                                 "f:currentNumberScheduled": {},
2021-07-19T04:15:39.6917912Z [cluster2]                                 "f:desiredNumberScheduled": {},
2021-07-19T04:15:39.6918731Z [cluster2]                                 "f:numberAvailable": {},
2021-07-19T04:15:39.6920254Z [cluster2]                                 "f:numberMisscheduled": {},
2021-07-19T04:15:39.6930286Z [cluster2]                                 "f:numberReady": {},
2021-07-19T04:15:39.6931322Z [cluster2]                                 "f:observedGeneration": {},
2021-07-19T04:15:39.6932074Z [cluster2]                                 "f:updatedNumberScheduled": {}
2021-07-19T04:15:39.6932681Z [cluster2]                             }
2021-07-19T04:15:39.6933092Z [cluster2]                         },
2021-07-19T04:15:39.6933556Z [cluster2]                         "f:gateways": {},
2021-07-19T04:15:39.6934401Z [cluster2]                         "f:globalCIDR": {},
2021-07-19T04:15:39.6935035Z [cluster2]                         "f:globalnetDaemonSetStatus": {
2021-07-19T04:15:39.6935837Z [cluster2]                             ".": {},
2021-07-19T04:15:39.6936384Z [cluster2]                             "f:lastResourceVersion": {},
2021-07-19T04:15:39.6937101Z [cluster2]                             "f:mismatchedContainerImages": {},
2021-07-19T04:15:39.6937890Z [cluster2]                             "f:nonReadyContainerStates": {},
2021-07-19T04:15:39.6938611Z [cluster2]                             "f:status": {
2021-07-19T04:15:39.6939026Z [cluster2]                                 ".": {},
2021-07-19T04:15:39.6939601Z [cluster2]                                 "f:currentNumberScheduled": {},
2021-07-19T04:15:39.6940320Z [cluster2]                                 "f:desiredNumberScheduled": {},
2021-07-19T04:15:39.6941778Z [cluster2]                                 "f:numberAvailable": {},
2021-07-19T04:15:39.6942497Z [cluster2]                                 "f:numberMisscheduled": {},
2021-07-19T04:15:39.6943166Z [cluster2]                                 "f:numberReady": {},
2021-07-19T04:15:39.6944012Z [cluster2]                                 "f:observedGeneration": {},
2021-07-19T04:15:39.6945267Z [cluster2]                                 "f:updatedNumberScheduled": {}
2021-07-19T04:15:39.6945857Z [cluster2]                             }
2021-07-19T04:15:39.6946241Z [cluster2]                         },
2021-07-19T04:15:39.6946773Z [cluster2]                         "f:loadBalancerStatus": {},
2021-07-19T04:15:39.6947341Z [cluster2]                         "f:natEnabled": {},
2021-07-19T04:15:39.6947884Z [cluster2]                         "f:networkPlugin": {},
2021-07-19T04:15:39.6948566Z [cluster2]                         "f:routeAgentDaemonSetStatus": {
2021-07-19T04:15:39.6949186Z [cluster2]                             ".": {},
2021-07-19T04:15:39.6949738Z [cluster2]                             "f:lastResourceVersion": {},
2021-07-19T04:15:39.6950481Z [cluster2]                             "f:mismatchedContainerImages": {},
2021-07-19T04:15:39.6951477Z [cluster2]                             "f:nonReadyContainerStates": {},
2021-07-19T04:15:39.6952293Z [cluster2]                             "f:status": {
2021-07-19T04:15:39.6952739Z [cluster2]                                 ".": {},
2021-07-19T04:15:39.6953701Z [cluster2]                                 "f:currentNumberScheduled": {},
2021-07-19T04:15:39.6954814Z [cluster2]                                 "f:desiredNumberScheduled": {},
2021-07-19T04:15:39.6955481Z [cluster2]                                 "f:numberAvailable": {},
2021-07-19T04:15:39.6956848Z [cluster2]                                 "f:numberMisscheduled": {},
2021-07-19T04:15:39.6957482Z [cluster2]                                 "f:numberReady": {},
2021-07-19T04:15:39.6958080Z [cluster2]                                 "f:observedGeneration": {},
2021-07-19T04:15:39.6958794Z [cluster2]                                 "f:updatedNumberScheduled": {}
2021-07-19T04:15:39.6959363Z [cluster2]                             }
2021-07-19T04:15:39.6959770Z [cluster2]                         },
2021-07-19T04:15:39.6991505Z [cluster2]                         "f:serviceCIDR": {}
2021-07-19T04:15:39.6992045Z [cluster2]                     }
2021-07-19T04:15:39.6992614Z [cluster2]                 },
2021-07-19T04:15:39.6993741Z [cluster2]                 "manager": "submariner-operator",
2021-07-19T04:15:39.6994568Z [cluster2]                 "operation": "Update",
2021-07-19T04:15:39.6995282Z [cluster2]                 "time": "2021-07-19T04:15:03Z"
2021-07-19T04:15:39.6995727Z [cluster2]             }
2021-07-19T04:15:39.6996103Z [cluster2]         ],
2021-07-19T04:15:39.6996562Z [cluster2]         "name": "submariner",
2021-07-19T04:15:39.6997389Z [cluster2]         "namespace": "submariner-operator",
2021-07-19T04:15:39.6998053Z [cluster2]         "resourceVersion": "2367",
2021-07-19T04:15:39.6999332Z [cluster2]         "selfLink": "/apis/submariner.io/v1alpha1/namespaces/submariner-operator/submariners/submariner",
2021-07-19T04:15:39.7000528Z [cluster2]         "uid": "0a869c8a-acac-48e5-b44f-d044d2cbee3a"
2021-07-19T04:15:39.7001459Z [cluster2]     },
2021-07-19T04:15:39.7001867Z [cluster2]     "spec": {
2021-07-19T04:15:39.7002328Z [cluster2]         "broker": "k8s",
2021-07-19T04:15:39.7002916Z [cluster2]         "brokerK8sApiServer": "172.18.0.7:6443",
2021-07-19T04:15:39.7115461Z [cluster2]         "brokerK8sApiServerToken": "***",
2021-07-19T04:15:39.7793962Z [cluster2]         "brokerK8sCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==",
2021-07-19T04:15:39.8362078Z [cluster2]         "brokerK8sRemoteNamespace": "submariner-k8s-broker",
2021-07-19T04:15:39.8362945Z [cluster2]         "cableDriver": "libreswan",
2021-07-19T04:15:39.8363541Z [cluster2]         "ceIPSecDebug": false,
2021-07-19T04:15:39.8364252Z [cluster2]         "ceIPSecIKEPort": 500,
2021-07-19T04:15:39.8364808Z [cluster2]         "ceIPSecNATTPort": 4500,
2021-07-19T04:15:39.8367496Z [cluster2]         "ceIPSecPSK": "/lcX3t1f1LkWM2NxjfGKcDYjF5h8RcIAsg0peNU7/wCAZuvDKZBlCQBKlFRsYd+/",
2021-07-19T04:15:39.8368840Z [cluster2]         "clusterCIDR": "",
2021-07-19T04:15:39.8369518Z [cluster2]         "clusterID": "cluster2",
2021-07-19T04:15:39.8370050Z [cluster2]         "colorCodes": "blue",
2021-07-19T04:15:39.8370638Z [cluster2]         "connectionHealthCheck": {
2021-07-19T04:15:39.8371526Z [cluster2]             "enabled": true,
2021-07-19T04:15:39.8372037Z [cluster2]             "intervalSeconds": 1,
2021-07-19T04:15:39.8372590Z [cluster2]             "maxPacketLossCount": 5
2021-07-19T04:15:39.8373058Z [cluster2]         },
2021-07-19T04:15:39.8373437Z [cluster2]         "debug": false,
2021-07-19T04:15:39.8373891Z [cluster2]         "globalCIDR": "242.254.2.0/24",
2021-07-19T04:15:39.8374741Z [cluster2]         "namespace": "submariner-operator",
2021-07-19T04:15:39.8375333Z [cluster2]         "natEnabled": false,
2021-07-19T04:15:39.8376232Z [cluster2]         "repository": "localhost:5000",
2021-07-19T04:15:39.8376927Z [cluster2]         "serviceCIDR": "",
2021-07-19T04:15:39.8377951Z [cluster2]         "serviceDiscoveryEnabled": true,
2021-07-19T04:15:39.8378574Z [cluster2]         "version": "local"
2021-07-19T04:15:39.8379006Z [cluster2]     },
2021-07-19T04:15:39.8379385Z [cluster2]     "status": {
2021-07-19T04:15:39.8379856Z [cluster2]         "clusterCIDR": "10.0.0.0/16",
2021-07-19T04:15:39.8380365Z [cluster2]         "clusterID": "cluster2",
2021-07-19T04:15:39.8381063Z [cluster2]         "colorCodes": "blue",
2021-07-19T04:15:39.8381549Z [cluster2]         "deploymentInfo": {},
2021-07-19T04:15:39.8382146Z [cluster2]         "gatewayDaemonSetStatus": {
2021-07-19T04:15:39.8382773Z [cluster2]             "lastResourceVersion": "2114",
2021-07-19T04:15:39.8383656Z [cluster2]             "mismatchedContainerImages": false,
2021-07-19T04:15:39.8384397Z [cluster2]             "nonReadyContainerStates": [],
2021-07-19T04:15:39.8384969Z [cluster2]             "status": {
2021-07-19T04:15:39.8385537Z [cluster2]                 "currentNumberScheduled": 1,
2021-07-19T04:15:39.8387165Z [cluster2]                 "desiredNumberScheduled": 1,
2021-07-19T04:15:39.8388122Z [cluster2]                 "numberAvailable": 1,
2021-07-19T04:15:39.8388776Z [cluster2]                 "numberMisscheduled": 0,
2021-07-19T04:15:39.8389405Z [cluster2]                 "numberReady": 1,
2021-07-19T04:15:39.8390329Z [cluster2]                 "observedGeneration": 1,
2021-07-19T04:15:39.8391499Z [cluster2]                 "updatedNumberScheduled": 1
2021-07-19T04:15:39.8392022Z [cluster2]             }
2021-07-19T04:15:39.8392377Z [cluster2]         },
2021-07-19T04:15:39.8392926Z [cluster2]         "gateways": [
2021-07-19T04:15:39.8393328Z [cluster2]             {
2021-07-19T04:15:39.8393756Z [cluster2]                 "connections": [
2021-07-19T04:15:39.8394177Z [cluster2]                     {
2021-07-19T04:15:39.8394604Z [cluster2]                         "endpoint": {
2021-07-19T04:15:39.8395105Z [cluster2]                             "backend": "libreswan",
2021-07-19T04:15:39.8396111Z [cluster2]                             "backend_config": {
2021-07-19T04:15:39.8397124Z [cluster2]                                 "natt-discovery-port": "4490",
2021-07-19T04:15:39.8398397Z [cluster2]                                 "preferred-server": "false"
2021-07-19T04:15:39.8398918Z [cluster2]                             },
2021-07-19T04:15:39.8399914Z [cluster2]                             "cable_name": "submariner-cable-cluster1-172-18-0-8",
2021-07-19T04:15:39.8400785Z [cluster2]                             "cluster_id": "cluster1",
2021-07-19T04:15:39.8401493Z [cluster2]                             "healthCheckIP": "242.254.1.254",
2021-07-19T04:15:39.8402272Z [cluster2]                             "hostname": "cluster1-worker",
2021-07-19T04:15:39.8402840Z [cluster2]                             "nat_enabled": false,
2021-07-19T04:15:39.8403586Z [cluster2]                             "private_ip": "172.18.0.8",
2021-07-19T04:15:39.8404454Z [cluster2]                             "public_ip": "40.124.48.108",
2021-07-19T04:15:39.8404925Z [cluster2]                             "subnets": [
2021-07-19T04:15:39.8405353Z [cluster2]                                 "242.254.1.0/24"
2021-07-19T04:15:39.8405759Z [cluster2]                             ]
2021-07-19T04:15:39.8406151Z [cluster2]                         },
2021-07-19T04:15:39.8406957Z [cluster2]                         "latencyRTT": {
2021-07-19T04:15:39.8407970Z [cluster2]                             "average": "199.856µs",
2021-07-19T04:15:39.8409862Z [cluster2]                             "last": "152.102µs",
2021-07-19T04:15:39.8410576Z [cluster2]                             "max": "415.705µs",
2021-07-19T04:15:39.8411240Z [cluster2]                             "min": "133.401µs",
2021-07-19T04:15:39.8418779Z [cluster2]                             "stdDev": "58.696µs"
2021-07-19T04:15:39.8419316Z [cluster2]                         },
2021-07-19T04:15:39.8420186Z [cluster2]                         "status": "connected",
2021-07-19T04:15:39.8421468Z [cluster2]                         "statusMessage": "",
2021-07-19T04:15:39.8422129Z [cluster2]                         "usingIP": "172.18.0.8"
2021-07-19T04:15:39.8422569Z [cluster2]                     }
2021-07-19T04:15:39.8423089Z [cluster2]                 ],
2021-07-19T04:15:39.8423674Z [cluster2]                 "haStatus": "active",
2021-07-19T04:15:39.8424189Z [cluster2]                 "localEndpoint": {
2021-07-19T04:15:39.8424868Z [cluster2]                     "backend": "libreswan",
2021-07-19T04:15:39.8425518Z [cluster2]                     "backend_config": {
2021-07-19T04:15:39.8426646Z [cluster2]                         "natt-discovery-port": "4490",
2021-07-19T04:15:39.8428850Z [cluster2]                         "preferred-server": "false"
2021-07-19T04:15:39.8429594Z [cluster2]                     },
2021-07-19T04:15:39.8430666Z [cluster2]                     "cable_name": "submariner-cable-cluster2-172-18-0-9",
2021-07-19T04:15:39.8431973Z [cluster2]                     "cluster_id": "cluster2",
2021-07-19T04:15:39.8432702Z [cluster2]                     "hostname": "cluster2-worker",
2021-07-19T04:15:39.8433239Z [cluster2]                     "nat_enabled": false,
2021-07-19T04:15:39.8433915Z [cluster2]                     "private_ip": "172.18.0.9",
2021-07-19T04:15:39.8434380Z [cluster2]                     "public_ip": "40.124.48.108",
2021-07-19T04:15:39.8435061Z [cluster2]                     "subnets": [
2021-07-19T04:15:39.8435506Z [cluster2]                         "242.254.2.0/24"
2021-07-19T04:15:39.8435905Z [cluster2]                     ]
2021-07-19T04:15:39.8436285Z [cluster2]                 },
2021-07-19T04:15:39.8436718Z [cluster2]                 "statusFailure": "",
2021-07-19T04:15:39.8437476Z [cluster2]                 "version": "devel-1441a80"
2021-07-19T04:15:39.8438164Z [cluster2]             }
2021-07-19T04:15:39.8449253Z [cluster2]         ],
2021-07-19T04:15:39.8449803Z [cluster2]         "globalCIDR": "242.254.2.0/24",
2021-07-19T04:15:39.8450868Z [cluster2]         "globalnetDaemonSetStatus": {
2021-07-19T04:15:39.8452107Z [cluster2]             "lastResourceVersion": "2134",
2021-07-19T04:15:39.8452880Z [cluster2]             "mismatchedContainerImages": false,
2021-07-19T04:15:39.8453673Z [cluster2]             "nonReadyContainerStates": [],
2021-07-19T04:15:39.8454479Z [cluster2]             "status": {
2021-07-19T04:15:40.1280484Z [cluster2]                 "currentNumberScheduled": 1,
2021-07-19T04:15:40.1281177Z [cluster2]                 "desiredNumberScheduled": 1,
2021-07-19T04:15:40.1281912Z [cluster2]                 "numberAvailable": 1,
2021-07-19T04:15:40.1282937Z [cluster2]                 "numberMisscheduled": 0,
2021-07-19T04:15:40.1283939Z [cluster2]                 "numberReady": 1,
2021-07-19T04:15:40.1284690Z [cluster2]                 "observedGeneration": 1,
2021-07-19T04:15:40.1285873Z [cluster2]                 "updatedNumberScheduled": 1
2021-07-19T04:15:40.1287430Z [cluster2]             }
2021-07-19T04:15:40.1287793Z [cluster2]         },
2021-07-19T04:15:40.1288293Z [cluster2]         "loadBalancerStatus": {},
2021-07-19T04:15:40.1289037Z [cluster2]         "natEnabled": false,
2021-07-19T04:15:40.1290813Z [cluster2]         "networkPlugin": "weave-net",
2021-07-19T04:15:40.1291475Z [cluster2]         "routeAgentDaemonSetStatus": {
2021-07-19T04:15:40.1292159Z [cluster2]             "lastResourceVersion": "2187",
2021-07-19T04:15:40.1293043Z [cluster2]             "mismatchedContainerImages": false,
2021-07-19T04:15:40.1294267Z [cluster2]             "nonReadyContainerStates": [],
2021-07-19T04:15:40.1294921Z [cluster2]             "status": {
2021-07-19T04:15:40.1295963Z [cluster2]                 "currentNumberScheduled": 3,
2021-07-19T04:15:40.1296959Z [cluster2]                 "desiredNumberScheduled": 3,
2021-07-19T04:15:40.1297809Z [cluster2]                 "numberAvailable": 3,
2021-07-19T04:15:40.1298393Z [cluster2]                 "numberMisscheduled": 0,
2021-07-19T04:15:40.1298936Z [cluster2]                 "numberReady": 3,
2021-07-19T04:15:40.1299496Z [cluster2]                 "observedGeneration": 1,
2021-07-19T04:15:40.1300138Z [cluster2]                 "updatedNumberScheduled": 3
2021-07-19T04:15:40.1300675Z [cluster2]             }
2021-07-19T04:15:40.1301021Z [cluster2]         },
2021-07-19T04:15:40.1301459Z [cluster2]         "serviceCIDR": "100.0.0.0/16"
2021-07-19T04:15:40.1301886Z [cluster2]     }
2021-07-19T04:15:40.1302229Z [cluster2] }
2021-07-19T04:15:40.1303438Z [submariner-operator]$ [cluster2] validate_equals .metadata.namespace submariner-operator
2021-07-19T04:15:40.1304780Z [submariner-operator]$ [cluster2] validate_equals .metadata.namespace submariner-operator
2021-07-19T04:15:40.1306620Z [submariner-operator]$ [cluster2] jq -r -M .metadata.namespace /tmp/submariner.cluster2.json
2021-07-19T04:15:40.1308123Z [submariner-operator]$ [cluster2] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-19T04:15:40.1309483Z [submariner-operator]$ [cluster2] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-19T04:15:40.1310711Z [submariner-operator]$ [cluster2] jq -r -M .apiVersion /tmp/submariner.cluster2.json
2021-07-19T04:15:40.1311824Z [submariner-operator]$ [cluster2] validate_equals .kind Submariner
2021-07-19T04:15:40.1312824Z [submariner-operator]$ [cluster2] validate_equals .kind Submariner
2021-07-19T04:15:40.1314045Z [submariner-operator]$ [cluster2] jq -r -M .kind /tmp/submariner.cluster2.json
2021-07-19T04:15:40.1315262Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariner
2021-07-19T04:15:40.1316709Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariner
2021-07-19T04:15:40.1318132Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/submariner.cluster2.json
2021-07-19T04:15:40.1319610Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sApiServer 172.18.0.7:6443
2021-07-19T04:15:40.1320852Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sApiServer 172.18.0.7:6443
2021-07-19T04:15:40.1322226Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sApiServer /tmp/submariner.cluster2.json
2021-07-19T04:15:40.1499795Z [submariner-operator]$ [cluster2] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-19T04:15:40.2891968Z [submariner-operator]$ [cluster2] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-19T04:15:40.3612902Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
2021-07-19T04:15:40.4214293Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sCA /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4221843Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-19T04:15:40.4224159Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-19T04:15:40.4226380Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sRemoteNamespace /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4228464Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecDebug false
2021-07-19T04:15:40.4229699Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecDebug false
2021-07-19T04:15:40.4231096Z [submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecDebug /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4237934Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecIKEPort 500
2021-07-19T04:15:40.4239651Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecIKEPort 500
2021-07-19T04:15:40.4241340Z [submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecIKEPort /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4242931Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-19T04:15:40.4244449Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-19T04:15:40.4246183Z [submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecNATTPort /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4247856Z [submariner-operator]$ [cluster1] validate_equals .spec.repository localhost:5000
2021-07-19T04:15:40.4249296Z [submariner-operator]$ [cluster1] validate_equals .spec.repository localhost:5000
2021-07-19T04:15:40.4258139Z [submariner-operator]$ [cluster1] jq -r -M .spec.repository /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4260833Z [submariner-operator]$ [cluster1] validate_equals .spec.version local
2021-07-19T04:15:40.4264858Z [submariner-operator]$ [cluster1] validate_equals .spec.version local
2021-07-19T04:15:40.4267008Z [submariner-operator]$ [cluster1] jq -r -M .spec.version /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4268412Z [submariner-operator]$ [cluster1] validate_equals .spec.broker k8s
2021-07-19T04:15:40.4270137Z [submariner-operator]$ [cluster1] validate_equals .spec.broker k8s
2021-07-19T04:15:40.4271738Z [submariner-operator]$ [cluster1] jq -r -M .spec.broker /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4273683Z [submariner-operator]$ [cluster1] jq -r .spec.clusterID /tmp/submariner.cluster1.json
2021-07-19T04:15:40.4275349Z [cluster1] Generated cluster id: cluster1
2021-07-19T04:15:40.4351010Z [submariner-operator]$ [cluster1] validate_equals .spe3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoic3VibWFyaW5lci1rOHMtYnJva2VyLWNsaWVudC10b2tlbi1qa25nbCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJzdWJtYXJpbmVyLWs4cy1icm9rZXItY2xpZW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZTZlMmMxZGYtNmI1YS00NWU4LThiOWYtNjBiNTliOGQxMmNmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OnN1Ym1hcmluZXItazhzLWJyb2tlcjpzdWJtYXJpbmVyLWs4cy1icm9rZXItY2xpZW50In0.hsT4p69D-V8ElgWMTm58Cor-VLVSCksyZLU8bEmjqu8j2HxQQyx_DdeIKqjHYTENQsCOm0C4uFjM_V6SdPIkdtIMBq1iUKSlM1D1zpXdl3m4ILUxvGRoMvWuns2BKJx1oyUiUMF-hdyA_xSywo8s0YXf0MPupB-x8DqaBnfu-5AdaQR4GE7cdv6pNNXQMOFrg4tznwLik7aS2PHRdSi3T22yZ276F48NhcEPZuBGC4pktlr4JlwdGCh381YsZNKgm-VjXWc6Jry9yIWlFa0XalvwCQ0rb0pKNc3LeLzhyhfNJRqM3jzx7tBDcdV56dFqrorcaqWwtn7_-EaHBEJVUA
2021-07-19T04:15:40.4427616Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sApiServerToken /tmp/submariner.cluster2.json
2021-07-19T04:15:40.5054647Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
2021-07-19T04:15:40.6324863Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hPVEEwTURZME9Gb1hEVE14TURjeE56QTBNRFkwT0Zvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBS3FxCldsOU5XRVRDL3RUOGYxNlBYSjlnRUlKeTJkV2dVVFFkYWt6NUlMVXJEUkc2SjNaQTRTRTFGM1RCaTIvSFVudDcKaTBOUU4weTQzc3J1T2grOGdNdWtBc2V1VEx4bm85Zm5sTlo1bmxweG9qeVNyZnJvUytJQVpZMTMyQlFBVVRPeApvVzloWGVDVmsxNTRtQXdaR1UyLzZ3TnZQU09iZ2RTb3FDQWNVYkUya1NXQmNLalFDRUg4S3kzNzB6NkNNVGxLCkhSNU1WR3ZRclNWcDc4NmxaQmJ5L0dVMWpsQTIwc21zeGU4ekhBSkNRVzhQaUpEZVVIOGFDMjdJb3dxMU1XRzYKbG8reUtlRHYvN1VxaW5USlBCZm5BRllrL3orK203Z1FkN2c5YVIrR0VWVCtQdnR5dUZmcXNLNStiRjB0dFY3KwpQc0NVeWVhVWpKQlFSUm9zQXdjQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZOcnFCS1lBNCtyL3V0aEJwMkNQYlJGVGVIdFhNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBTXRxdjBTeUFpaEdUNVpNM0dKc1lCRHkwQ1o4Mmc2VDlxTHphUWRRRUo1ZzJJcWxqWAphSWtXT1ZrNFNvREJhdlM0eVJEeHAvcGdKdG4wSU9pN2d3akpaaFNTN1lOWjNRLzdTNzJIL0RrN2lPSlVvVDBqCnE1clJRcjhqSzhUWE81YS9XdDRDRDhLYzkyTm80NWhmSFJXODFLWFNEN2ZTR2YrOWMvZkdQY3I2ZDcwcGFPckwKRStZcDVKR2pJVFhkNU1FREFzOWZMTmtxQnprbFlHZTAwajJ2QUliM0RMdk5oOUhyc3puNUFwUlJDWDhaVEFRRgpqa2hVcVV6UUpodTdBMUhGbnhJVkJBdzBoMU5pWXl2bDJUUXF2bS9DanovVnRmRS9tWWdYZlJrWHIvKytGSHFNCjFYU2NGeXg4Tm0yRVZWTFJ6RzMyM1pwTVArcy9pY2V4L3lFdgotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
2021-07-19T04:15:40.7050813Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sCA /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5432537Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-19T04:15:41.5435077Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-19T04:15:41.5437079Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sRemoteNamespace /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5438581Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecDebug false
2021-07-19T04:15:41.5440283Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecDebug false
2021-07-19T04:15:41.5442195Z [submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecDebug /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5444134Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecIKEPort 500
2021-07-19T04:15:41.5445625Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecIKEPort 500
2021-07-19T04:15:41.5447319Z [submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecIKEPort /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5448787Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-19T04:15:41.5450055Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-19T04:15:41.5451593Z [submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecNATTPort /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5453046Z [submariner-operator]$ [cluster2] validate_equals .spec.repository localhost:5000
2021-07-19T04:15:41.5454490Z [submariner-operator]$ [cluster2] validate_equals .spec.repository localhost:5000
2021-07-19T04:15:41.5456230Z [submariner-operator]$ [cluster2] jq -r -M .spec.repository /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5457594Z [submariner-operator]$ [cluster2] validate_equals .spec.version local
2021-07-19T04:15:41.5458732Z [submariner-operator]$ [cluster2] validate_equals .spec.version local
2021-07-19T04:15:41.5460012Z [submariner-operator]$ [cluster2] jq -r -M .spec.version /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5461264Z [submariner-operator]$ [cluster2] validate_equals .spec.broker k8s
2021-07-19T04:15:41.5462346Z [submariner-operator]$ [cluster2] validate_equals .spec.broker k8s
2021-07-19T04:15:41.5463824Z [submariner-operator]$ [cluster2] jq -r -M .spec.broker /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5465620Z [submariner-operator]$ [cluster2] jq -r .spec.clusterID /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5466577Z [cluster2] Generated cluster id: cluster2
2021-07-19T04:15:41.5467625Z [submariner-operator]$ [cluster2] validate_equals .spec.colorCodes blue
2021-07-19T04:15:41.5468799Z [submariner-operator]$ [cluster2] validate_equals .spec.colorCodes blue
2021-07-19T04:15:41.5470132Z [submariner-operator]$ [cluster2] jq -r -M .spec.colorCodes /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5471416Z [submariner-operator]$ [cluster2] validate_equals .spec.debug false
2021-07-19T04:15:41.5472687Z [submariner-operator]$ [cluster2] validate_equals .spec.debug false
2021-07-19T04:15:41.5473968Z [submariner-operator]$ [cluster2] jq -r -M .spec.debug /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5475370Z [submariner-operator]$ [cluster2] validate_equals .spec.namespace submariner-operator
2021-07-19T04:15:41.5476930Z [submariner-operator]$ [cluster2] validate_equals .spec.namespace submariner-operator
2021-07-19T04:15:41.5478302Z [submariner-operator]$ [cluster2] jq -r -M .spec.namespace /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5479577Z [submariner-operator]$ [cluster2] validate_equals .spec.natEnabled false
2021-07-19T04:15:41.5480850Z [submariner-operator]$ [cluster2] validate_equals .spec.natEnabled false
2021-07-19T04:15:41.5482279Z [submariner-operator]$ [cluster2] jq -r -M .spec.natEnabled /tmp/submariner.cluster2.json
2021-07-19T04:15:41.5483673Z [submariner-operator]$ [cluster2] verify_subm_cr_status_with_retries
2021-07-19T04:15:41.5484701Z [submariner-operator]$ [cluster2] verify_subm_cr_status_with_retries
2021-07-19T04:15:41.5485746Z [submariner-operator]$ [cluster2] with_retries 5 verify_subm_cr_status_
2021-07-19T04:15:41.5486777Z [submariner-operator]$ [cluster2] with_retries 5 verify_subm_cr_status_
2021-07-19T04:15:41.5487727Z [submariner-operator]$ [cluster2] iteration=1
2021-07-19T04:15:41.5488581Z [submariner-operator]$ [cluster2] wait 8704
2021-07-19T04:15:41.5489474Z [submariner-operator]$ [cluster2] verify_subm_cr_status_
2021-07-19T04:15:41.5490592Z [submariner-operator]$ [cluster2] verify_subm_cr_status_
2021-07-19T04:15:41.5491895Z [submariner-operator]$ [cluster2] verify_subm_cr_status
2021-07-19T04:15:41.5493480Z [submariner-operator]$ [cluster2] verify_subm_cr_status
2021-07-19T04:15:41.5495274Z [submariner-operator]$ [cluster2] json_file=/tmp/submariner.cluster2.json
2021-07-19T04:15:41.5536865Z [cluster2] 
2021-07-19T04:15:41.5537566Z [cluster2] 
2021-07-19T04:15:41.5538801Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:41.5540488Z [submariner-operator]$ [cluster2] validate_equals .status.serviceCIDR 100.0.0.0/16
2021-07-19T04:15:41.8374063Z [submaric.colorCodes blue
2021-07-19T04:15:41.8375436Z [submariner-operator]$ [cluster1] validate_equals .spec.colorCodes blue
2021-07-19T04:15:41.8377087Z [submariner-operator]$ [cluster1] jq -r -M .spec.colorCodes /tmp/submariner.cluster1.json
2021-07-19T04:15:41.8378393Z [submariner-operator]$ [cluster1] validate_equals .spec.debug false
2021-07-19T04:15:41.8379490Z [submariner-operator]$ [cluster1] validate_equals .spec.debug false
2021-07-19T04:15:41.8380747Z [submariner-operator]$ [cluster1] jq -r -M .spec.debug /tmp/submariner.cluster1.json
2021-07-19T04:15:41.8382174Z [submariner-operator]$ [cluster1] validate_equals .spec.namespace submariner-operator
2021-07-19T04:15:41.8383566Z [submariner-operator]$ [cluster1] validate_equals .spec.namespace submariner-operator
2021-07-19T04:15:41.8384993Z [submariner-operator]$ [cluster1] jq -r -M .spec.namespace /tmp/submariner.cluster1.json
2021-07-19T04:15:41.8386510Z [submariner-operator]$ [cluster1] validate_equals .spec.natEnabled false
2021-07-19T04:15:41.8387792Z [submariner-operator]$ [cluster1] validate_equals .spec.natEnabled false
2021-07-19T04:15:41.8389117Z [submariner-operator]$ [cluster1] jq -r -M .spec.natEnabled /tmp/submariner.cluster1.json
2021-07-19T04:15:41.8390394Z [submariner-operator]$ [cluster1] verify_subm_cr_status_with_retries
2021-07-19T04:15:41.8391614Z [submariner-operator]$ [cluster1] verify_subm_cr_status_with_retries
2021-07-19T04:15:41.8392848Z [submariner-operator]$ [cluster1] with_retries 5 verify_subm_cr_status_
2021-07-19T04:15:41.8393978Z [submariner-operator]$ [cluster1] with_retries 5 verify_subm_cr_status_
2021-07-19T04:15:41.8394952Z [submariner-operator]$ [cluster1] iteration=1
2021-07-19T04:15:41.8396135Z [submariner-operator]$ [cluster1] wait 8653
2021-07-19T04:15:41.8397015Z [submariner-operator]$ [cluster1] verify_subm_cr_status_
2021-07-19T04:15:41.8397961Z [submariner-operator]$ [cluster1] verify_subm_cr_status_
2021-07-19T04:15:41.8398901Z [submariner-operator]$ [cluster1] verify_subm_cr_status
2021-07-19T04:15:41.8399819Z [submariner-operator]$ [cluster1] verify_subm_cr_status
2021-07-19T04:15:41.8400920Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner.cluster1.json
2021-07-19T04:15:41.8401907Z [cluster1] 
2021-07-19T04:15:41.8402415Z [cluster1] 
2021-07-19T04:15:41.8403583Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner submariner --namespace=submariner-operator -o json
2021-07-19T04:15:41.8405236Z [submariner-operator]$ [cluster1] validate_equals .status.serviceCIDR 100.0.0.0/16
2021-07-19T04:15:41.8406525Z [submariner-operator]$ [cluster1] validate_equals .status.serviceCIDR 100.0.0.0/16
2021-07-19T04:15:41.8407851Z [submariner-operator]$ [cluster1] jq -r -M .status.serviceCIDR /tmp/submariner.cluster1.json
2021-07-19T04:15:41.8409187Z [submariner-operator]$ [cluster1] validate_equals .status.clusterCIDR 10.0.0.0/16
2021-07-19T04:15:41.8410361Z [submariner-operator]$ [cluster1] validate_equals .status.clusterCIDR 10.0.0.0/16
2021-07-19T04:15:41.8411696Z [submariner-operator]$ [cluster1] jq -r -M .status.clusterCIDR /tmp/submariner.cluster1.json
2021-07-19T04:15:41.8412822Z [submariner-operator]$ [cluster1] return 0
2021-07-19T04:15:41.8413679Z [submariner-operator]$ [cluster1] return 0
2021-07-19T04:15:41.8414793Z [submariner-operator]$ [cluster1] verify_subm_gateway_deployment
2021-07-19T04:15:41.8416070Z [submariner-operator]$ [cluster1] verify_subm_gateway_deployment
2021-07-19T04:15:41.8417340Z [submariner-operator]$ [cluster1] verify_daemonset submariner-gateway submariner-gateway
2021-07-19T04:15:41.8418701Z [submariner-operator]$ [cluster1] verify_daemonset submariner-gateway submariner-gateway
2021-07-19T04:15:41.8419943Z [submariner-operator]$ [cluster1] daemonset_app=submariner-gateway
2021-07-19T04:15:41.8421054Z [submariner-operator]$ [cluster1] daemonset_name=submariner-gateway
2021-07-19T04:15:41.8422131Z [submariner-operator]$ [cluster1] with_retries 60 daemonset_created
2021-07-19T04:15:41.8423158Z [submariner-operator]$ [cluster1] with_retries 60 daemonset_created
2021-07-19T04:15:41.8424072Z [submariner-operator]$ [cluster1] iteration=1
2021-07-19T04:15:41.8424902Z [submariner-operator]$ [cluster1] wait 8735
2021-07-19T04:15:41.8425765Z [submariner-operator]$ [cluster1] daemonset_created
2021-07-19T04:15:41.8426682Z [submariner-operator]$ [cluster1] daemonset_created
2021-07-19T04:15:41.8427546Z [cluster1] 
2021-07-19T04:15:41.8428236Z [submariner-operator]$ [cluster1] return 0
2021-07-19T04:15:41.8429196Z [submariner-operator]$ [cluster1] with_retries 120 daemonset_deployed
2021-07-19T04:15:41.8430664Z [submariner-operator]$ [cluster1] with_retries 120 daemonset_deployed
2021-07-19T04:15:41.8431691Z [submariner-operator]$ [cluster1] iteration=1
2021-07-19T04:15:41.8432576Z [submariner-operator]$ [cluster1] daemonset_deployed
2021-07-19T04:15:41.8433438Z [submariner-operator]$ [cluster1] wait 8784
2021-07-19T04:15:41.8434292Z [submariner-operator]$ [cluster1] daemonset_deployed
2021-07-19T04:15:43.4751787Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScner-operator]$ [cluster2] validate_equals .status.serviceCIDR 100.0.0.0/16
2021-07-19T04:15:43.4754399Z [submariner-operator]$ [cluster2] jq -r -M .status.serviceCIDR /tmp/submariner.cluster2.json
2021-07-19T04:15:43.4755747Z [submariner-operator]$ [cluster2] validate_equals .status.clusterCIDR 10.0.0.0/16
2021-07-19T04:15:43.4756937Z [submariner-operator]$ [cluster2] validate_equals .status.clusterCIDR 10.0.0.0/16
2021-07-19T04:15:43.4758270Z [submariner-operator]$ [cluster2] jq -r -M .status.clusterCIDR /tmp/submariner.cluster2.json
2021-07-19T04:15:43.4759689Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:43.4760754Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:43.4761925Z [submariner-operator]$ [cluster2] verify_subm_gateway_deployment
2021-07-19T04:15:43.4763011Z [submariner-operator]$ [cluster2] verify_subm_gateway_deployment
2021-07-19T04:15:43.4764567Z [submariner-operator]$ [cluster2] verify_daemonset submariner-gateway submariner-gateway
2021-07-19T04:15:43.4766264Z [submariner-operator]$ [cluster2] verify_daemonset submariner-gateway submariner-gateway
2021-07-19T04:15:43.4767490Z [submariner-operator]$ [cluster2] daemonset_app=submariner-gateway
2021-07-19T04:15:43.4768628Z [submariner-operator]$ [cluster2] daemonset_name=submariner-gateway
2021-07-19T04:15:43.4769701Z [submariner-operator]$ [cluster2] with_retries 60 daemonset_created
2021-07-19T04:15:43.4770933Z [submariner-operator]$ [cluster2] with_retries 60 daemonset_created
2021-07-19T04:15:43.4771929Z [submariner-operator]$ [cluster2] iteration=1
2021-07-19T04:15:43.4772802Z [submariner-operator]$ [cluster2] wait 8767
2021-07-19T04:15:43.4774053Z [submariner-operator]$ [cluster2] daemonset_created
2021-07-19T04:15:43.4774959Z [submariner-operator]$ [cluster2] daemonset_created
2021-07-19T04:15:43.4775816Z [cluster2] 
2021-07-19T04:15:43.4776505Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:43.4777457Z [submariner-operator]$ [cluster2] with_retries 120 daemonset_deployed
2021-07-19T04:15:43.4778501Z [submariner-operator]$ [cluster2] with_retries 120 daemonset_deployed
2021-07-19T04:15:43.4779424Z [submariner-operator]$ [cluster2] iteration=1
2021-07-19T04:15:43.4780491Z [submariner-operator]$ [cluster2] daemonset_deployed
2021-07-19T04:15:43.4781801Z [submariner-operator]$ [cluster2] wait 8802
2021-07-19T04:15:43.4782735Z [submariner-operator]$ [cluster2] daemonset_deployed
2021-07-19T04:15:43.4784734Z [submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-19T04:15:43.4786501Z [submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-19T04:15:43.4788381Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-19T04:15:43.4790325Z [submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-19T04:15:43.4792388Z [submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-19T04:15:43.4794817Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-19T04:15:43.4796143Z [submariner-operator]$ [cluster2] return 0
2021-07-19T04:15:43.4797032Z [submariner-operator]$ [cluster2] verify_subm_gateway_pod
2021-07-19T04:15:43.4797952Z [submariner-operator]$ [cluster2] verify_subm_gateway_pod
2021-07-19T04:15:43.4799365Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-19T04:15:43.4801904Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-19T04:15:43.4804672Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-19T04:15:43.4806122Z [cluster2] pod/submariner-gateway-zzp28 condition met
2021-07-19T04:15:43.4807851Z [submariner-operator]$ [cluster2] subm_gateway_pod_name=$(kubectl get pods --namespace=$subm_ns -l app=$gateway_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-19T04:15:43.4809890Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:43.4812110Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:44.2947061Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods --namespace=submariner-operator -l app=submariner-gateway -o=jheduled}
2021-07-19T04:15:44.2983160Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-19T04:15:44.2985461Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-19T04:15:44.2987529Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-19T04:15:44.2989174Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-19T04:15:44.2990751Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-19T04:15:44.2992617Z [submariner-operator]$ [cluster1] return 0
2021-07-19T04:15:44.2993959Z [submariner-operator]$ [cluster1] verify_subm_gateway_pod
2021-07-19T04:15:44.2994894Z [submariner-operator]$ [cluster1] verify_subm_gateway_pod
2021-07-19T04:15:44.2996593Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-19T04:15:44.2998623Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-19T04:15:44.3000748Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-19T04:15:44.3002521Z [cluster1] pod/submariner-gateway-xf78j condition met
2021-07-19T04:15:44.3004222Z [submariner-operator]$ [cluster1] subm_gateway_pod_name=$(kubectl get pods --namespace=$subm_ns -l app=$gateway_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-19T04:15:44.3006536Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:44.3009261Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:44.3011462Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-19T04:15:44.3013141Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner-gateway-xf78j.cluster1.json
2021-07-19T04:15:44.3015385Z [submariner-operator]$ [cluster1] kubectl get pod submariner-gateway-xf78j --namespace=submariner-operator -o json
2021-07-19T04:15:44.3043649Z [submariner-operator]$ [cluster1] kubectl get pod submariner-gateway-xf78j --namespace=submariner-operator -o json
2021-07-19T04:15:44.3045372Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-gateway-xf78j --namespace=submariner-operator -o json
2021-07-19T04:15:44.3047193Z [submariner-oper