subm_random_fail_20210716-1

logs.txt

2021-07-16T18:04:58.3973222Z Found online and idle hosted runner(s) in the current repository's organization account that matches the required labels: 'ubuntu-latest'
2021-07-16T18:04:58.3973286Z Waiting for a hosted runner in 'organization' to pick this job...
2021-07-16T18:05:07.0932084Z Current runner version: '2.278.0'
2021-07-16T18:05:07.0959968Z ##[group]Operating System
2021-07-16T18:05:07.0961020Z Ubuntu
2021-07-16T18:05:07.0961494Z 20.04.2
2021-07-16T18:05:07.0962012Z LTS
2021-07-16T18:05:07.0962514Z ##[endgroup]
2021-07-16T18:05:07.0963184Z ##[group]Virtual Environment
2021-07-16T18:05:07.0964205Z Environment: ubuntu-20.04
2021-07-16T18:05:07.0964855Z Version: 20210712.0
2021-07-16T18:05:07.0965960Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu20/20210712.0/images/linux/Ubuntu2004-README.md
2021-07-16T18:05:07.0967508Z Image Release: https://github.com/actions/virtual-environments/releases/tag/ubuntu20%2F20210712.0
2021-07-16T18:05:07.0968526Z ##[endgroup]
2021-07-16T18:05:07.0970708Z ##[group]GITHUB_TOKEN Permissions
2021-07-16T18:05:07.0972028Z Actions: write
2021-07-16T18:05:07.0974725Z Checks: write
2021-07-16T18:05:07.0975284Z Contents: write
2021-07-16T18:05:07.0975956Z Deployments: write
2021-07-16T18:05:07.0976725Z Discussions: write
2021-07-16T18:05:07.0977376Z Issues: write
2021-07-16T18:05:07.0977923Z Metadata: read
2021-07-16T18:05:07.0978523Z Packages: write
2021-07-16T18:05:07.0979132Z PullRequests: write
2021-07-16T18:05:07.0979861Z RepositoryProjects: write
2021-07-16T18:05:07.0980547Z SecurityEvents: write
2021-07-16T18:05:07.0981205Z Statuses: write
2021-07-16T18:05:07.0981894Z ##[endgroup]
2021-07-16T18:05:07.0985242Z Prepare workflow directory
2021-07-16T18:05:07.1679058Z Prepare all required actions
2021-07-16T18:05:07.1692101Z Getting action download info
2021-07-16T18:05:07.3411205Z Download action repository 'actions/checkout@v2'
2021-07-16T18:05:09.1563373Z Download action repository 'submariner-io/shipyard@devel'
2021-07-16T18:05:09.5842486Z ##[group]Run actions/checkout@v2
2021-07-16T18:05:09.5843221Z with:
2021-07-16T18:05:09.5844106Z   repository: submariner-io/submariner-operator
2021-07-16T18:05:09.5845366Z   token: ***
2021-07-16T18:05:09.5845863Z   ssh-strict: true
2021-07-16T18:05:09.5846496Z   persist-credentials: true
2021-07-16T18:05:09.5847100Z   clean: true
2021-07-16T18:05:09.5847605Z   fetch-depth: 1
2021-07-16T18:05:09.5848091Z   lfs: false
2021-07-16T18:05:09.5848593Z   submodules: false
2021-07-16T18:05:09.5849158Z ##[endgroup]
2021-07-16T18:05:09.8312610Z Syncing repository: submariner-io/submariner-operator
2021-07-16T18:05:09.8317246Z ##[group]Getting Git version info
2021-07-16T18:05:09.8318801Z Working directory is '/home/runner/work/submariner-operator/submariner-operator'
2021-07-16T18:05:09.8366981Z [command]/usr/bin/git version
2021-07-16T18:05:09.8530527Z git version 2.32.0
2021-07-16T18:05:09.8553002Z ##[endgroup]
2021-07-16T18:05:09.8561123Z Deleting the contents of '/home/runner/work/submariner-operator/submariner-operator'
2021-07-16T18:05:09.8564243Z ##[group]Initializing the repository
2021-07-16T18:05:09.8570055Z [command]/usr/bin/git init /home/runner/work/submariner-operator/submariner-operator
2021-07-16T18:05:09.8660843Z hint: Using 'master' as the name for the initial branch. This default branch name
2021-07-16T18:05:09.8662209Z hint: is subject to change. To configure the initial branch name to use in all
2021-07-16T18:05:09.8663421Z hint: of your new repositories, which will suppress this warning, call:
2021-07-16T18:05:09.8664351Z hint: 
2021-07-16T18:05:09.8665354Z hint: 	git config --global init.defaultBranch <name>
2021-07-16T18:05:09.8666144Z hint: 
2021-07-16T18:05:09.8667140Z hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
2021-07-16T18:05:09.8668411Z hint: 'development'. The just-created branch can be renamed via this command:
2021-07-16T18:05:09.8669263Z hint: 
2021-07-16T18:05:09.8670032Z hint: 	git branch -m <name>
2021-07-16T18:05:09.8676116Z Initialized empty Git repository in /home/runner/work/submariner-operator/submariner-operator/.git/
2021-07-16T18:05:09.8694654Z [command]/usr/bin/git remote add origin https://github.com/submariner-io/submariner-operator
2021-07-16T18:05:09.8745862Z ##[endgroup]
2021-07-16T18:05:09.8746874Z ##[group]Disabling automatic garbage collection
2021-07-16T18:05:09.8750983Z [command]/usr/bin/git config --local gc.auto 0
2021-07-16T18:05:09.8785630Z ##[endgroup]
2021-07-16T18:05:09.8790499Z ##[group]Setting up auth
2021-07-16T18:05:09.8798336Z [command]/usr/bin/git config --local --name-only --get-regexp core\.sshCommand
2021-07-16T18:05:09.8831945Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'core\.sshCommand' && git config --local --unset-all 'core.sshCommand' || :
2021-07-16T18:05:09.9213714Z [command]/usr/bin/git config --local --name-only --get-regexp http\.https\:\/\/github\.com\/\.extraheader
2021-07-16T18:05:09.9259167Z [command]/usr/bin/git submodule foreach --recursive git config --local --name-only --get-regexp 'http\.https\:\/\/github\.com\/\.extraheader' && git config --local --unset-all 'http.https://github.com/.extraheader' || :
2021-07-16T18:05:09.9500889Z [command]/usr/bin/git config --local http.https://github.com/.extraheader AUTHORIZATION: basic ***
2021-07-16T18:05:10.3012044Z ##[endgroup]
2021-07-16T18:05:10.3065588Z ##[group]Fetching the repository
2021-07-16T18:05:10.3068146Z [command]/usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +0aaed33c534ce14f1788dcfc42c5b2e0a4619827:refs/remotes/origin/devel
2021-07-16T18:05:10.3069792Z remote: Enumerating objects: 486, done.        
2021-07-16T18:05:10.3070626Z remote: Counting objects:   0% (1/486)        
2021-07-16T18:05:10.3071398Z remote: Counting objects:   1% (5/486)        
2021-07-16T18:05:10.3072155Z remote: Counting objects:   2% (10/486)        
2021-07-16T18:05:10.3072917Z remote: Counting objects:   3% (15/486)        
2021-07-16T18:05:10.3074032Z remote: Counting objects:   4% (20/486)        
2021-07-16T18:05:10.3074819Z remote: Counting objects:   5% (25/486)        
2021-07-16T18:05:10.3075584Z remote: Counting objects:   6% (30/486)        
2021-07-16T18:05:10.3076336Z remote: Counting objects:   7% (35/486)        
2021-07-16T18:05:10.3077122Z remote: Counting objects:   8% (39/486)        
2021-07-16T18:05:10.3077885Z remote: Counting objects:   9% (44/486)        
2021-07-16T18:05:10.3078639Z remote: Counting objects:  10% (49/486)        
2021-07-16T18:05:10.3079398Z remote: Counting objects:  11% (54/486)        
2021-07-16T18:05:10.3080148Z remote: Counting objects:  12% (59/486)        
2021-07-16T18:05:10.3080913Z remote: Counting objects:  13% (64/486)        
2021-07-16T18:05:10.3081674Z remote: Counting objects:  14% (69/486)        
2021-07-16T18:05:10.3082425Z remote: Counting objects:  15% (73/486)        
2021-07-16T18:05:10.3083192Z remote: Counting objects:  16% (78/486)        
2021-07-16T18:05:10.3084870Z remote: Counting objects:  17% (83/486)        
2021-07-16T18:05:10.3085646Z remote: Counting objects:  18% (88/486)        
2021-07-16T18:05:10.3086413Z remote: Counting objects:  19% (93/486)        
2021-07-16T18:05:10.3087161Z remote: Counting objects:  20% (98/486)        
2021-07-16T18:05:10.3088011Z remote: Counting objects:  21% (103/486)        
2021-07-16T18:05:10.3088668Z remote: Counting objects:  22% (107/486)        
2021-07-16T18:05:10.3089321Z remote: Counting objects:  23% (112/486)        
2021-07-16T18:05:10.3089970Z remote: Counting objects:  24% (117/486)        
2021-07-16T18:05:10.3090747Z remote: Counting objects:  25% (122/486)        
2021-07-16T18:05:10.3091516Z remote: Counting objects:  26% (127/486)        
2021-07-16T18:05:10.3092264Z remote: Counting objects:  27% (132/486)        
2021-07-16T18:05:10.3093034Z remote: Counting objects:  28% (137/486)        
2021-07-16T18:05:10.3094157Z remote: Counting objects:  29% (141/486)        
2021-07-16T18:05:10.3094928Z remote: Counting objects:  30% (146/486)        
2021-07-16T18:05:10.3095704Z remote: Counting objects:  31% (151/486)        
2021-07-16T18:05:10.3096474Z remote: Counting objects:  32% (156/486)        
2021-07-16T18:05:10.3097227Z remote: Counting objects:  33% (161/486)        
2021-07-16T18:05:10.3097994Z remote: Counting objects:  34% (166/486)        
2021-07-16T18:05:10.3098945Z remote: Counting objects:  35% (171/486)        
2021-07-16T18:05:10.3099716Z remote: Counting objects:  36% (175/486)        
2021-07-16T18:05:10.3100481Z remote: Counting objects:  37% (180/486)        
2021-07-16T18:05:10.3101227Z remote: Counting objects:  38% (185/486)        
2021-07-16T18:05:10.3101989Z remote: Counting objects:  39% (190/486)        
2021-07-16T18:05:10.3102747Z remote: Counting objects:  40% (195/486)        
2021-07-16T18:05:10.3103513Z remote: Counting objects:  41% (200/486)        
2021-07-16T18:05:10.3104285Z remote: Counting objects:  42% (205/486)        
2021-07-16T18:05:10.3105048Z remote: Counting objects:  43% (209/486)        
2021-07-16T18:05:10.3105825Z remote: Counting objects:  44% (214/486)        
2021-07-16T18:05:10.3106595Z remote: Counting objects:  45% (219/486)        
2021-07-16T18:05:10.3107356Z remote: Counting objects:  46% (224/486)        
2021-07-16T18:05:10.3108122Z remote: Counting objects:  47% (229/486)        
2021-07-16T18:05:10.3108884Z remote: Counting objects:  48% (234/486)        
2021-07-16T18:05:10.3109661Z remote: Counting objects:  49% (239/486)        
2021-07-16T18:05:10.3110425Z remote: Counting objects:  50% (243/486)        
2021-07-16T18:05:10.3111176Z remote: Counting objects:  51% (248/486)        
2021-07-16T18:05:10.3111941Z remote: Counting objects:  52% (253/486)        
2021-07-16T18:05:10.3112693Z remote: Counting objects:  53% (258/486)        
2021-07-16T18:05:10.3113459Z remote: Counting objects:  54% (263/486)        
2021-07-16T18:05:10.3114223Z remote: Counting objects:  55% (268/486)        
2021-07-16T18:05:10.3115073Z remote: Counting objects:  56% (273/486)        
2021-07-16T18:05:10.3115864Z remote: Counting objects:  57% (278/486)        
2021-07-16T18:05:10.3116620Z remote: Counting objects:  58% (282/486)        
2021-07-16T18:05:10.3117396Z remote: Counting objects:  59% (287/486)        
2021-07-16T18:05:10.3118162Z remote: Counting objects:  60% (292/486)        
2021-07-16T18:05:10.3118930Z remote: Counting objects:  61% (297/486)        
2021-07-16T18:05:10.3119700Z remote: Counting objects:  62% (302/486)        
2021-07-16T18:05:10.3120467Z remote: Counting objects:  63% (307/486)        
2021-07-16T18:05:10.3121227Z remote: Counting objects:  64% (312/486)        
2021-07-16T18:05:10.3121995Z remote: Counting objects:  65% (316/486)        
2021-07-16T18:05:10.3122741Z remote: Counting objects:  66% (321/486)        
2021-07-16T18:05:10.3123509Z remote: Counting objects:  67% (326/486)        
2021-07-16T18:05:10.3124462Z remote: Counting objects:  68% (331/486)        
2021-07-16T18:05:10.3125212Z remote: Counting objects:  69% (336/486)        
2021-07-16T18:05:10.3125987Z remote: Counting objects:  70% (341/486)        
2021-07-16T18:05:10.3126734Z remote: Counting objects:  71% (346/486)        
2021-07-16T18:05:10.3127503Z remote: Counting objects:  72% (350/486)        
2021-07-16T18:05:10.3128268Z remote: Counting objects:  73% (355/486)        
2021-07-16T18:05:10.3129030Z remote: Counting objects:  74% (360/486)        
2021-07-16T18:05:10.3129797Z remote: Counting objects:  75% (365/486)        
2021-07-16T18:05:10.3130556Z remote: Counting objects:  76% (370/486)        
2021-07-16T18:05:10.3131323Z remote: Counting objects:  77% (375/486)        
2021-07-16T18:05:10.3132095Z remote: Counting objects:  78% (380/486)        
2021-07-16T18:05:10.3132840Z remote: Counting objects:  79% (384/486)        
2021-07-16T18:05:10.3133710Z remote: Counting objects:  80% (389/486)        
2021-07-16T18:05:10.3134483Z remote: Counting objects:  81% (394/486)        
2021-07-16T18:05:10.3135236Z remote: Counting objects:  82% (399/486)        
2021-07-16T18:05:10.3136017Z remote: Counting objects:  83% (404/486)        
2021-07-16T18:05:10.3136769Z remote: Counting objects:  84% (409/486)        
2021-07-16T18:05:10.3137533Z remote: Counting objects:  85% (414/486)        
2021-07-16T18:05:10.3138300Z remote: Counting objects:  86% (418/486)        
2021-07-16T18:05:10.3139189Z remote: Counting objects:  87% (423/486)        
2021-07-16T18:05:10.3139968Z remote: Counting objects:  88% (428/486)        
2021-07-16T18:05:10.3140714Z remote: Counting objects:  89% (433/486)        
2021-07-16T18:05:10.3141474Z remote: Counting objects:  90% (438/486)        
2021-07-16T18:05:10.3142240Z remote: Counting objects:  91% (443/486)        
2021-07-16T18:05:10.3142991Z remote: Counting objects:  92% (448/486)        
2021-07-16T18:05:10.3143878Z remote: Counting objects:  93% (452/486)        
2021-07-16T18:05:10.3144690Z remote: Counting objects:  94% (457/486)        
2021-07-16T18:05:10.3145445Z remote: Counting objects:  95% (462/486)        
2021-07-16T18:05:10.3146221Z remote: Counting objects:  96% (467/486)        
2021-07-16T18:05:10.3146989Z remote: Counting objects:  97% (472/486)        
2021-07-16T18:05:10.3147744Z remote: Counting objects:  98% (477/486)        
2021-07-16T18:05:10.3148507Z remote: Counting objects:  99% (482/486)        
2021-07-16T18:05:10.3149262Z remote: Counting objects: 100% (486/486)        
2021-07-16T18:05:10.3150317Z remote: Counting objects: 100% (486/486), done.        
2021-07-16T18:05:10.3151163Z remote: Compressing objects:   0% (1/351)        
2021-07-16T18:05:10.3157670Z remote: Compressing objects:   1% (4/351)        
2021-07-16T18:05:10.3158527Z remote: Compressing objects:   2% (8/351)        
2021-07-16T18:05:10.3159366Z remote: Compressing objects:   3% (11/351)        
2021-07-16T18:05:10.3160178Z remote: Compressing objects:   4% (15/351)        
2021-07-16T18:05:10.3161001Z remote: Compressing objects:   5% (18/351)        
2021-07-16T18:05:10.3161824Z remote: Compressing objects:   6% (22/351)        
2021-07-16T18:05:10.3162771Z remote: Compressing objects:   7% (25/351)        
2021-07-16T18:05:10.3164398Z remote: Compressing objects:   8% (29/351)        
2021-07-16T18:05:10.3165226Z remote: Compressing objects:   9% (32/351)        
2021-07-16T18:05:10.3166045Z remote: Compressing objects:  10% (36/351)        
2021-07-16T18:05:10.3166871Z remote: Compressing objects:  11% (39/351)        
2021-07-16T18:05:10.3167669Z remote: Compressing objects:  12% (43/351)        
2021-07-16T18:05:10.3168484Z remote: Compressing objects:  13% (46/351)        
2021-07-16T18:05:10.3169279Z remote: Compressing objects:  14% (50/351)        
2021-07-16T18:05:10.3170091Z remote: Compressing objects:  15% (53/351)        
2021-07-16T18:05:10.3170902Z remote: Compressing objects:  16% (57/351)        
2021-07-16T18:05:10.3171707Z remote: Compressing objects:  17% (60/351)        
2021-07-16T18:05:10.3172521Z remote: Compressing objects:  18% (64/351)        
2021-07-16T18:05:10.3173342Z remote: Compressing objects:  19% (67/351)        
2021-07-16T18:05:10.3174142Z remote: Compressing objects:  20% (71/351)        
2021-07-16T18:05:10.3174950Z remote: Compressing objects:  21% (74/351)        
2021-07-16T18:05:10.3175742Z remote: Compressing objects:  22% (78/351)        
2021-07-16T18:05:10.3176550Z remote: Compressing objects:  23% (81/351)        
2021-07-16T18:05:10.3177365Z remote: Compressing objects:  24% (85/351)        
2021-07-16T18:05:10.3178166Z remote: Compressing objects:  25% (88/351)        
2021-07-16T18:05:10.3178972Z remote: Compressing objects:  26% (92/351)        
2021-07-16T18:05:10.3179770Z remote: Compressing objects:  27% (95/351)        
2021-07-16T18:05:10.3180582Z remote: Compressing objects:  28% (99/351)        
2021-07-16T18:05:10.3181392Z remote: Compressing objects:  29% (102/351)        
2021-07-16T18:05:10.3182199Z remote: Compressing objects:  30% (106/351)        
2021-07-16T18:05:10.3183015Z remote: Compressing objects:  31% (109/351)        
2021-07-16T18:05:10.3183844Z remote: Compressing objects:  32% (113/351)        
2021-07-16T18:05:10.3184642Z remote: Compressing objects:  33% (116/351)        
2021-07-16T18:05:10.3185520Z remote: Compressing objects:  34% (120/351)        
2021-07-16T18:05:10.3186317Z remote: Compressing objects:  35% (123/351)        
2021-07-16T18:05:10.3187137Z remote: Compressing objects:  36% (127/351)        
2021-07-16T18:05:10.3188090Z remote: Compressing objects:  37% (130/351)        
2021-07-16T18:05:10.3188888Z remote: Compressing objects:  38% (134/351)        
2021-07-16T18:05:10.3189697Z remote: Compressing objects:  39% (137/351)        
2021-07-16T18:05:10.3190499Z remote: Compressing objects:  40% (141/351)        
2021-07-16T18:05:10.3191311Z remote: Compressing objects:  41% (144/351)        
2021-07-16T18:05:10.3192053Z remote: Compressing objects:  42% (148/351)        
2021-07-16T18:05:10.3192980Z remote: Compressing objects:  43% (151/351)        
2021-07-16T18:05:10.3193818Z remote: Compressing objects:  44% (155/351)        
2021-07-16T18:05:10.3194626Z remote: Compressing objects:  45% (158/351)        
2021-07-16T18:05:10.3195450Z remote: Compressing objects:  46% (162/351)        
2021-07-16T18:05:10.3196265Z remote: Compressing objects:  47% (165/351)        
2021-07-16T18:05:10.3197062Z remote: Compressing objects:  48% (169/351)        
2021-07-16T18:05:10.3197884Z remote: Compressing objects:  49% (172/351)        
2021-07-16T18:05:10.3198700Z remote: Compressing objects:  50% (176/351)        
2021-07-16T18:05:10.3199502Z remote: Compressing objects:  51% (180/351)        
2021-07-16T18:05:10.3200315Z remote: Compressing objects:  52% (183/351)        
2021-07-16T18:05:10.3201112Z remote: Compressing objects:  53% (187/351)        
2021-07-16T18:05:10.3201924Z remote: Compressing objects:  54% (190/351)        
2021-07-16T18:05:10.3202740Z remote: Compressing objects:  55% (194/351)        
2021-07-16T18:05:10.3203679Z remote: Compressing objects:  56% (197/351)        
2021-07-16T18:05:10.3204510Z remote: Compressing objects:  57% (201/351)        
2021-07-16T18:05:10.3205425Z remote: Compressing objects:  58% (204/351)        
2021-07-16T18:05:10.3206268Z remote: Compressing objects:  59% (208/351)        
2021-07-16T18:05:10.3207080Z remote: Compressing objects:  60% (211/351)        
2021-07-16T18:05:10.3207882Z remote: Compressing objects:  61% (215/351)        
2021-07-16T18:05:10.3208705Z remote: Compressing objects:  62% (218/351)        
2021-07-16T18:05:10.3209536Z remote: Compressing objects:  63% (222/351)        
2021-07-16T18:05:10.3210350Z remote: Compressing objects:  64% (225/351)        
2021-07-16T18:05:10.3211163Z remote: Compressing objects:  65% (229/351)        
2021-07-16T18:05:10.3211960Z remote: Compressing objects:  66% (232/351)        
2021-07-16T18:05:10.3212775Z remote: Compressing objects:  67% (236/351)        
2021-07-16T18:05:10.3213593Z remote: Compressing objects:  68% (239/351)        
2021-07-16T18:05:10.3214391Z remote: Compressing objects:  69% (243/351)        
2021-07-16T18:05:10.3215214Z remote: Compressing objects:  70% (246/351)        
2021-07-16T18:05:10.3216026Z remote: Compressing objects:  71% (250/351)        
2021-07-16T18:05:10.3216836Z remote: Compressing objects:  72% (253/351)        
2021-07-16T18:05:10.3217655Z remote: Compressing objects:  73% (257/351)        
2021-07-16T18:05:10.3218452Z remote: Compressing objects:  74% (260/351)        
2021-07-16T18:05:10.3219272Z remote: Compressing objects:  75% (264/351)        
2021-07-16T18:05:10.3220069Z remote: Compressing objects:  76% (267/351)        
2021-07-16T18:05:10.3220880Z remote: Compressing objects:  77% (271/351)        
2021-07-16T18:05:10.3221689Z remote: Compressing objects:  78% (274/351)        
2021-07-16T18:05:10.3222481Z remote: Compressing objects:  79% (278/351)        
2021-07-16T18:05:10.3223298Z remote: Compressing objects:  80% (281/351)        
2021-07-16T18:05:10.3224113Z remote: Compressing objects:  81% (285/351)        
2021-07-16T18:05:10.3224912Z remote: Compressing objects:  82% (288/351)        
2021-07-16T18:05:10.3225734Z remote: Compressing objects:  83% (292/351)        
2021-07-16T18:05:10.3226529Z remote: Compressing objects:  84% (295/351)        
2021-07-16T18:05:10.3227346Z remote: Compressing objects:  85% (299/351)        
2021-07-16T18:05:10.3228161Z remote: Compressing objects:  86% (302/351)        
2021-07-16T18:05:10.3229098Z remote: Compressing objects:  87% (306/351)        
2021-07-16T18:05:10.3229912Z remote: Compressing objects:  88% (309/351)        
2021-07-16T18:05:10.3230705Z remote: Compressing objects:  89% (313/351)        
2021-07-16T18:05:10.3231521Z remote: Compressing objects:  90% (316/351)        
2021-07-16T18:05:10.3232346Z remote: Compressing objects:  91% (320/351)        
2021-07-16T18:05:10.3233145Z remote: Compressing objects:  92% (323/351)        
2021-07-16T18:05:10.3233956Z remote: Compressing objects:  93% (327/351)        
2021-07-16T18:05:10.3234773Z remote: Compressing objects:  94% (330/351)        
2021-07-16T18:05:10.3235589Z remote: Compressing objects:  95% (334/351)        
2021-07-16T18:05:10.3236407Z remote: Compressing objects:  96% (337/351)        
2021-07-16T18:05:10.3237215Z remote: Compressing objects:  97% (341/351)        
2021-07-16T18:05:10.3238028Z remote: Compressing objects:  98% (344/351)        
2021-07-16T18:05:10.3238838Z remote: Compressing objects:  99% (348/351)        
2021-07-16T18:05:10.3239652Z remote: Compressing objects: 100% (351/351)        
2021-07-16T18:05:10.3240492Z remote: Compressing objects: 100% (351/351), done.        
2021-07-16T18:05:10.3241263Z Receiving objects:   0% (1/486)
2021-07-16T18:05:10.3241970Z Receiving objects:   1% (5/486)
2021-07-16T18:05:10.3242678Z Receiving objects:   2% (10/486)
2021-07-16T18:05:10.3243375Z Receiving objects:   3% (15/486)
2021-07-16T18:05:10.3244218Z Receiving objects:   4% (20/486)
2021-07-16T18:05:10.3244952Z Receiving objects:   5% (25/486)
2021-07-16T18:05:10.3245659Z Receiving objects:   6% (30/486)
2021-07-16T18:05:10.3246373Z Receiving objects:   7% (35/486)
2021-07-16T18:05:10.3247174Z Receiving objects:   8% (39/486)
2021-07-16T18:05:10.3247908Z Receiving objects:   9% (44/486)
2021-07-16T18:05:10.3248609Z Receiving objects:  10% (49/486)
2021-07-16T18:05:10.3249315Z Receiving objects:  11% (54/486)
2021-07-16T18:05:10.3250025Z Receiving objects:  12% (59/486)
2021-07-16T18:05:10.3250717Z Receiving objects:  13% (64/486)
2021-07-16T18:05:10.3251431Z Receiving objects:  14% (69/486)
2021-07-16T18:05:10.3252122Z Receiving objects:  15% (73/486)
2021-07-16T18:05:10.3252833Z Receiving objects:  16% (78/486)
2021-07-16T18:05:10.3253564Z Receiving objects:  17% (83/486)
2021-07-16T18:05:10.3254272Z Receiving objects:  18% (88/486)
2021-07-16T18:05:10.3254967Z Receiving objects:  19% (93/486)
2021-07-16T18:05:10.3255678Z Receiving objects:  20% (98/486)
2021-07-16T18:05:10.3256386Z Receiving objects:  21% (103/486)
2021-07-16T18:05:10.3257089Z Receiving objects:  22% (107/486)
2021-07-16T18:05:10.3257801Z Receiving objects:  23% (112/486)
2021-07-16T18:05:10.3258503Z Receiving objects:  24% (117/486)
2021-07-16T18:05:10.3259217Z Receiving objects:  25% (122/486)
2021-07-16T18:05:10.3259932Z Receiving objects:  26% (127/486)
2021-07-16T18:05:10.3260625Z Receiving objects:  27% (132/486)
2021-07-16T18:05:10.3261337Z Receiving objects:  28% (137/486)
2021-07-16T18:05:10.3262028Z Receiving objects:  29% (141/486)
2021-07-16T18:05:10.3262771Z Receiving objects:  30% (146/486)
2021-07-16T18:05:10.3263488Z Receiving objects:  31% (151/486)
2021-07-16T18:05:10.3264185Z Receiving objects:  32% (156/486)
2021-07-16T18:05:10.3264900Z Receiving objects:  33% (161/486)
2021-07-16T18:05:10.3265593Z Receiving objects:  34% (166/486)
2021-07-16T18:05:10.3266310Z Receiving objects:  35% (171/486)
2021-07-16T18:05:10.3267019Z Receiving objects:  36% (175/486)
2021-07-16T18:05:10.3267723Z Receiving objects:  37% (180/486)
2021-07-16T18:05:10.3268435Z Receiving objects:  38% (185/486)
2021-07-16T18:05:10.3269130Z Receiving objects:  39% (190/486)
2021-07-16T18:05:10.3269846Z Receiving objects:  40% (195/486)
2021-07-16T18:05:10.3270568Z Receiving objects:  41% (200/486)
2021-07-16T18:05:10.3271270Z Receiving objects:  42% (205/486)
2021-07-16T18:05:10.3271980Z Receiving objects:  43% (209/486)
2021-07-16T18:05:10.3272671Z Receiving objects:  44% (214/486)
2021-07-16T18:05:10.3273379Z Receiving objects:  45% (219/486)
2021-07-16T18:05:10.3274203Z Receiving objects:  46% (224/486)
2021-07-16T18:05:10.3274913Z Receiving objects:  47% (229/486)
2021-07-16T18:05:10.3275618Z Receiving objects:  48% (234/486)
2021-07-16T18:05:10.3276321Z Receiving objects:  49% (239/486)
2021-07-16T18:05:10.3277029Z Receiving objects:  50% (243/486)
2021-07-16T18:05:10.3277739Z Receiving objects:  51% (248/486)
2021-07-16T18:05:10.3278463Z Receiving objects:  52% (253/486)
2021-07-16T18:05:10.3279173Z Receiving objects:  53% (258/486)
2021-07-16T18:05:10.3279868Z Receiving objects:  54% (263/486)
2021-07-16T18:05:10.3280578Z Receiving objects:  55% (268/486)
2021-07-16T18:05:10.3281287Z Receiving objects:  56% (273/486)
2021-07-16T18:05:10.3281988Z Receiving objects:  57% (278/486)
2021-07-16T18:05:10.3282697Z Receiving objects:  58% (282/486)
2021-07-16T18:05:10.3283399Z Receiving objects:  59% (287/486)
2021-07-16T18:05:10.3284250Z Receiving objects:  60% (292/486)
2021-07-16T18:05:10.3284960Z Receiving objects:  61% (297/486)
2021-07-16T18:05:10.3285657Z Receiving objects:  62% (302/486)
2021-07-16T18:05:10.3286373Z Receiving objects:  63% (307/486)
2021-07-16T18:05:10.3287067Z Receiving objects:  64% (312/486)
2021-07-16T18:05:10.3287779Z Receiving objects:  65% (316/486)
2021-07-16T18:05:10.3288490Z Receiving objects:  66% (321/486)
2021-07-16T18:05:10.3289180Z Receiving objects:  67% (326/486)
2021-07-16T18:05:10.3289889Z Receiving objects:  68% (331/486)
2021-07-16T18:05:10.3290580Z Receiving objects:  69% (336/486)
2021-07-16T18:05:10.3291293Z Receiving objects:  70% (341/486)
2021-07-16T18:05:10.3291999Z Receiving objects:  71% (346/486)
2021-07-16T18:05:10.3292691Z Receiving objects:  72% (350/486)
2021-07-16T18:05:10.3293537Z Receiving objects:  73% (355/486)
2021-07-16T18:05:10.3294356Z Receiving objects:  74% (360/486)
2021-07-16T18:05:10.3295065Z Receiving objects:  75% (365/486)
2021-07-16T18:05:10.3295777Z Receiving objects:  76% (370/486)
2021-07-16T18:05:10.3296469Z Receiving objects:  77% (375/486)
2021-07-16T18:05:10.3297180Z Receiving objects:  78% (380/486)
2021-07-16T18:05:10.3297882Z Receiving objects:  79% (384/486)
2021-07-16T18:05:10.3298593Z Receiving objects:  80% (389/486)
2021-07-16T18:05:10.3299300Z Receiving objects:  81% (394/486)
2021-07-16T18:05:10.3299996Z Receiving objects:  82% (399/486)
2021-07-16T18:05:10.3300700Z Receiving objects:  83% (404/486)
2021-07-16T18:05:10.3301391Z Receiving objects:  84% (409/486)
2021-07-16T18:05:10.3302103Z Receiving objects:  85% (414/486)
2021-07-16T18:05:10.3302818Z Receiving objects:  86% (418/486)
2021-07-16T18:05:10.3303510Z Receiving objects:  87% (423/486)
2021-07-16T18:05:10.3304220Z Receiving objects:  88% (428/486)
2021-07-16T18:05:10.3304922Z Receiving objects:  89% (433/486)
2021-07-16T18:05:10.3305634Z Receiving objects:  90% (438/486)
2021-07-16T18:05:10.3306343Z Receiving objects:  91% (443/486)
2021-07-16T18:05:10.3307045Z Receiving objects:  92% (448/486)
2021-07-16T18:05:10.3307759Z Receiving objects:  93% (452/486)
2021-07-16T18:05:10.3309631Z Receiving objects:  94% (457/486)
2021-07-16T18:05:10.3311035Z remote: Total 486 (delta 154), reused 306 (delta 89), pack-reused 0        
2021-07-16T18:05:10.3311748Z Receiving objects:  95% (462/486)
2021-07-16T18:05:10.3312344Z Receiving objects:  96% (467/486)
2021-07-16T18:05:10.3312938Z Receiving objects:  97% (472/486)
2021-07-16T18:05:10.3313519Z Receiving objects:  98% (477/486)
2021-07-16T18:05:10.3314109Z Receiving objects:  99% (482/486)
2021-07-16T18:05:10.3314687Z Receiving objects: 100% (486/486)
2021-07-16T18:05:10.3315354Z Receiving objects: 100% (486/486), 349.04 KiB | 10.91 MiB/s, done.
2021-07-16T18:05:10.3315989Z Resolving deltas:   0% (0/154)
2021-07-16T18:05:10.3316822Z Resolving deltas:   1% (2/154)
2021-07-16T18:05:10.3317533Z Resolving deltas:   2% (4/154)
2021-07-16T18:05:10.3318220Z Resolving deltas:   3% (5/154)
2021-07-16T18:05:10.3318918Z Resolving deltas:   4% (7/154)
2021-07-16T18:05:10.3319704Z Resolving deltas:   5% (8/154)
2021-07-16T18:05:10.3320422Z Resolving deltas:   6% (10/154)
2021-07-16T18:05:10.3321113Z Resolving deltas:   7% (11/154)
2021-07-16T18:05:10.3321989Z Resolving deltas:   8% (13/154)
2021-07-16T18:05:10.3322693Z Resolving deltas:   9% (14/154)
2021-07-16T18:05:10.3323374Z Resolving deltas:  10% (16/154)
2021-07-16T18:05:10.3324278Z Resolving deltas:  11% (17/154)
2021-07-16T18:05:10.3324963Z Resolving deltas:  12% (19/154)
2021-07-16T18:05:10.3325662Z Resolving deltas:  13% (21/154)
2021-07-16T18:05:10.3326362Z Resolving deltas:  14% (22/154)
2021-07-16T18:05:10.3327043Z Resolving deltas:  15% (24/154)
2021-07-16T18:05:10.3327735Z Resolving deltas:  16% (25/154)
2021-07-16T18:05:10.3328417Z Resolving deltas:  17% (27/154)
2021-07-16T18:05:10.3329135Z Resolving deltas:  18% (28/154)
2021-07-16T18:05:10.3329834Z Resolving deltas:  19% (30/154)
2021-07-16T18:05:10.3330516Z Resolving deltas:  20% (31/154)
2021-07-16T18:05:10.3331213Z Resolving deltas:  21% (33/154)
2021-07-16T18:05:10.3331890Z Resolving deltas:  22% (34/154)
2021-07-16T18:05:10.3332517Z Resolving deltas:  23% (36/154)
2021-07-16T18:05:10.3333221Z Resolving deltas:  24% (37/154)
2021-07-16T18:05:10.3333909Z Resolving deltas:  25% (39/154)
2021-07-16T18:05:10.3334604Z Resolving deltas:  26% (41/154)
2021-07-16T18:05:10.3335286Z Resolving deltas:  27% (42/154)
2021-07-16T18:05:10.3335984Z Resolving deltas:  28% (44/154)
2021-07-16T18:05:10.3336678Z Resolving deltas:  29% (45/154)
2021-07-16T18:05:10.3337358Z Resolving deltas:  30% (47/154)
2021-07-16T18:05:10.3338053Z Resolving deltas:  31% (48/154)
2021-07-16T18:05:10.3338729Z Resolving deltas:  32% (50/154)
2021-07-16T18:05:10.3339434Z Resolving deltas:  33% (51/154)
2021-07-16T18:05:10.3340125Z Resolving deltas:  34% (53/154)
2021-07-16T18:05:10.3340910Z Resolving deltas:  35% (54/154)
2021-07-16T18:05:10.3341628Z Resolving deltas:  36% (56/154)
2021-07-16T18:05:10.3342310Z Resolving deltas:  37% (57/154)
2021-07-16T18:05:10.3343012Z Resolving deltas:  38% (59/154)
2021-07-16T18:05:10.3343711Z Resolving deltas:  39% (61/154)
2021-07-16T18:05:10.3344385Z Resolving deltas:  40% (62/154)
2021-07-16T18:05:10.3345127Z Resolving deltas:  41% (64/154)
2021-07-16T18:05:10.3345814Z Resolving deltas:  42% (65/154)
2021-07-16T18:05:10.3346507Z Resolving deltas:  43% (67/154)
2021-07-16T18:05:10.3347201Z Resolving deltas:  44% (68/154)
2021-07-16T18:05:10.3347885Z Resolving deltas:  45% (70/154)
2021-07-16T18:05:10.3348579Z Resolving deltas:  46% (71/154)
2021-07-16T18:05:10.3349259Z Resolving deltas:  47% (73/154)
2021-07-16T18:05:10.3349963Z Resolving deltas:  48% (74/154)
2021-07-16T18:05:10.3350658Z Resolving deltas:  49% (76/154)
2021-07-16T18:05:10.3351333Z Resolving deltas:  50% (77/154)
2021-07-16T18:05:10.3352025Z Resolving deltas:  51% (79/154)
2021-07-16T18:05:10.3352714Z Resolving deltas:  52% (81/154)
2021-07-16T18:05:10.3353410Z Resolving deltas:  53% (82/154)
2021-07-16T18:05:10.3354173Z Resolving deltas:  54% (84/154)
2021-07-16T18:05:10.3354851Z Resolving deltas:  55% (85/154)
2021-07-16T18:05:10.3355697Z Resolving deltas:  56% (87/154)
2021-07-16T18:05:10.3356519Z Resolving deltas:  57% (88/154)
2021-07-16T18:05:10.3357431Z Resolving deltas:  58% (90/154)
2021-07-16T18:05:10.3358135Z Resolving deltas:  59% (91/154)
2021-07-16T18:05:10.3358820Z Resolving deltas:  60% (93/154)
2021-07-16T18:05:10.3359516Z Resolving deltas:  61% (94/154)
2021-07-16T18:05:10.3360200Z Resolving deltas:  62% (96/154)
2021-07-16T18:05:10.3360896Z Resolving deltas:  63% (98/154)
2021-07-16T18:05:10.3361589Z Resolving deltas:  64% (99/154)
2021-07-16T18:05:10.3362270Z Resolving deltas:  65% (101/154)
2021-07-16T18:05:10.3362977Z Resolving deltas:  66% (102/154)
2021-07-16T18:05:10.3363828Z Resolving deltas:  67% (104/154)
2021-07-16T18:05:10.3364552Z Resolving deltas:  68% (105/154)
2021-07-16T18:05:10.3365280Z Resolving deltas:  69% (107/154)
2021-07-16T18:05:10.3365962Z Resolving deltas:  70% (108/154)
2021-07-16T18:05:10.3366711Z Resolving deltas:  71% (110/154)
2021-07-16T18:05:10.3367404Z Resolving deltas:  72% (111/154)
2021-07-16T18:05:10.3368105Z Resolving deltas:  73% (113/154)
2021-07-16T18:05:10.3368803Z Resolving deltas:  74% (114/154)
2021-07-16T18:05:10.3369651Z Resolving deltas:  75% (116/154)
2021-07-16T18:05:10.3370354Z Resolving deltas:  76% (118/154)
2021-07-16T18:05:10.3371043Z Resolving deltas:  77% (119/154)
2021-07-16T18:05:10.3371748Z Resolving deltas:  78% (121/154)
2021-07-16T18:05:10.3372448Z Resolving deltas:  79% (122/154)
2021-07-16T18:05:10.3373136Z Resolving deltas:  80% (124/154)
2021-07-16T18:05:10.3373840Z Resolving deltas:  81% (125/154)
2021-07-16T18:05:10.3374547Z Resolving deltas:  82% (127/154)
2021-07-16T18:05:10.3375245Z Resolving deltas:  83% (128/154)
2021-07-16T18:05:10.3375948Z Resolving deltas:  84% (130/154)
2021-07-16T18:05:10.3376645Z Resolving deltas:  85% (131/154)
2021-07-16T18:05:10.3377348Z Resolving deltas:  86% (133/154)
2021-07-16T18:05:10.3378038Z Resolving deltas:  87% (134/154)
2021-07-16T18:05:10.3378738Z Resolving deltas:  88% (136/154)
2021-07-16T18:05:10.3379440Z Resolving deltas:  89% (138/154)
2021-07-16T18:05:10.3380127Z Resolving deltas:  90% (139/154)
2021-07-16T18:05:10.3380844Z Resolving deltas:  91% (141/154)
2021-07-16T18:05:10.3381529Z Resolving deltas:  92% (142/154)
2021-07-16T18:05:10.3382229Z Resolving deltas:  93% (144/154)
2021-07-16T18:05:10.3382926Z Resolving deltas:  94% (145/154)
2021-07-16T18:05:10.3383608Z Resolving deltas:  95% (147/154)
2021-07-16T18:05:10.3384313Z Resolving deltas:  96% (148/154)
2021-07-16T18:05:10.3384997Z Resolving deltas:  97% (150/154)
2021-07-16T18:05:10.3385696Z Resolving deltas:  98% (151/154)
2021-07-16T18:05:10.3386394Z Resolving deltas:  99% (153/154)
2021-07-16T18:05:10.3387075Z Resolving deltas: 100% (154/154)
2021-07-16T18:05:10.3387800Z Resolving deltas: 100% (154/154), done.
2021-07-16T18:05:10.3389392Z From https://github.com/submariner-io/submariner-operator
2021-07-16T18:05:10.3390968Z  * [new ref]         0aaed33c534ce14f1788dcfc42c5b2e0a4619827 -> origin/devel
2021-07-16T18:05:10.3392105Z ##[endgroup]
2021-07-16T18:05:10.3392859Z ##[group]Determining the checkout info
2021-07-16T18:05:10.3393600Z ##[endgroup]
2021-07-16T18:05:10.3394401Z ##[group]Checking out the ref
2021-07-16T18:05:10.3395606Z [command]/usr/bin/git checkout --progress --force -B devel refs/remotes/origin/devel
2021-07-16T18:05:10.3396728Z Switched to a new branch 'devel'
2021-07-16T18:05:10.3397793Z Branch 'devel' set up to track remote branch 'devel' from 'origin'.
2021-07-16T18:05:10.3398622Z ##[endgroup]
2021-07-16T18:05:10.3399511Z [command]/usr/bin/git log -1 --format='%H'
2021-07-16T18:05:10.3400651Z '0aaed33c534ce14f1788dcfc42c5b2e0a4619827'
2021-07-16T18:05:10.3623538Z ##[group]Run submariner-io/shipyard/gh-actions/e2e@devel
2021-07-16T18:05:10.3624241Z with:
2021-07-16T18:05:10.3624726Z   k8s_version: 1.18
2021-07-16T18:05:10.3625192Z   using:   
2021-07-16T18:05:10.3625739Z   working-directory: .
2021-07-16T18:05:10.3626289Z   target: e2e
2021-07-16T18:05:10.3626762Z   focus: \[.*\]
2021-07-16T18:05:10.3627336Z   skip: \[external-dataplane\]
2021-07-16T18:05:10.3627955Z   testdir: test/e2e
2021-07-16T18:05:10.3628455Z ##[endgroup]
2021-07-16T18:05:10.3809995Z ##[group]Reclaiming free space
2021-07-16T18:05:26.8553875Z Filesystem      Size  Used Avail Use% Mounted on
2021-07-16T18:05:26.8569611Z /dev/root        84G   42G   43G  50% /
2021-07-16T18:05:26.8570619Z devtmpfs        3.4G     0  3.4G   0% /dev
2021-07-16T18:05:26.8571731Z tmpfs           3.4G   12K  3.4G   1% /dev/shm
2021-07-16T18:05:26.8572643Z tmpfs           696M  1.1M  695M   1% /run
2021-07-16T18:05:26.8573640Z tmpfs           5.0M     0  5.0M   0% /run/lock
2021-07-16T18:05:26.8574852Z tmpfs           3.4G     0  3.4G   0% /sys/fs/cgroup
2021-07-16T18:05:26.8575747Z /dev/loop0       56M   56M     0 100% /snap/core18/2066
2021-07-16T18:05:26.8576800Z /dev/loop1       56M   56M     0 100% /snap/core18/2074
2021-07-16T18:05:26.8577574Z /dev/loop2       68M   68M     0 100% /snap/lxd/20326
2021-07-16T18:05:26.8578384Z /dev/loop3       33M   33M     0 100% /snap/snapd/12159
2021-07-16T18:05:26.8579241Z /dev/loop4       33M   33M     0 100% /snap/snapd/12398
2021-07-16T18:05:26.8580206Z /dev/sda15      105M  7.9M   97M   8% /boot/efi
2021-07-16T18:05:26.8580997Z /dev/sdb1        14G  4.1G  9.0G  32% /mnt
2021-07-16T18:05:26.8582296Z ##[endgroup]
2021-07-16T18:05:26.8748560Z ##[group]Disable swap
2021-07-16T18:05:26.9069896Z ##[endgroup]
2021-07-16T18:05:26.9215465Z ##[group]Report available RAM
2021-07-16T18:05:26.9365873Z               total        used        free      shared  buff/cache   available
2021-07-16T18:05:26.9366786Z Mem:          6.8Gi       471Mi       5.3Gi       7.0Mi       1.0Gi       6.0Gi
2021-07-16T18:05:26.9367370Z Swap:            0B          0B          0B
2021-07-16T18:05:26.9367921Z ##[endgroup]
2021-07-16T18:05:26.9759622Z Downloading Makefile.dapper
2021-07-16T18:05:26.9996198Z Downloading dapper
2021-07-16T18:05:27.1201852Z .dapper.tmp version v0.5.6
2021-07-16T18:05:27.1249679Z Downloading Dockerfile.dapper
2021-07-16T18:05:27.1629295Z docker network create -d bridge kind
2021-07-16T18:05:27.5440884Z 31a67e7c1804526cd087b649bbb3d3656bc9d81174a1ba5f96e9288fc2e34922
2021-07-16T18:05:27.5471288Z ./.dapper -m bind  make --debug=b e2e
2021-07-16T18:05:27.7873744Z Sending build context to Docker daemon  7.517MB
2021-07-16T18:05:27.7874437Z 
2021-07-16T18:05:27.7981210Z Step 1/8 : ARG BASE_BRANCH
2021-07-16T18:05:27.7984581Z Step 2/8 : FROM quay.io/submariner/shipyard-dapper-base:${BASE_BRANCH}
2021-07-16T18:05:28.0974278Z devel: Pulling from submariner/shipyard-dapper-base
2021-07-16T18:05:28.0983476Z b1495d80d526: Pulling fs layer
2021-07-16T18:05:28.0984900Z fb3037ce215d: Pulling fs layer
2021-07-16T18:05:28.0986434Z 95344349df1d: Pulling fs layer
2021-07-16T18:05:28.0987948Z a3e11b0fdbf6: Pulling fs layer
2021-07-16T18:05:28.0989399Z b2d0ef6cbe4f: Pulling fs layer
2021-07-16T18:05:28.0990892Z fa188fe30729: Pulling fs layer
2021-07-16T18:05:28.0992330Z 05ca4b0545ef: Pulling fs layer
2021-07-16T18:05:28.0993782Z b17ce80b21da: Pulling fs layer
2021-07-16T18:05:28.0996983Z b2d0ef6cbe4f: Waiting
2021-07-16T18:05:28.1000092Z fa188fe30729: Waiting
2021-07-16T18:05:28.1000610Z 05ca4b0545ef: Waiting
2021-07-16T18:05:28.1001059Z b17ce80b21da: Waiting
2021-07-16T18:05:28.1001898Z a3e11b0fdbf6: Waiting
2021-07-16T18:05:28.7722495Z b1495d80d526: Verifying Checksum
2021-07-16T18:05:28.7773499Z b1495d80d526: Download complete
2021-07-16T18:05:28.9082741Z a3e11b0fdbf6: Verifying Checksum
2021-07-16T18:05:28.9142344Z a3e11b0fdbf6: Download complete
2021-07-16T18:05:28.9473488Z 95344349df1d: Verifying Checksum
2021-07-16T18:05:28.9474180Z 95344349df1d: Download complete
2021-07-16T18:05:29.0232723Z b2d0ef6cbe4f: Verifying Checksum
2021-07-16T18:05:29.0233769Z b2d0ef6cbe4f: Download complete
2021-07-16T18:05:29.0468719Z fa188fe30729: Verifying Checksum
2021-07-16T18:05:29.0469675Z fa188fe30729: Download complete
2021-07-16T18:05:29.1592567Z 05ca4b0545ef: Verifying Checksum
2021-07-16T18:05:29.1593176Z 05ca4b0545ef: Download complete
2021-07-16T18:05:29.1627449Z b17ce80b21da: Verifying Checksum
2021-07-16T18:05:29.1628057Z b17ce80b21da: Download complete
2021-07-16T18:05:31.5270841Z fb3037ce215d: Verifying Checksum
2021-07-16T18:05:31.5273006Z fb3037ce215d: Download complete
2021-07-16T18:05:33.1747463Z b1495d80d526: Pull complete
2021-07-16T18:05:44.1701741Z fb3037ce215d: Pull complete
2021-07-16T18:05:45.6743650Z 95344349df1d: Pull complete
2021-07-16T18:05:45.7543337Z a3e11b0fdbf6: Pull complete
2021-07-16T18:05:45.8470374Z b2d0ef6cbe4f: Pull complete
2021-07-16T18:05:46.0229144Z fa188fe30729: Pull complete
2021-07-16T18:05:46.0953580Z 05ca4b0545ef: Pull complete
2021-07-16T18:05:46.1859566Z b17ce80b21da: Pull complete
2021-07-16T18:05:46.1916292Z Digest: sha256:29762595cdd1efd22ed179cd45188c3fef4adc232f27415f2ad4f15e621053b8
2021-07-16T18:05:46.1938876Z Status: Downloaded newer image for quay.io/submariner/shipyard-dapper-base:devel
2021-07-16T18:05:46.1953468Z  ---> 92b726655a41
2021-07-16T18:05:46.1953950Z Step 3/8 : ARG PROJECT
2021-07-16T18:05:46.2142315Z  ---> Running in 7011b009c91f
2021-07-16T18:05:47.2172713Z Removing intermediate container 7011b009c91f
2021-07-16T18:05:47.2174190Z  ---> c6ddb3a50a8c
2021-07-16T18:05:47.2175902Z Step 4/8 : ENV DAPPER_ENV="QUAY_USERNAME QUAY_PASSWORD CLUSTERS_ARGS DEPLOY_ARGS CLEANUP_ARGS E2E_ARGS RELEASE_ARGS MAKEFLAGS FOCUS SKIP PLUGIN E2E_TESTDIR"     DAPPER_SOURCE=/go/src/github.com/submariner-io/${PROJECT} DAPPER_DOCKER_SOCKET=true
2021-07-16T18:05:47.2390980Z  ---> Running in 134afe9afecc
2021-07-16T18:05:48.2200077Z Removing intermediate container 134afe9afecc
2021-07-16T18:05:48.2226659Z  ---> 6667def5332c
2021-07-16T18:05:48.2227676Z Step 5/8 : ENV DAPPER_OUTPUT=${DAPPER_SOURCE}/output
2021-07-16T18:05:48.2394504Z  ---> Running in aefb0027a937
2021-07-16T18:05:49.2674119Z Removing intermediate container aefb0027a937
2021-07-16T18:05:49.2678664Z  ---> 38bff7b16840
2021-07-16T18:05:49.2679285Z Step 6/8 : WORKDIR ${DAPPER_SOURCE}
2021-07-16T18:05:49.2927783Z  ---> Running in c3fd150405e2
2021-07-16T18:05:50.2899349Z Removing intermediate container c3fd150405e2
2021-07-16T18:05:50.2900682Z  ---> a4832945514a
2021-07-16T18:05:50.2901317Z Step 7/8 : ENTRYPOINT ["/opt/shipyard/scripts/entry"]
2021-07-16T18:05:50.3065489Z  ---> Running in 24b4d0a28f1e
2021-07-16T18:05:51.3208055Z Removing intermediate container 24b4d0a28f1e
2021-07-16T18:05:51.3209420Z  ---> 041b541641fc
2021-07-16T18:05:51.3209848Z Step 8/8 : CMD ["sh"]
2021-07-16T18:05:51.3387414Z  ---> Running in 8b3bef87e99f
2021-07-16T18:05:52.3508161Z Removing intermediate container 8b3bef87e99f
2021-07-16T18:05:52.3509484Z  ---> bad85f88ee6c
2021-07-16T18:05:52.3646392Z Successfully built bad85f88ee6c
2021-07-16T18:05:52.3704578Z Successfully tagged submariner-operator:devel
2021-07-16T18:05:52.9473591Z [submariner-operator]$ trap chown -R 1001:121 . exit
2021-07-16T18:05:52.9488042Z [submariner-operator]$ mkdir -p bin dist output
2021-07-16T18:05:52.9529542Z [submariner-operator]$ make --debug=b e2e
2021-07-16T18:05:52.9554062Z GNU Make 4.3
2021-07-16T18:05:52.9555223Z Built for x86_64-redhat-linux-gnu
2021-07-16T18:05:52.9556483Z Copyright (C) 1988-2020 Free Software Foundation, Inc.
2021-07-16T18:05:52.9557837Z License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
2021-07-16T18:05:52.9563308Z This is free software: you are free to change and redistribute it.
2021-07-16T18:05:52.9564771Z There is NO WARRANTY, to the extent permitted by law.
2021-07-16T18:05:52.9565684Z Reading makefiles...
2021-07-16T18:05:53.0840679Z Makefile:94: warning: overriding recipe for target 'e2e'
2021-07-16T18:05:53.0842817Z /opt/shipyard/Makefile.inc:106: warning: ignoring old recipe for target 'e2e'
2021-07-16T18:05:53.3263939Z Updating makefiles....
2021-07-16T18:05:53.3274850Z Updating goal targets....
2021-07-16T18:05:53.3276037Z  File 'e2e' does not exist.
2021-07-16T18:05:53.3276729Z      File 'clusters' does not exist.
2021-07-16T18:05:53.3277450Z     Must remake target 'clusters'.
2021-07-16T18:05:53.3278944Z /opt/shipyard/scripts/clusters.sh --k8s_version="1.18" --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-16T18:05:53.3312804Z Downloading shflags 1.0.3
2021-07-16T18:05:53.3374648Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2021-07-16T18:05:53.3376615Z                                  Dload  Upload   Total   Spent    Left  Speed
2021-07-16T18:05:53.3377025Z 
2021-07-16T18:05:53.3588996Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
2021-07-16T18:05:53.3589847Z 100 31091  100 31091    0     0  1445k      0 --:--:-- --:--:-- --:--:-- 1445k
2021-07-16T18:05:53.4343759Z Running with: k8s_version=1.18.19, olm_version=0.14.1, olm=false, globalnet=false, registry_inmemory=true, cluster_settings=/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings, timeout=5m
2021-07-16T18:05:53.4367378Z [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
2021-07-16T18:05:53.4377783Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:05:53.4391737Z [submariner-operator]$ script_name=utils
2021-07-16T18:05:53.4402297Z [submariner-operator]$ exec_name=clusters.sh
2021-07-16T18:05:53.4423261Z [submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings
2021-07-16T18:05:53.4436976Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:05:53.4450553Z [submariner-operator]$ script_name=cluster_settings
2021-07-16T18:05:53.4460680Z [submariner-operator]$ exec_name=clusters.sh
2021-07-16T18:05:53.4482943Z [submariner-operator]$ broker=cluster1
2021-07-16T18:05:53.4495786Z [submariner-operator]$ declare -gA cluster_nodes
2021-07-16T18:05:53.4508796Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker
2021-07-16T18:05:53.4521545Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker
2021-07-16T18:05:53.4536346Z [submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker
2021-07-16T18:05:53.4548797Z [submariner-operator]$ declare -gA cluster_subm
2021-07-16T18:05:53.4561429Z [submariner-operator]$ cluster_subm[cluster1]=true
2021-07-16T18:05:53.4574752Z [submariner-operator]$ cluster_subm[cluster2]=true
2021-07-16T18:05:53.4587118Z [submariner-operator]$ cluster_subm[cluster3]=true
2021-07-16T18:05:53.4599392Z [submariner-operator]$ declare -gA cluster_cni
2021-07-16T18:05:53.4612528Z [submariner-operator]$ cluster_cni[cluster2]=weave
2021-07-16T18:05:53.4624665Z [submariner-operator]$ cluster_cni[cluster3]=weave
2021-07-16T18:05:53.4645712Z [submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-16T18:05:53.4659147Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:05:53.4672036Z [submariner-operator]$ script_name=cluster_settings
2021-07-16T18:05:53.4684958Z [submariner-operator]$ exec_name=clusters.sh
2021-07-16T18:05:53.4707830Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker
2021-07-16T18:05:53.4720859Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker
2021-07-16T18:05:53.4753131Z [submariner-operator]$ cat <<EOM
2021-07-16T18:05:53.4754483Z Cluster settings::
2021-07-16T18:05:53.4755131Z   broker - ${broker@Q}
2021-07-16T18:05:53.4755765Z   clusters - ${clusters[*]@Q}
2021-07-16T18:05:53.4756737Z   cni - $(typeset -p cluster_cni | cut -f 2- -d=)
2021-07-16T18:05:53.4757650Z   nodes per cluster - $(typeset -p cluster_nodes | cut -f 2- -d=)
2021-07-16T18:05:53.4758615Z   install submariner - $(typeset -p cluster_subm | cut -f 2- -d=)
2021-07-16T18:05:53.4759295Z EOM
2021-07-16T18:05:53.4781549Z [submariner-operator]$ typeset -p cluster_cni
2021-07-16T18:05:53.4796806Z [submariner-operator]$ cut -f 2- -d=
2021-07-16T18:05:53.4828906Z [submariner-operator]$ typeset -p cluster_nodes
2021-07-16T18:05:53.4845167Z [submariner-operator]$ cut -f 2- -d=
2021-07-16T18:05:53.4876454Z [submariner-operator]$ typeset -p cluster_subm
2021-07-16T18:05:53.4892130Z [submariner-operator]$ cut -f 2- -d=
2021-07-16T18:05:53.4915405Z Cluster settings::
2021-07-16T18:05:53.4916392Z   broker - 'cluster1'
2021-07-16T18:05:53.4917076Z   clusters - 'cluster1' 'cluster2'
2021-07-16T18:05:53.4917867Z   cni - ([cluster2]="weave" [cluster1]="weave" )
2021-07-16T18:05:53.4919291Z   nodes per cluster - ([cluster3]="control-plane worker worker" [cluster2]="control-plane worker worker" [cluster1]="control-plane worker worker" )
2021-07-16T18:05:53.4920663Z   install submariner - ([cluster2]="true" [cluster1]="true" )
2021-07-16T18:05:53.4935165Z [submariner-operator]$ rm -rf /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs
2021-07-16T18:05:53.4955550Z [submariner-operator]$ mkdir -p /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs
2021-07-16T18:05:53.4980296Z [submariner-operator]$ run_local_registry
2021-07-16T18:05:53.4992941Z [submariner-operator]$ run_local_registry
2021-07-16T18:05:53.5006492Z [submariner-operator]$ registry_running
2021-07-16T18:05:53.5016347Z [submariner-operator]$ registry_running
2021-07-16T18:05:53.5029433Z [submariner-operator]$ docker ps --filter name=^/?kind-registry$
2021-07-16T18:05:53.5044924Z [submariner-operator]$ grep kind-registry
2021-07-16T18:05:53.8021511Z [submariner-operator]$ return 0
2021-07-16T18:05:53.8035530Z Deploying local registry kind-registry to serve images centrally.
2021-07-16T18:05:53.8051982Z [submariner-operator]$ volume_flag=-v /dev/shm/kind-registry:/var/lib/registry
2021-07-16T18:05:53.8065468Z [submariner-operator]$ selinuxenabled
2021-07-16T18:05:53.8092938Z [submariner-operator]$ docker run -d -v /dev/shm/kind-registry:/var/lib/registry -p 127.0.0.1:5000:5000 --restart=always --name kind-registry registry:2
2021-07-16T18:05:54.1053684Z Unable to find image 'registry:2' locally
2021-07-16T18:05:54.2643367Z 2: Pulling from library/registry
2021-07-16T18:05:54.3256679Z ddad3d7c1e96: Already exists
2021-07-16T18:05:54.3316525Z 6eda6749503f: Pulling fs layer
2021-07-16T18:05:54.3317237Z 363ab70c2143: Pulling fs layer
2021-07-16T18:05:54.3318216Z 5b94580856e6: Pulling fs layer
2021-07-16T18:05:54.3318724Z 12008541203a: Pulling fs layer
2021-07-16T18:05:54.3319151Z 12008541203a: Waiting
2021-07-16T18:05:54.3933810Z 5b94580856e6: Verifying Checksum
2021-07-16T18:05:54.3982983Z 6eda6749503f: Verifying Checksum
2021-07-16T18:05:54.3983635Z 6eda6749503f: Download complete
2021-07-16T18:05:54.3986237Z 5b94580856e6: Download complete
2021-07-16T18:05:54.4537505Z 12008541203a: Verifying Checksum
2021-07-16T18:05:54.4538091Z 12008541203a: Download complete
2021-07-16T18:05:54.5052477Z 363ab70c2143: Verifying Checksum
2021-07-16T18:05:54.5063123Z 363ab70c2143: Download complete
2021-07-16T18:05:54.6453576Z 6eda6749503f: Pull complete
2021-07-16T18:05:55.0222523Z 363ab70c2143: Pull complete
2021-07-16T18:05:55.1024710Z 5b94580856e6: Pull complete
2021-07-16T18:05:55.1725559Z 12008541203a: Pull complete
2021-07-16T18:05:55.1771792Z Digest: sha256:aba2bfe9f0cff1ac0618ec4a54bfefb2e685bbac67c8ebaf3b6405929b3e616f
2021-07-16T18:05:55.1802242Z Status: Downloaded newer image for registry:2
2021-07-16T18:05:55.1949799Z cd34f1cd93957ff5cc98ca0cd3481f30e2c6e11f7abd51d6fbc863810e8cc081
2021-07-16T18:05:55.9595755Z [submariner-operator]$ docker network connect kind kind-registry
2021-07-16T18:05:56.3948345Z [submariner-operator]$ declare_cidrs
2021-07-16T18:05:56.3957487Z [submariner-operator]$ declare_cidrs
2021-07-16T18:05:56.3976837Z [submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
2021-07-16T18:05:56.3988724Z [submariner-operator]$ i=1
2021-07-16T18:05:56.4009073Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-16T18:05:56.4010405Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-16T18:05:56.4022724Z [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16
2021-07-16T18:05:56.4090765Z [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16
2021-07-16T18:05:56.4091873Z [submariner-operator]$ [cluster1] i=$(("$i"+1))
2021-07-16T18:05:56.4093919Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-16T18:05:56.4114433Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-16T18:05:56.4134457Z [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16
2021-07-16T18:05:56.4193612Z [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16
2021-07-16T18:05:56.4194737Z [submariner-operator]$ [cluster2] i=$(("$i"+1))
2021-07-16T18:05:56.4195927Z [submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster
2021-07-16T18:05:56.4197515Z [submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster
2021-07-16T18:05:56.4200843Z [submariner-operator]$ [cluster2] wait 229
2021-07-16T18:05:56.4202157Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 with_retries 3 create_kind_cluster
2021-07-16T18:05:56.4216096Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 with_retries 3 create_kind_cluster
2021-07-16T18:05:56.4224566Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-16T18:05:56.4256256Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-16T18:05:56.4289035Z [submariner-operator]$ [cluster1] pids[cluster1]=238
2021-07-16T18:05:56.4291169Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-16T18:05:56.4297412Z [submariner-operator]$ [cluster2] pids[cluster2]=241
2021-07-16T18:05:56.4312570Z [submariner-operator]$ [cluster1] with_retries 3 create_kind_cluster
2021-07-16T18:05:56.4318426Z [submariner-operator]$ [cluster2] wait 241
2021-07-16T18:05:56.4329020Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-16T18:05:56.4335481Z [submariner-operator]$ [cluster2] with_retries 3 create_kind_cluster
2021-07-16T18:05:56.4348085Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-16T18:05:56.4355809Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-16T18:08:07.3427557Z [submariner-operator]$ [cluster2] with_retries
2021-07-16T18:08:07.3428826Z [submariner-operator]$ [cluster2] iteration=1
2021-07-16T18:08:07.3429826Z [submariner-operator]$ [cluster2] wait 259
2021-07-16T18:08:07.3430851Z [submariner-operator]$ [cluster2] create_kind_cluster
2021-07-16T18:08:07.3431944Z [submariner-operator]$ [cluster2] create_kind_cluster
2021-07-16T18:08:07.3433693Z [submariner-operator]$ [cluster2] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:07.3435929Z [submariner-operator]$ [cluster2] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:07.3437489Z [submariner-operator]$ [cluster2] kind get clusters
2021-07-16T18:08:07.3438669Z [submariner-operator]$ [cluster2] grep -q ^cluster2$
2021-07-16T18:08:07.3439422Z [cluster2] No kind clusters found.
2021-07-16T18:08:07.3440212Z [cluster2] Creating KIND cluster...
2021-07-16T18:08:07.3441199Z [submariner-operator]$ [cluster2] generate_cluster_yaml
2021-07-16T18:08:07.3442319Z [submariner-operator]$ [cluster2] generate_cluster_yaml
2021-07-16T18:08:07.3443385Z [submariner-operator]$ [cluster2] disable_cni=true
2021-07-16T18:08:07.3444490Z [submariner-operator]$ [cluster2] nodes=
2021-07-16T18:08:07.3445341Z [cluster2] - role: control-plane
2021-07-16T18:08:07.3446224Z [submariner-operator]$ [cluster2] nodes=
2021-07-16T18:08:07.3447044Z [cluster2] - role: control-plane
2021-07-16T18:08:07.3447751Z [cluster2] - role: worker
2021-07-16T18:08:07.3448556Z [submariner-operator]$ [cluster2] nodes=
2021-07-16T18:08:07.3449391Z [cluster2] - role: control-plane
2021-07-16T18:08:07.3450065Z [cluster2] - role: worker
2021-07-16T18:08:07.3450872Z [cluster2] - role: worker
2021-07-16T18:08:07.3451483Z [cluster2] 
2021-07-16T18:08:07.3452031Z [cluster2] 
2021-07-16T18:08:07.3452881Z [submariner-operator]$ [cluster2] eval "echo \"$(cat $1)\""
2021-07-16T18:08:07.3454266Z [submariner-operator]$ [cluster2] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml
2021-07-16T18:08:07.3455827Z [submariner-operator]$ [cluster2] image_flag=--image=kindest/node:v1.18.19
2021-07-16T18:08:07.3456933Z [submariner-operator]$ [cluster2] kind version
2021-07-16T18:08:07.3457660Z [cluster2] kind v0.11.1 go1.16.4 linux/amd64
2021-07-16T18:08:07.3459162Z [submariner-operator]$ [cluster2] cat /opt/shipyard/scripts/resources/cluster2-config.yaml
2021-07-16T18:08:07.3460198Z [cluster2] ---
2021-07-16T18:08:07.3460665Z [cluster2] kind: Cluster
2021-07-16T18:08:07.3461440Z [cluster2] apiVersion: kind.x-k8s.io/v1alpha4
2021-07-16T18:08:07.3462075Z [cluster2] networking:
2021-07-16T18:08:07.3462682Z [cluster2]   disableDefaultCNI: true
2021-07-16T18:08:07.3463314Z [cluster2]   podSubnet: 10.2.0.0/16
2021-07-16T18:08:07.3463895Z [cluster2]   serviceSubnet: 100.2.0.0/16
2021-07-16T18:08:07.3464623Z [cluster2] containerdConfigPatches:
2021-07-16T18:08:07.3465378Z [cluster2]   - |-
2021-07-16T18:08:07.3466263Z [cluster2]     [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
2021-07-16T18:08:07.3467552Z [cluster2]       endpoint = ["http://kind-registry:5000"]
2021-07-16T18:08:07.3468383Z [cluster2] kubeadmConfigPatches:
2021-07-16T18:08:07.3469232Z [cluster2]   - |
2021-07-16T18:08:07.3469827Z [cluster2]     apiVersion: kubeadm.k8s.io/v1beta2
2021-07-16T18:08:07.3470594Z [cluster2]     kind: ClusterConfiguration
2021-07-16T18:08:07.3471200Z [cluster2]     metadata:
2021-07-16T18:08:07.3471696Z [cluster2]       name: config
2021-07-16T18:08:07.3472196Z [cluster2]     networking:
2021-07-16T18:08:07.3472735Z [cluster2]       podSubnet: 10.2.0.0/16
2021-07-16T18:08:07.3473422Z [cluster2]       serviceSubnet: 100.2.0.0/16
2021-07-16T18:08:07.3474079Z [cluster2]       dnsDomain: cluster2.local
2021-07-16T18:08:07.3474634Z [cluster2] nodes: 
2021-07-16T18:08:07.3475329Z [cluster2] - role: control-plane
2021-07-16T18:08:07.3476045Z [cluster2] - role: worker
2021-07-16T18:08:07.3476679Z [cluster2] - role: worker
2021-07-16T18:08:07.3478322Z [submariner-operator]$ [cluster2] kind create cluster --image=kindest/node:v1.18.19 --name=cluster2 --config=/opt/shipyard/scripts/resources/cluster2-config.yaml
2021-07-16T18:08:07.3479592Z [cluster2] Creating cluster "cluster2" ...
2021-07-16T18:08:07.3481711Z [cluster2]  • Ensuring node image (kindest/node:v1.18.19) 🖼  ...
2021-07-16T18:08:07.3482729Z [cluster2]  ✓ Ensuring node image (kindest/node:v1.18.19) 🖼
2021-07-16T18:08:07.3483962Z [cluster2]  • Preparing nodes 📦 📦 📦   ...
2021-07-16T18:08:07.3484736Z [cluster2]  ✓ Preparing nodes 📦 📦 📦 
2021-07-16T18:08:07.3485532Z [cluster2]  • Writing configuration 📜  ...
2021-07-16T18:08:07.3486328Z [cluster2]  ✓ Writing configuration 📜
2021-07-16T18:08:07.3487165Z [cluster2]  • Starting control-plane 🕹️  ...
2021-07-16T18:08:07.3488228Z [cluster2]  ✓ Starting control-plane 🕹️
2021-07-16T18:08:07.3489099Z [cluster2]  • Installing StorageClass 💾  ...
2021-07-16T18:08:07.3490720Z [cluster2]  ✓ Installing StorageClass 💾
2021-07-16T18:08:07.3491519Z [cluster2]  • Joining worker nodes 🚜  ...
2021-07-16T18:08:07.3492269Z [cluster2]  ✓ Joining worker nodes 🚜
2021-07-16T18:08:07.3493095Z [cluster2] Set kubectl context to "kind-cluster2"
2021-07-16T18:08:07.3493799Z [cluster2] You can now use your cluster with:
2021-07-16T18:08:07.3494332Z [cluster2] 
2021-07-16T18:08:07.3495136Z [cluster2] kubectl cluster-info --context kind-cluster2
2021-07-16T18:08:07.3495756Z [cluster2] 
2021-07-16T18:08:07.3496819Z [cluster2] Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
2021-07-16T18:08:07.3498071Z [submariner-operator]$ [cluster2] kind_fixup_config
2021-07-16T18:08:07.3499116Z [submariner-operator]$ [cluster2] kind_fixup_config
2021-07-16T18:08:07.3500966Z [submariner-operator]$ [cluster2] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster2-control-plane
2021-07-16T18:08:07.3502503Z [submariner-operator]$ [cluster2] head -n 1
2021-07-16T18:08:08.0388760Z [submariner-operator]$ [cluster1] with_retries
2021-07-16T18:08:08.0389879Z [submariner-operator]$ [cluster1] iteration=1
2021-07-16T18:08:08.0390888Z [submariner-operator]$ [cluster1] wait 261
2021-07-16T18:08:08.0391913Z [submariner-operator]$ [cluster1] create_kind_cluster
2021-07-16T18:08:08.0704471Z [submariner-operator]$ [cluster1] create_kind_cluster
2021-07-16T18:08:08.0706346Z [submariner-operator]$ [cluster1] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:08.0708584Z [submariner-operator]$ [cluster1] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:08.0710189Z [submariner-operator]$ [cluster1] kind get clusters
2021-07-16T18:08:08.0711258Z [submariner-operator]$ [cluster1] grep -q ^cluster1$
2021-07-16T18:08:08.0712012Z [cluster1] No kind clusters found.
2021-07-16T18:08:08.0712601Z [cluster1] Creating KIND cluster...
2021-07-16T18:08:08.0713562Z [submariner-operator]$ [cluster1] generate_cluster_yaml
2021-07-16T18:08:08.0714661Z [submariner-operator]$ [cluster1] generate_cluster_yaml
2021-07-16T18:08:08.0715745Z [submariner-operator]$ [cluster1] disable_cni=true
2021-07-16T18:08:08.0716727Z [submariner-operator]$ [cluster1] nodes=
2021-07-16T18:08:08.0717590Z [cluster1] - role: control-plane
2021-07-16T18:08:08.0718451Z [submariner-operator]$ [cluster1] nodes=
2021-07-16T18:08:08.0719288Z [cluster1] - role: control-plane
2021-07-16T18:08:08.0719996Z [cluster1] - role: worker
2021-07-16T18:08:08.0720797Z [submariner-operator]$ [cluster1] nodes=
2021-07-16T18:08:08.0721626Z [cluster1] - role: control-plane
2021-07-16T18:08:08.0722306Z [cluster1] - role: worker
2021-07-16T18:08:08.0722954Z [cluster1] - role: worker
2021-07-16T18:08:08.0723814Z [cluster1] 
2021-07-16T18:08:08.0724400Z [cluster1] 
2021-07-16T18:08:08.0725238Z [submariner-operator]$ [cluster1] eval "echo \"$(cat $1)\""
2021-07-16T18:08:08.0726642Z [submariner-operator]$ [cluster1] cat /opt/shipyard/scripts/resources/kind-cluster-config.yaml
2021-07-16T18:08:08.0728097Z [submariner-operator]$ [cluster1] image_flag=--image=kindest/node:v1.18.19
2021-07-16T18:08:08.0729228Z [submariner-operator]$ [cluster1] kind version
2021-07-16T18:08:08.0729968Z [cluster1] kind v0.11.1 go1.16.4 linux/amd64
2021-07-16T18:08:08.0731159Z [submariner-operator]$ [cluster1] cat /opt/shipyard/scripts/resources/cluster1-config.yaml
2021-07-16T18:08:08.0732177Z [cluster1] ---
2021-07-16T18:08:08.0732640Z [cluster1] kind: Cluster
2021-07-16T18:08:08.0733428Z [cluster1] apiVersion: kind.x-k8s.io/v1alpha4
2021-07-16T18:08:08.0734268Z [cluster1] networking:
2021-07-16T18:08:08.0734900Z [cluster1]   disableDefaultCNI: true
2021-07-16T18:08:08.0735518Z [cluster1]   podSubnet: 10.1.0.0/16
2021-07-16T18:08:08.0736110Z [cluster1]   serviceSubnet: 100.1.0.0/16
2021-07-16T18:08:08.0736830Z [cluster1] containerdConfigPatches:
2021-07-16T18:08:08.0737613Z [cluster1]   - |-
2021-07-16T18:08:08.0738503Z [cluster1]     [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
2021-07-16T18:08:08.0739805Z [cluster1]       endpoint = ["http://kind-registry:5000"]
2021-07-16T18:08:08.0740640Z [cluster1] kubeadmConfigPatches:
2021-07-16T18:08:08.0741345Z [cluster1]   - |
2021-07-16T18:08:08.0741957Z [cluster1]     apiVersion: kubeadm.k8s.io/v1beta2
2021-07-16T18:08:08.0742704Z [cluster1]     kind: ClusterConfiguration
2021-07-16T18:08:08.0743328Z [cluster1]     metadata:
2021-07-16T18:08:08.0743813Z [cluster1]       name: config
2021-07-16T18:08:08.0744332Z [cluster1]     networking:
2021-07-16T18:08:08.0744874Z [cluster1]       podSubnet: 10.1.0.0/16
2021-07-16T18:08:08.0745462Z [cluster1]       serviceSubnet: 100.1.0.0/16
2021-07-16T18:08:08.0746123Z [cluster1]       dnsDomain: cluster1.local
2021-07-16T18:08:08.0746673Z [cluster1] nodes: 
2021-07-16T18:08:08.0747365Z [cluster1] - role: control-plane
2021-07-16T18:08:08.0748050Z [cluster1] - role: worker
2021-07-16T18:08:08.0748694Z [cluster1] - role: worker
2021-07-16T18:08:08.0750190Z [submariner-operator]$ [cluster1] kind create cluster --image=kindest/node:v1.18.19 --name=cluster1 --config=/opt/shipyard/scripts/resources/cluster1-config.yaml
2021-07-16T18:08:08.0751587Z [cluster1] Creating cluster "cluster1" ...
2021-07-16T18:08:08.0752668Z [cluster1]  • Ensuring node image (kindest/node:v1.18.19) 🖼  ...
2021-07-16T18:08:08.0753666Z [cluster1]  ✓ Ensuring node image (kindest/node:v1.18.19) 🖼
2021-07-16T18:08:08.0754616Z [cluster1]  • Preparing nodes 📦 📦 📦   ...
2021-07-16T18:08:08.0784723Z [cluster1]  ✓ Preparing nodes 📦 📦 📦 
2021-07-16T18:08:08.0785992Z [cluster1]  • Writing configuration 📜  ...
2021-07-16T18:08:08.0786830Z [cluster1]  ✓ Writing configuration 📜
2021-07-16T18:08:08.0787686Z [cluster1]  • Starting control-plane 🕹️  ...
2021-07-16T18:08:08.0788523Z [cluster1]  ✓ Starting control-plane 🕹️
2021-07-16T18:08:08.0789379Z [cluster1]  • Installing StorageClass 💾  ...
2021-07-16T18:08:08.0790218Z [cluster1]  ✓ Installing StorageClass 💾
2021-07-16T18:08:08.0791024Z [cluster1]  • Joining worker nodes 🚜  ...
2021-07-16T18:08:08.0791802Z [cluster1]  ✓ Joining worker nodes 🚜
2021-07-16T18:08:08.0792656Z [cluster1] Set kubectl context to "kind-cluster1"
2021-07-16T18:08:08.0793364Z [cluster1] You can now use your cluster with:
2021-07-16T18:08:08.0793878Z [cluster1] 
2021-07-16T18:08:08.0794684Z [cluster1] kubectl cluster-info --context kind-cluster1
2021-07-16T18:08:08.0795393Z [cluster1] 
2021-07-16T18:08:08.0796479Z [cluster1] Not sure what to do next? 😅  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
2021-07-16T18:08:08.0797744Z [submariner-operator]$ [cluster1] kind_fixup_config
2021-07-16T18:08:08.0798819Z [submariner-operator]$ [cluster1] kind_fixup_config
2021-07-16T18:08:08.0800479Z [submariner-operator]$ [cluster1] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster1-control-plane
2021-07-16T18:08:08.0802021Z [submariner-operator]$ [cluster1] head -n 1
2021-07-16T18:08:48.2300558Z [submariner-operator]$ [cluster2] sed -i -- s/server: .*/server: https:\/\/172.18.0.7:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:48.2303155Z [submariner-operator]$ [cluster2] sed -i -- s/user: kind-.*/user: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:48.2305782Z [submariner-operator]$ [cluster2] sed -i -- s/name: kind-.*/name: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:48.2308254Z [submariner-operator]$ [cluster2] sed -i -- s/cluster: kind-.*/cluster: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:48.2310870Z [submariner-operator]$ [cluster2] sed -i -- s/current-context: .*/current-context: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:48.2313295Z [submariner-operator]$ [cluster2] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
2021-07-16T18:08:48.2314807Z [submariner-operator]$ [cluster2] wait 1524
2021-07-16T18:08:48.2315913Z [submariner-operator]$ [cluster2] deploy_cluster_capabilities
2021-07-16T18:08:48.2317123Z [submariner-operator]$ [cluster2] deploy_cluster_capabilities
2021-07-16T18:08:48.2318226Z [submariner-operator]$ [cluster2] deploy_cni
2021-07-16T18:08:48.2319214Z [submariner-operator]$ [cluster2] deploy_cni
2021-07-16T18:08:48.2320278Z [submariner-operator]$ [cluster2] eval deploy_weave_cni
2021-07-16T18:08:48.2321359Z [submariner-operator]$ [cluster2] deploy_weave_cni
2021-07-16T18:08:48.2322400Z [submariner-operator]$ [cluster2] deploy_weave_cni
2021-07-16T18:08:48.2323177Z [cluster2] Applying weave network...
2021-07-16T18:08:48.2324959Z [submariner-operator]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.18.19&env.IPALLOC_RANGE=10.2.0.0/16
2021-07-16T18:08:48.2327116Z [submariner-operator]$ [cluster2] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.18.19&env.IPALLOC_RANGE=10.2.0.0/16
2021-07-16T18:08:48.2329266Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.18.19&env.IPALLOC_RANGE=10.2.0.0/16
2021-07-16T18:08:48.2330786Z [cluster2] serviceaccount/weave-net created
2021-07-16T18:08:48.2332100Z [cluster2] clusterrole.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:48.2333900Z [cluster2] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:48.2335558Z [cluster2] role.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:48.2337606Z [cluster2] rolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:48.2339094Z [cluster2] daemonset.apps/weave-net created
2021-07-16T18:08:48.2340026Z [cluster2] Waiting for weave-net pods to be ready...
2021-07-16T18:08:48.2341369Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-16T18:08:48.2343064Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-16T18:08:48.2344904Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-16T18:08:48.2346288Z [cluster2] pod/weave-net-7dslf condition met
2021-07-16T18:08:48.2347206Z [cluster2] pod/weave-net-8zdpd condition met
2021-07-16T18:08:48.2348101Z [cluster2] pod/weave-net-jtlvj condition met
2021-07-16T18:08:48.2349055Z [cluster2] Waiting for core-dns deployment to be ready...
2021-07-16T18:08:48.2350328Z [submariner-operator]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-16T18:08:48.2351847Z [submariner-operator]$ [cluster2] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-16T18:08:48.2353492Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-16T18:08:48.2354859Z [cluster2] Waiting for deployment "coredns" rollout to finish: 0 of 2 updated replicas are available...
2021-07-16T18:08:48.2356015Z [cluster2] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
2021-07-16T18:08:48.2357114Z [cluster2] deployment "coredns" successfully rolled out
2021-07-16T18:08:48.2358140Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:08:48.2412723Z [submariner-operator]$ [cluster2] wait 238
2021-07-16T18:08:50.6460488Z [submariner-operator]$ [cluster1] sed -i -- s/server: .*/server: https:\/\/172.18.0.8:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:50.6463528Z [submariner-operator]$ [cluster1] sed -i -- s/user: kind-.*/user: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:50.6466187Z [submariner-operator]$ [cluster1] sed -i -- s/name: kind-.*/name: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:50.6468781Z [submariner-operator]$ [cluster1] sed -i -- s/cluster: kind-.*/cluster: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:50.6471644Z [submariner-operator]$ [cluster1] sed -i -- s/current-context: .*/current-context: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:50.6474200Z [submariner-operator]$ [cluster1] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
2021-07-16T18:08:50.6479048Z [submariner-operator]$ [cluster1] deploy_cluster_capabilities
2021-07-16T18:08:50.6480594Z [submariner-operator]$ [cluster1] deploy_cluster_capabilities
2021-07-16T18:08:50.6481693Z [submariner-operator]$ [cluster1] deploy_cni
2021-07-16T18:08:50.6482696Z [submariner-operator]$ [cluster1] deploy_cni
2021-07-16T18:08:50.6483854Z [submariner-operator]$ [cluster1] eval deploy_weave_cni
2021-07-16T18:08:50.6485250Z [submariner-operator]$ [cluster1] deploy_weave_cni
2021-07-16T18:08:50.6486306Z [submariner-operator]$ [cluster1] deploy_weave_cni
2021-07-16T18:08:50.6487087Z [cluster1] Applying weave network...
2021-07-16T18:08:50.6487980Z [submariner-operator]$ [cluster1] wait 1561
2021-07-16T18:08:50.6489531Z [submariner-operator]$ [cluster1] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.18.19&env.IPALLOC_RANGE=10.1.0.0/16
2021-07-16T18:08:50.6491532Z [submariner-operator]$ [cluster1] kubectl apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.18.19&env.IPALLOC_RANGE=10.1.0.0/16
2021-07-16T18:08:50.6493671Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f https://cloud.weave.works/k8s/net?k8s-version=v1.18.19&env.IPALLOC_RANGE=10.1.0.0/16
2021-07-16T18:08:50.6495207Z [cluster1] serviceaccount/weave-net created
2021-07-16T18:08:50.6496510Z [cluster1] clusterrole.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:50.6498312Z [cluster1] clusterrolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:50.6499976Z [cluster1] role.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:50.6501439Z [cluster1] rolebinding.rbac.authorization.k8s.io/weave-net created
2021-07-16T18:08:50.6502706Z [cluster1] daemonset.apps/weave-net created
2021-07-16T18:08:50.6503615Z [cluster1] Waiting for weave-net pods to be ready...
2021-07-16T18:08:50.6504971Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-16T18:08:50.6506690Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-16T18:08:50.6508517Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 wait --for=condition=Ready pods -l name=weave-net -n kube-system --timeout=5m
2021-07-16T18:08:50.6509895Z [cluster1] pod/weave-net-5j2hp condition met
2021-07-16T18:08:50.6512389Z [cluster1] pod/weave-net-bzj6w condition met
2021-07-16T18:08:50.6513537Z [cluster1] pod/weave-net-qfpb8 condition met
2021-07-16T18:08:50.6514521Z [cluster1] Waiting for core-dns deployment to be ready...
2021-07-16T18:08:50.6515795Z [submariner-operator]$ [cluster1] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-16T18:08:50.6517317Z [submariner-operator]$ [cluster1] kubectl -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-16T18:08:50.6518965Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n kube-system rollout status deploy/coredns --timeout=5m
2021-07-16T18:08:50.6520372Z [cluster1] Waiting for deployment "coredns" rollout to finish: 0 of 2 updated replicas are available...
2021-07-16T18:08:50.6521529Z [cluster1] Waiting for deployment "coredns" rollout to finish: 1 of 2 updated replicas are available...
2021-07-16T18:08:50.6522520Z [cluster1] deployment "coredns" successfully rolled out
2021-07-16T18:08:50.6523515Z [submariner-operator]$ [cluster1] return 0
2021-07-16T18:08:50.6555132Z [submariner-operator]$ [cluster2] print_clusters_message
2021-07-16T18:08:50.6574413Z [submariner-operator]$ [cluster2] print_clusters_message
2021-07-16T18:08:50.6591532Z [submariner-operator]$ [cluster2] cat <<EOM
2021-07-16T18:08:50.6592497Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-16T18:08:50.6593071Z 
2021-07-16T18:08:50.6594061Z export KUBECONFIG=\$(find \$(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-16T18:08:50.6594896Z 
2021-07-16T18:08:50.6595693Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-16T18:08:50.6596202Z 
2021-07-16T18:08:50.6596897Z To clean everthing up, just run: make clean-clusters
2021-07-16T18:08:50.6601186Z EOM
2021-07-16T18:08:50.6623897Z Your virtual cluster(s) are deployed and working properly and can be accessed with:
2021-07-16T18:08:50.6624543Z 
2021-07-16T18:08:50.6625716Z export KUBECONFIG=$(find $(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
2021-07-16T18:08:50.6626377Z 
2021-07-16T18:08:50.6631099Z $ kubectl config use-context cluster1 # or cluster2, cluster3..
2021-07-16T18:08:50.6631654Z 
2021-07-16T18:08:50.6632392Z To clean everthing up, just run: make clean-clusters
2021-07-16T18:08:50.6648431Z     Successfully remade target file 'clusters'.
2021-07-16T18:08:50.6649510Z      File 'preload-images' does not exist.
2021-07-16T18:08:50.6650867Z        File 'images' does not exist.
2021-07-16T18:08:50.6652074Z          File 'package/.image.submariner-operator' does not exist.
2021-07-16T18:08:50.6858447Z find: 'bin/submariner-operator': No such file or directory
2021-07-16T18:08:51.0439478Z              File 'bin/submariner-operator' does not exist.
2021-07-16T18:08:51.0440537Z                File 'vendor/modules.txt' does not exist.
2021-07-16T18:08:51.0441378Z               Must remake target 'vendor/modules.txt'.
2021-07-16T18:08:51.0441939Z go mod download
2021-07-16T18:09:43.5932194Z go mod vendor
2021-07-16T18:09:45.8194936Z go mod tidy
2021-07-16T18:09:46.5169277Z               Successfully remade target file 'vendor/modules.txt'.
2021-07-16T18:09:46.5178283Z                File 'generate-embeddedyamls' does not exist.
2021-07-16T18:09:46.5179272Z                  File 'generate' does not exist.
2021-07-16T18:09:46.5180481Z                    File '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen' does not exist.
2021-07-16T18:09:46.5185480Z                   Must remake target '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen'.
2021-07-16T18:09:46.5189333Z mkdir -p /go/src/github.com/submariner-io/submariner-operator/bin
2021-07-16T18:09:46.5227503Z go build -o /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen sigs.k8s.io/controller-tools/cmd/controller-gen
2021-07-16T18:10:07.3378772Z                   Successfully remade target file '/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen'.
2021-07-16T18:10:07.3380392Z                 Must remake target 'generate'.
2021-07-16T18:10:07.3791382Z /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt,year=2021" paths="./..."
2021-07-16T18:10:20.9230083Z                 Successfully remade target file 'generate'.
2021-07-16T18:10:20.9231340Z                  File 'pkg/subctl/operator/common/embeddedyamls/yamls.go' does not exist.
2021-07-16T18:10:20.9235590Z                    File 'deploy/crds/submariner.io_servicediscoveries.yaml' does not exist.
2021-07-16T18:10:20.9237136Z                   Must remake target 'deploy/crds/submariner.io_servicediscoveries.yaml'.
2021-07-16T18:10:20.9239031Z /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen "crd:crdVersions=v1,trivialVersions=false" paths="./..." output:crd:artifacts:config=deploy/crds
2021-07-16T18:10:23.2039530Z                   Successfully remade target file 'deploy/crds/submariner.io_servicediscoveries.yaml'.
2021-07-16T18:10:23.2048137Z                    File 'deploy/submariner/crds/submariner.io_clusters.yaml' does not exist.
2021-07-16T18:10:23.2053328Z                   Must remake target 'deploy/submariner/crds/submariner.io_clusters.yaml'.
2021-07-16T18:10:23.2058957Z cd vendor/github.com/submariner-io/submariner && /go/src/github.com/submariner-io/submariner-operator/bin/controller-gen "crd:crdVersions=v1,trivialVersions=false" paths="./..." output:crd:artifacts:config=../../../../deploy/submariner/crds
2021-07-16T18:10:24.5765430Z                   Successfully remade target file 'deploy/submariner/crds/submariner.io_clusters.yaml'.
2021-07-16T18:10:24.5870821Z                 Must remake target 'pkg/subctl/operator/common/embeddedyamls/yamls.go'.
2021-07-16T18:10:24.5871868Z go generate pkg/subctl/operator/common/embeddedyamls/generate.go
2021-07-16T18:10:25.3798111Z Generating yamls.go
2021-07-16T18:10:25.3802605Z deploy/crds/submariner.io_brokers.yaml
2021-07-16T18:10:25.3808288Z deploy/crds/submariner.io_submariners.yaml
2021-07-16T18:10:25.3840506Z deploy/crds/submariner.io_servicediscoveries.yaml
2021-07-16T18:10:25.3846608Z deploy/submariner/crds/submariner.io_clusters.yaml
2021-07-16T18:10:25.3851587Z deploy/submariner/crds/submariner.io_endpoints.yaml
2021-07-16T18:10:25.3856857Z deploy/submariner/crds/submariner.io_gateways.yaml
2021-07-16T18:10:25.3863928Z deploy/submariner/crds/submariner.io_clusterglobalegressips.yaml
2021-07-16T18:10:25.3871972Z deploy/submariner/crds/submariner.io_globalegressips.yaml
2021-07-16T18:10:25.3884085Z deploy/submariner/crds/submariner.io_globalingressips.yaml
2021-07-16T18:10:25.3893106Z deploy/mcsapi/crds/multicluster.x_k8s.io_serviceexports.yaml
2021-07-16T18:10:25.3899627Z deploy/mcsapi/crds/multicluster.x_k8s.io_serviceimports.yaml
2021-07-16T18:10:25.3908078Z config/broker/broker-admin/service_account.yaml
2021-07-16T18:10:25.3911743Z config/broker/broker-admin/role.yaml
2021-07-16T18:10:25.3915030Z config/broker/broker-admin/role_binding.yaml
2021-07-16T18:10:25.3918374Z config/broker/broker-client/service_account.yaml
2021-07-16T18:10:25.3921535Z config/broker/broker-client/role.yaml
2021-07-16T18:10:25.3925386Z config/broker/broker-client/role_binding.yaml
2021-07-16T18:10:25.3928789Z config/rbac/submariner-operator/service_account.yaml
2021-07-16T18:10:25.3933609Z config/rbac/submariner-operator/role.yaml
2021-07-16T18:10:25.3940456Z config/rbac/submariner-operator/role_binding.yaml
2021-07-16T18:10:25.3945422Z config/rbac/submariner-operator/cluster_role.yaml
2021-07-16T18:10:25.3953394Z config/rbac/submariner-operator/cluster_role_binding.yaml
2021-07-16T18:10:25.3954608Z config/rbac/submariner-gateway/service_account.yaml
2021-07-16T18:10:25.3958992Z config/rbac/submariner-gateway/role.yaml
2021-07-16T18:10:25.3965475Z config/rbac/submariner-gateway/role_binding.yaml
2021-07-16T18:10:25.3966565Z config/rbac/submariner-gateway/cluster_role.yaml
2021-07-16T18:10:25.3970232Z config/rbac/submariner-gateway/cluster_role_binding.yaml
2021-07-16T18:10:25.3976340Z config/rbac/submariner-route-agent/service_account.yaml
2021-07-16T18:10:25.3979540Z config/rbac/submariner-route-agent/role.yaml
2021-07-16T18:10:25.3984496Z config/rbac/submariner-route-agent/role_binding.yaml
2021-07-16T18:10:25.3988119Z config/rbac/submariner-route-agent/cluster_role.yaml
2021-07-16T18:10:25.3992749Z config/rbac/submariner-route-agent/cluster_role_binding.yaml
2021-07-16T18:10:25.3996758Z config/rbac/submariner-globalnet/service_account.yaml
2021-07-16T18:10:25.4000294Z config/rbac/submariner-globalnet/role.yaml
2021-07-16T18:10:25.4005097Z config/rbac/submariner-globalnet/role_binding.yaml
2021-07-16T18:10:25.4010314Z config/rbac/submariner-globalnet/cluster_role.yaml
2021-07-16T18:10:25.4022969Z config/rbac/submariner-globalnet/cluster_role_binding.yaml
2021-07-16T18:10:25.4027849Z config/rbac/lighthouse-agent/service_account.yaml
2021-07-16T18:10:25.4031390Z config/rbac/lighthouse-agent/cluster_role.yaml
2021-07-16T18:10:25.4037170Z config/rbac/lighthouse-agent/cluster_role_binding.yaml
2021-07-16T18:10:25.4041004Z config/rbac/lighthouse-coredns/service_account.yaml
2021-07-16T18:10:25.4045105Z config/rbac/lighthouse-coredns/cluster_role.yaml
2021-07-16T18:10:25.4049097Z config/rbac/lighthouse-coredns/cluster_role_binding.yaml
2021-07-16T18:10:25.4052255Z config/rbac/networkplugin_syncer/service_account.yaml
2021-07-16T18:10:25.4056254Z config/rbac/networkplugin_syncer/cluster_role.yaml
2021-07-16T18:10:25.4063060Z config/rbac/networkplugin_syncer/cluster_role_binding.yaml
2021-07-16T18:10:25.4068691Z config/rbac/submariner-metrics-reader/role.yaml
2021-07-16T18:10:25.4072172Z config/rbac/submariner-metrics-reader/role_binding.yaml
2021-07-16T18:10:25.4103666Z                 Successfully remade target file 'pkg/subctl/operator/common/embeddedyamls/yamls.go'.
2021-07-16T18:10:25.4105110Z               Must remake target 'generate-embeddedyamls'.
2021-07-16T18:10:25.4106436Z               Successfully remade target file 'generate-embeddedyamls'.
2021-07-16T18:10:25.4107684Z             Must remake target 'bin/submariner-operator'.
2021-07-16T18:10:25.4108516Z /opt/shipyard/scripts/compile.sh \
2021-07-16T18:10:25.4109885Z --ldflags "-X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33" \
2021-07-16T18:10:25.4111215Z bin/submariner-operator main.go 
2021-07-16T18:10:25.4730585Z [submariner-operator]$ mkdir -p bin
2021-07-16T18:10:25.4747217Z Building 'bin/submariner-operator' (ldflags: '-X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33')
2021-07-16T18:10:25.4761196Z [submariner-operator]$ ldflags=-s -w -X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33
2021-07-16T18:10:25.4773460Z [submariner-operator]$ CGO_ENABLED=0 go build -trimpath -ldflags -s -w -X=github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33 -o bin/submariner-operator main.go
2021-07-16T18:12:01.5529522Z [submariner-operator]$ upx bin/submariner-operator
2021-07-16T18:12:07.5761608Z                        Ultimate Packer for eXecutables
2021-07-16T18:12:07.5764016Z                           Copyright (C) 1996 - 2020
2021-07-16T18:12:07.5765181Z UPX 3.96        Markus Oberhumer, Laszlo Molnar & John Reiser   Jan 23rd 2020
2021-07-16T18:12:07.5765847Z 
2021-07-16T18:12:07.5766740Z         File size         Ratio      Format      Name
2021-07-16T18:12:07.5768184Z    --------------------   ------   -----------   -----------
2021-07-16T18:12:07.5769239Z   34459648 ->   9933024   28.83%   linux/amd64   submariner-operator
2021-07-16T18:12:07.5769851Z 
2021-07-16T18:12:07.5770229Z Packed 1 file.
2021-07-16T18:12:07.5775387Z             Successfully remade target file 'bin/submariner-operator'.
2021-07-16T18:12:07.5777274Z            Prerequisite 'bin/submariner-operator' is newer than target 'package/Dockerfile.submariner-operator'.
2021-07-16T18:12:07.5779147Z           Must remake target 'package/Dockerfile.submariner-operator'.
2021-07-16T18:12:07.5781058Z           Successfully remade target file 'package/Dockerfile.submariner-operator'.
2021-07-16T18:12:07.5782451Z            File 'FORCE_IMAGE' does not exist.
2021-07-16T18:12:07.5783404Z           Must remake target 'FORCE_IMAGE'.
2021-07-16T18:12:07.5784421Z           Successfully remade target file 'FORCE_IMAGE'.
2021-07-16T18:12:07.5785687Z         Must remake target 'package/.image.submariner-operator'.
2021-07-16T18:12:07.5787810Z /opt/shipyard/scripts/build_image.sh -i submariner-operator -f package/Dockerfile.submariner-operator -h package/.image.submariner-operator 
2021-07-16T18:12:07.6327501Z flags:WARN flag short name (h) already defined
2021-07-16T18:12:07.7083038Z [submariner-operator]$ set -e
2021-07-16T18:12:07.7096065Z [submariner-operator]$ cache_image=quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:07.7105995Z [submariner-operator]$ cache_flag=
2021-07-16T18:12:07.7121440Z [submariner-operator]$ cache_flag=--cache-from quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:07.7142224Z [submariner-operator]$ docker image ls -q quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:08.0343255Z [submariner-operator]$ docker pull quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:08.5213123Z devel: Pulling from submariner/submariner-operator
2021-07-16T18:12:08.5222761Z 16b3838259a1: Pulling fs layer
2021-07-16T18:12:08.6928722Z 16b3838259a1: Verifying Checksum
2021-07-16T18:12:08.6929376Z 16b3838259a1: Download complete
2021-07-16T18:12:09.0727535Z 16b3838259a1: Pull complete
2021-07-16T18:12:09.0782439Z Digest: sha256:3e2788fae3e11a10b9728e6797f3ba25c5d4c6aa583e9c0095a169b2bc33572a
2021-07-16T18:12:09.0808573Z Status: Downloaded newer image for quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:09.0835063Z quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:09.0850200Z [submariner-operator]$ grep FROM package/Dockerfile.submariner-operator
2021-07-16T18:12:09.0878151Z [submariner-operator]$ cut -f2 -d 
2021-07-16T18:12:09.0891263Z [submariner-operator]$ grep -v scratch
2021-07-16T18:12:09.0906604Z [submariner-operator]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel
2021-07-16T18:12:09.0915112Z 
2021-07-16T18:12:09.9689065Z [submariner-operator]$ docker buildx use buildx_builder
2021-07-16T18:12:10.8170539Z error: failed to find instance "buildx_builder": open /root/.docker/buildx/instances/buildx_builder: no such file or directory
2021-07-16T18:12:10.8239378Z [submariner-operator]$ docker buildx create --name buildx_builder --use
2021-07-16T18:12:11.6795145Z buildx_builder
2021-07-16T18:12:11.6893689Z [submariner-operator]$ docker buildx build --load -t quay.io/submariner/submariner-operator:dev --cache-from quay.io/submariner/submariner-operator:devel -f package/Dockerfile.submariner-operator --iidfile package/.image.submariner-operator --platform linux/amd64 --build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel .
2021-07-16T18:12:12.5871700Z #1 [internal] booting buildkit
2021-07-16T18:12:12.5872971Z #1 sha256:ff92efe171751cbba244f29fc2df6f4c3c6c038ca764e38a3626709093631da1
2021-07-16T18:12:12.7375073Z #1 pulling image moby/buildkit:buildx-stable-1
2021-07-16T18:12:15.6970445Z #1 pulling image moby/buildkit:buildx-stable-1 3.1s done
2021-07-16T18:12:15.8473764Z #1 creating container buildx_buildkit_buildx_builder0
2021-07-16T18:12:16.4135249Z #1 creating container buildx_buildkit_buildx_builder0 0.7s done
2021-07-16T18:12:16.5633531Z #1 DONE 3.8s
2021-07-16T18:12:16.6812385Z 
2021-07-16T18:12:16.6815583Z #2 [internal] load build definition from Dockerfile.submariner-operator
2021-07-16T18:12:16.6816892Z #2 sha256:04629bdf24f724951db44f4297eb92557aca1d81336746730579d3a12a4a12db
2021-07-16T18:12:16.6817852Z #2 transferring dockerfile: 230B done
2021-07-16T18:12:16.6818377Z #2 DONE 0.0s
2021-07-16T18:12:16.6818641Z 
2021-07-16T18:12:16.6819086Z #3 [internal] load .dockerignore
2021-07-16T18:12:16.6820451Z #3 sha256:e6b1b25ffb314a73f006c41f32e0ce97ef39e63917806e9bfaf0fb71474117e4
2021-07-16T18:12:16.6821557Z #3 transferring context: 2B done
2021-07-16T18:12:16.6822042Z #3 DONE 0.0s
2021-07-16T18:12:16.6822300Z 
2021-07-16T18:12:16.6823246Z #6 importing cache manifest from quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:16.6825142Z #6 sha256:db6e513456d7d63fd9e86cca8d8a83f1eb0b2eaefc7f50d06048094c1b3ec182
2021-07-16T18:12:16.8312954Z #6 ...
2021-07-16T18:12:16.8313327Z 
2021-07-16T18:12:16.8313866Z #4 [internal] load build context
2021-07-16T18:12:16.8314944Z #4 sha256:d33c3ddd57156811f87ef99a14dee5cbee56901f4427b79c73605e5e66a0917b
2021-07-16T18:12:16.8315936Z #4 DONE 0.0s
2021-07-16T18:12:16.9685718Z 
2021-07-16T18:12:17.7611452Z #6 importing cache manifest from quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:17.7620029Z #6 sha256:db6e513456d7d63fd9e86cca8d8a83f1eb0b2eaefc7f50d06048094c1b3ec182
2021-07-16T18:12:17.7621377Z #6 DONE 0.3s
2021-07-16T18:12:17.7621763Z 
2021-07-16T18:12:17.7622315Z #4 [internal] load build context
2021-07-16T18:12:17.7623503Z #4 sha256:d33c3ddd57156811f87ef99a14dee5cbee56901f4427b79c73605e5e66a0917b
2021-07-16T18:12:17.7624703Z #4 transferring context: 9.94MB 0.3s done
2021-07-16T18:12:17.7625289Z #4 DONE 0.3s
2021-07-16T18:12:17.7625637Z 
2021-07-16T18:12:17.7626812Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-16T18:12:17.7628243Z #5 sha256:24c8fd7c2e874f26b1a9f5f9b4125b6a700b6cefe662dc47967c957f28286fa4
2021-07-16T18:12:17.7629901Z #5 CACHED
2021-07-16T18:12:17.7630243Z 
2021-07-16T18:12:17.7630785Z #7 exporting to oci image format
2021-07-16T18:12:17.7632003Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-16T18:12:17.7633156Z #7 exporting layers done
2021-07-16T18:12:17.7634397Z #7 exporting manifest sha256:237566480f4d82aaff9259daaf4b2ed49d985d414db4e64d4b64b9906b066159 done
2021-07-16T18:12:17.7636094Z #7 exporting config sha256:e785f002e774a86471557d1dc2023536a375b474b9022ff8efa19558879d0cec done
2021-07-16T18:12:17.7637082Z #7 ...
2021-07-16T18:12:17.7637405Z 
2021-07-16T18:12:17.7638435Z #5 [1/1] COPY bin/submariner-operator /submariner-operator
2021-07-16T18:12:17.7639847Z #5 sha256:24c8fd7c2e874f26b1a9f5f9b4125b6a700b6cefe662dc47967c957f28286fa4
2021-07-16T18:12:17.7641377Z #5 sha256:16b3838259a10843876d2a8b66d213c56265837cb98c4ad6401db64d9d685857 9.62MB / 9.62MB 0.1s done
2021-07-16T18:12:17.7642271Z #5 CACHED
2021-07-16T18:12:17.7642624Z 
2021-07-16T18:12:17.7643097Z #7 exporting to oci image format
2021-07-16T18:12:17.7644467Z #7 sha256:69a2560eef4d3ece902c3b5149d142e9bd132f25db0bc9e35b94201534c415d2
2021-07-16T18:12:17.7645589Z #7 sending tarball
2021-07-16T18:12:17.7646162Z #7 sending tarball 0.2s done
2021-07-16T18:12:17.7646687Z #7 DONE 0.4s
2021-07-16T18:12:17.7647043Z 
2021-07-16T18:12:17.7647538Z #9 exporting cache
2021-07-16T18:12:17.7648616Z #9 sha256:2700d4ef94dee473593c5c614b55b2dedcca7893909811a8f2b48291a1f581e4
2021-07-16T18:12:17.7649813Z #9 preparing build cache for export done
2021-07-16T18:12:17.7650413Z #9 DONE 0.0s
2021-07-16T18:12:17.7650750Z 
2021-07-16T18:12:17.7651181Z #8 importing to docker
2021-07-16T18:12:17.7652304Z #8 sha256:f52da3d31c433290f923de47465f3a704bfdb720ad0caba6d4ef2cf6d935afed
2021-07-16T18:12:17.7653355Z #8 DONE 0.0s
2021-07-16T18:12:17.7654847Z [submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev quay.io/submariner/submariner-operator:devel
2021-07-16T18:12:17.9823395Z         Successfully remade target file 'package/.image.submariner-operator'.
2021-07-16T18:12:17.9824500Z          File 'build' does not exist.
2021-07-16T18:12:17.9825217Z            File 'bin/subctl' does not exist.
2021-07-16T18:12:17.9826275Z              File 'bin/subctl-devel-0aaed33-linux-amd64' does not exist.
2021-07-16T18:12:18.0084788Z             Must remake target 'bin/subctl-devel-0aaed33-linux-amd64'.
2021-07-16T18:12:18.0085981Z mkdir -p bin
2021-07-16T18:12:18.0109200Z target=bin/subctl-devel-0aaed33-linux-amd64; \
2021-07-16T18:12:18.0112658Z target=${target%.exe}; \
2021-07-16T18:12:18.0113420Z components=($(echo ${target//-/ })); \
2021-07-16T18:12:18.0114106Z GOOS=${components[-2]}; \
2021-07-16T18:12:18.0114746Z GOARCH=${components[-1]}; \
2021-07-16T18:12:18.0115260Z export GOARCH GOOS; \
2021-07-16T18:12:18.0115812Z /opt/shipyard/scripts/compile.sh \
2021-07-16T18:12:18.0117057Z 	--ldflags "-X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33 \
2021-07-16T18:12:18.0119146Z 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=${DEFAULT_IMAGE_VERSION#v}" \
2021-07-16T18:12:18.0121274Z 	--noupx bin/subctl-devel-0aaed33-linux-amd64 ./pkg/subctl/main.go 
2021-07-16T18:12:18.0668450Z [submariner-operator]$ mkdir -p bin
2021-07-16T18:12:18.0688286Z Building 'bin/subctl-devel-0aaed33-linux-amd64' (ldflags: $'-X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33 \t\t   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel')
2021-07-16T18:12:18.0697902Z [submariner-operator]$ ldflags=-s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel
2021-07-16T18:12:18.0708019Z [submariner-operator]$ CGO_ENABLED=0 go build -trimpath -ldflags -s -w -X github.com/submariner-io/submariner-operator/pkg/version.Version=devel-0aaed33 		   -X=github.com/submariner-io/submariner-operator/pkg/versions.DefaultSubmarinerOperatorVersion=devel -o bin/subctl-devel-0aaed33-linux-amd64 ./pkg/subctl/main.go
2021-07-16T18:12:55.3627143Z             Successfully remade target file 'bin/subctl-devel-0aaed33-linux-amd64'.
2021-07-16T18:12:55.3628745Z           Must remake target 'bin/subctl'.
2021-07-16T18:12:55.3629961Z ln -sf subctl-devel-0aaed33-linux-amd64 bin/subctl
2021-07-16T18:12:55.3659911Z           Successfully remade target file 'bin/subctl'.
2021-07-16T18:12:55.3661023Z         Must remake target 'build'.
2021-07-16T18:12:55.3661924Z         Successfully remade target file 'build'.
2021-07-16T18:12:55.3663221Z       Must remake target 'images'.
2021-07-16T18:12:55.3664152Z       Successfully remade target file 'images'.
2021-07-16T18:12:55.3665105Z     Must remake target 'preload-images'.
2021-07-16T18:12:55.3665879Z set -e;\
2021-07-16T18:12:55.3666484Z . /opt/shipyard/scripts/lib/deploy_funcs;\
2021-07-16T18:12:55.3667243Z . /opt/shipyard/scripts/lib/debug_functions;\
2021-07-16T18:12:55.3668913Z for image in submariner-operator submariner-gateway submariner-route-agent lighthouse-agent lighthouse-coredns; do\
2021-07-16T18:12:55.3670321Z 	import_image quay.io/submariner/${image};\
2021-07-16T18:12:55.3670936Z done
2021-07-16T18:12:55.3713727Z [submariner-operator]$ import_image quay.io/submariner/submariner-operator
2021-07-16T18:12:55.3723515Z [submariner-operator]$ import_image quay.io/submariner/submariner-operator
2021-07-16T18:12:55.3740372Z [submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev localhost:5000/submariner-operator:local
2021-07-16T18:12:55.6591233Z [submariner-operator]$ docker push localhost:5000/submariner-operator:local
2021-07-16T18:12:55.9567131Z The push refers to repository [localhost:5000/submariner-operator]
2021-07-16T18:12:56.0018878Z 7dd6c3b905fa: Preparing
2021-07-16T18:12:56.6409740Z 7dd6c3b905fa: Pushed
2021-07-16T18:12:56.6643030Z local: digest: sha256:3e2788fae3e11a10b9728e6797f3ba25c5d4c6aa583e9c0095a169b2bc33572a size: 528
2021-07-16T18:12:56.6682967Z [submariner-operator]$ import_image quay.io/submariner/submariner-gateway
2021-07-16T18:12:56.6697965Z [submariner-operator]$ import_image quay.io/submariner/submariner-gateway
2021-07-16T18:12:56.6707616Z [submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:dev localhost:5000/submariner-gateway:local
2021-07-16T18:12:56.9515295Z Error response from daemon: No such image: quay.io/submariner/submariner-gateway:dev
2021-07-16T18:12:56.9538819Z [submariner-operator]$ docker pull quay.io/submariner/submariner-gateway:devel
2021-07-16T18:12:57.4170803Z devel: Pulling from submariner/submariner-gateway
2021-07-16T18:12:57.4171579Z 8c741e919b67: Pulling fs layer
2021-07-16T18:12:57.4172092Z 13495efbbac4: Pulling fs layer
2021-07-16T18:12:57.4172628Z 6df6df193f2d: Pulling fs layer
2021-07-16T18:12:57.4173129Z 747167f8325e: Pulling fs layer
2021-07-16T18:12:57.4176510Z 747167f8325e: Waiting
2021-07-16T18:12:57.4661280Z 13495efbbac4: Verifying Checksum
2021-07-16T18:12:57.4664888Z 13495efbbac4: Download complete
2021-07-16T18:12:57.6253550Z 747167f8325e: Verifying Checksum
2021-07-16T18:12:57.6254194Z 747167f8325e: Download complete
2021-07-16T18:12:57.6264617Z 6df6df193f2d: Verifying Checksum
2021-07-16T18:12:57.6265274Z 6df6df193f2d: Download complete
2021-07-16T18:12:58.0401174Z 8c741e919b67: Verifying Checksum
2021-07-16T18:12:58.0401797Z 8c741e919b67: Download complete
2021-07-16T18:13:01.3846253Z 8c741e919b67: Pull complete
2021-07-16T18:13:02.4846017Z 13495efbbac4: Pull complete
2021-07-16T18:13:03.5783937Z 6df6df193f2d: Pull complete
2021-07-16T18:13:03.7658195Z 747167f8325e: Pull complete
2021-07-16T18:13:03.7699947Z Digest: sha256:c57547568ee3a8d71547861c0c114d06078afd8749343c1a18593277e3e2266a
2021-07-16T18:13:03.7718852Z Status: Downloaded newer image for quay.io/submariner/submariner-gateway:devel
2021-07-16T18:13:03.7756311Z quay.io/submariner/submariner-gateway:devel
2021-07-16T18:13:03.7768387Z [submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:devel quay.io/submariner/submariner-gateway:dev
2021-07-16T18:13:04.0644498Z [submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:dev localhost:5000/submariner-gateway:local
2021-07-16T18:13:04.3708286Z [submariner-operator]$ docker push localhost:5000/submariner-gateway:local
2021-07-16T18:13:04.6548764Z The push refers to repository [localhost:5000/submariner-gateway]
2021-07-16T18:13:04.6572174Z f705cfc1afff: Preparing
2021-07-16T18:13:04.6572750Z 3c7f2d60eacb: Preparing
2021-07-16T18:13:04.6573236Z 00c5f6897675: Preparing
2021-07-16T18:13:04.6573772Z b3b62c837788: Preparing
2021-07-16T18:13:05.3128850Z 00c5f6897675: Pushed
2021-07-16T18:13:05.6328124Z f705cfc1afff: Pushed
2021-07-16T18:13:09.1298709Z 3c7f2d60eacb: Pushed
2021-07-16T18:13:17.4635481Z b3b62c837788: Pushed
2021-07-16T18:13:17.4874751Z local: digest: sha256:c57547568ee3a8d71547861c0c114d06078afd8749343c1a18593277e3e2266a size: 1159
2021-07-16T18:13:17.4928293Z [submariner-operator]$ import_image quay.io/submariner/submariner-route-agent
2021-07-16T18:13:17.4929871Z [submariner-operator]$ import_image quay.io/submariner/submariner-route-agent
2021-07-16T18:13:17.4942236Z [submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local
2021-07-16T18:13:18.2766772Z Error response from daemon: No such image: quay.io/submariner/submariner-route-agent:dev
2021-07-16T18:13:18.2768281Z devel: Pulling from submariner/submariner-route-agent
2021-07-16T18:13:18.2771832Z [submariner-operator]$ docker pull quay.io/submariner/submariner-route-agent:devel
2021-07-16T18:13:18.2773524Z 8c741e919b67: Already exists
2021-07-16T18:13:18.2774132Z 13495efbbac4: Already exists
2021-07-16T18:13:18.2775129Z 70cc6422093d: Pulling fs layer
2021-07-16T18:13:18.2778056Z 0da2ffe78276: Pulling fs layer
2021-07-16T18:13:18.2778658Z cae561c40adc: Pulling fs layer
2021-07-16T18:13:18.2779485Z 18cad3c4caf1: Pulling fs layer
2021-07-16T18:13:18.2779992Z 18cad3c4caf1: Waiting
2021-07-16T18:13:18.3187081Z cae561c40adc: Verifying Checksum
2021-07-16T18:13:18.3187741Z cae561c40adc: Download complete
2021-07-16T18:13:18.4264087Z 0da2ffe78276: Verifying Checksum
2021-07-16T18:13:18.4265174Z 0da2ffe78276: Download complete
2021-07-16T18:13:18.4303412Z 18cad3c4caf1: Verifying Checksum
2021-07-16T18:13:18.4304251Z 18cad3c4caf1: Download complete
2021-07-16T18:13:18.4939650Z 70cc6422093d: Verifying Checksum
2021-07-16T18:13:18.4940742Z 70cc6422093d: Download complete
2021-07-16T18:13:19.9385297Z 70cc6422093d: Pull complete
2021-07-16T18:13:20.1118040Z 0da2ffe78276: Pull complete
2021-07-16T18:13:20.1839632Z cae561c40adc: Pull complete
2021-07-16T18:13:20.2597468Z 18cad3c4caf1: Pull complete
2021-07-16T18:13:20.2639593Z Digest: sha256:2e4b027dc05037363b3b878223a31256be2838d3c5c417088b167968f2c4f7f1
2021-07-16T18:13:20.2662294Z Status: Downloaded newer image for quay.io/submariner/submariner-route-agent:devel
2021-07-16T18:13:20.2696187Z [submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:devel quay.io/submariner/submariner-route-agent:dev
2021-07-16T18:13:20.2697850Z quay.io/submariner/submariner-route-agent:devel
2021-07-16T18:13:20.5671705Z [submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local
2021-07-16T18:13:20.8556695Z [submariner-operator]$ docker push localhost:5000/submariner-route-agent:local
2021-07-16T18:13:21.1404062Z The push refers to repository [localhost:5000/submariner-route-agent]
2021-07-16T18:13:21.1449903Z 638361ed1a86: Preparing
2021-07-16T18:13:21.1450510Z 1d78e2cccc9b: Preparing
2021-07-16T18:13:21.1451035Z aa5aa8bd86ba: Preparing
2021-07-16T18:13:21.1451546Z 6931e591f8e2: Preparing
2021-07-16T18:13:21.1452008Z 00c5f6897675: Preparing
2021-07-16T18:13:21.1452491Z b3b62c837788: Preparing
2021-07-16T18:13:21.1452956Z b3b62c837788: Waiting
2021-07-16T18:13:21.3764156Z 00c5f6897675: Mounted from submariner-gateway
2021-07-16T18:13:21.9023736Z 1d78e2cccc9b: Pushed
2021-07-16T18:13:21.9208539Z 638361ed1a86: Pushed
2021-07-16T18:13:21.9276926Z b3b62c837788: Mounted from submariner-gateway
2021-07-16T18:13:21.9312855Z aa5aa8bd86ba: Pushed
2021-07-16T18:13:26.1112482Z 6931e591f8e2: Pushed
2021-07-16T18:13:26.1350971Z local: digest: sha256:2e4b027dc05037363b3b878223a31256be2838d3c5c417088b167968f2c4f7f1 size: 1575
2021-07-16T18:13:26.1390227Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-agent
2021-07-16T18:13:26.1402450Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-agent
2021-07-16T18:13:26.1417346Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:dev localhost:5000/lighthouse-agent:local
2021-07-16T18:13:26.4445342Z Error response from daemon: No such image: quay.io/submariner/lighthouse-agent:dev
2021-07-16T18:13:26.4482649Z [submariner-operator]$ docker pull quay.io/submariner/lighthouse-agent:devel
2021-07-16T18:13:26.9221261Z devel: Pulling from submariner/lighthouse-agent
2021-07-16T18:13:26.9227395Z ecd69139dc76: Pulling fs layer
2021-07-16T18:13:26.9231419Z 8261c1767de3: Pulling fs layer
2021-07-16T18:13:26.9952477Z ecd69139dc76: Verifying Checksum
2021-07-16T18:13:26.9953120Z ecd69139dc76: Download complete
2021-07-16T18:13:27.0316359Z 8261c1767de3: Verifying Checksum
2021-07-16T18:13:27.0316952Z 8261c1767de3: Download complete
2021-07-16T18:13:27.0925401Z ecd69139dc76: Pull complete
2021-07-16T18:13:27.2778911Z 8261c1767de3: Pull complete
2021-07-16T18:13:27.2838367Z Digest: sha256:93957d5e6a2d72de099dcf9f589a4321618b426900e681d3837a17c64c38d69f
2021-07-16T18:13:27.2859464Z Status: Downloaded newer image for quay.io/submariner/lighthouse-agent:devel
2021-07-16T18:13:27.2886240Z quay.io/submariner/lighthouse-agent:devel
2021-07-16T18:13:27.2901932Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:devel quay.io/submariner/lighthouse-agent:dev
2021-07-16T18:13:27.5853650Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:dev localhost:5000/lighthouse-agent:local
2021-07-16T18:13:27.8793259Z [submariner-operator]$ docker push localhost:5000/lighthouse-agent:local
2021-07-16T18:13:28.1690586Z The push refers to repository [localhost:5000/lighthouse-agent]
2021-07-16T18:13:28.1823925Z 83c79e393e11: Preparing
2021-07-16T18:13:28.1824522Z 0026d2f62bbf: Preparing
2021-07-16T18:13:28.2410555Z 0026d2f62bbf: Pushed
2021-07-16T18:13:28.7348255Z 83c79e393e11: Pushed
2021-07-16T18:13:28.7578609Z local: digest: sha256:93957d5e6a2d72de099dcf9f589a4321618b426900e681d3837a17c64c38d69f size: 735
2021-07-16T18:13:28.7617499Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns
2021-07-16T18:13:28.7631254Z [submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns
2021-07-16T18:13:28.7647348Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:dev localhost:5000/lighthouse-coredns:local
2021-07-16T18:13:29.0518062Z Error response from daemon: No such image: quay.io/submariner/lighthouse-coredns:dev
2021-07-16T18:13:29.0548396Z [submariner-operator]$ docker pull quay.io/submariner/lighthouse-coredns:devel
2021-07-16T18:13:29.5611624Z devel: Pulling from submariner/lighthouse-coredns
2021-07-16T18:13:29.5612425Z 088e56360f76: Pulling fs layer
2021-07-16T18:13:29.5612948Z 18aed70882d2: Pulling fs layer
2021-07-16T18:13:29.6164829Z 088e56360f76: Verifying Checksum
2021-07-16T18:13:29.6165655Z 088e56360f76: Download complete
2021-07-16T18:13:29.7176312Z 088e56360f76: Pull complete
2021-07-16T18:13:29.8462297Z 18aed70882d2: Verifying Checksum
2021-07-16T18:13:29.8463010Z 18aed70882d2: Download complete
2021-07-16T18:13:30.0860918Z 18aed70882d2: Pull complete
2021-07-16T18:13:30.0902992Z Digest: sha256:d7e32908d5d6c680b813e0444db977c5b7578f7480fc650826c6dc993bbce595
2021-07-16T18:13:30.0925753Z Status: Downloaded newer image for quay.io/submariner/lighthouse-coredns:devel
2021-07-16T18:13:30.0969456Z quay.io/submariner/lighthouse-coredns:devel
2021-07-16T18:13:30.1007067Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:devel quay.io/submariner/lighthouse-coredns:dev
2021-07-16T18:13:30.3871564Z [submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:dev localhost:5000/lighthouse-coredns:local
2021-07-16T18:13:30.6733020Z [submariner-operator]$ docker push localhost:5000/lighthouse-coredns:local
2021-07-16T18:13:30.9553919Z The push refers to repository [localhost:5000/lighthouse-coredns]
2021-07-16T18:13:30.9587329Z 7fca351e51be: Preparing
2021-07-16T18:13:30.9587964Z 4fa0ece06e25: Preparing
2021-07-16T18:13:31.0028923Z 4fa0ece06e25: Pushed
2021-07-16T18:13:31.7465962Z 7fca351e51be: Pushed
2021-07-16T18:13:31.7772808Z local: digest: sha256:d7e32908d5d6c680b813e0444db977c5b7578f7480fc650826c6dc993bbce595 size: 739
2021-07-16T18:13:31.7821922Z     Successfully remade target file 'preload-images'.
2021-07-16T18:13:31.7846298Z    Prerequisite 'clusters' of target 'deploy' does not exist.
2021-07-16T18:13:31.7847460Z    Prerequisite 'preload-images' of target 'deploy' does not exist.
2021-07-16T18:13:31.7849366Z    Prerequisite 'bin/subctl' is newer than target 'deploy'.
2021-07-16T18:13:31.7850224Z   Must remake target 'deploy'.
2021-07-16T18:13:31.7852346Z /opt/shipyard/scripts/deploy.sh --cluster_settings /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-16T18:13:31.8852026Z Running with: globalnet='false', deploytool='operator', deploytool_broker_args='', deploytool_submariner_args='', cluster_settings='/go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings', timeout=5m, image_tag=local, cable_driver=libreswan, service_discovery=false
2021-07-16T18:13:31.8888240Z [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
2021-07-16T18:13:31.8907118Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:13:31.8919785Z [submariner-operator]$ script_name=utils
2021-07-16T18:13:31.8934301Z [submariner-operator]$ exec_name=deploy.sh
2021-07-16T18:13:31.8980730Z [submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs
2021-07-16T18:13:31.8994593Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:13:31.9065650Z [submariner-operator]$ script_name=deploy_funcs
2021-07-16T18:13:31.9087392Z [submariner-operator]$ exec_name=deploy.sh
2021-07-16T18:13:31.9096433Z [submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings
2021-07-16T18:13:31.9111491Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:13:31.9122777Z [submariner-operator]$ script_name=cluster_settings
2021-07-16T18:13:31.9134550Z [submariner-operator]$ exec_name=deploy.sh
2021-07-16T18:13:31.9157586Z [submariner-operator]$ broker=cluster1
2021-07-16T18:13:31.9171285Z [submariner-operator]$ declare -gA cluster_nodes
2021-07-16T18:13:31.9185749Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker
2021-07-16T18:13:31.9197413Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker
2021-07-16T18:13:31.9215455Z [submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker
2021-07-16T18:13:31.9229780Z [submariner-operator]$ declare -gA cluster_subm
2021-07-16T18:13:31.9244754Z [submariner-operator]$ cluster_subm[cluster1]=true
2021-07-16T18:13:31.9255435Z [submariner-operator]$ cluster_subm[cluster2]=true
2021-07-16T18:13:31.9270175Z [submariner-operator]$ cluster_subm[cluster3]=true
2021-07-16T18:13:31.9284169Z [submariner-operator]$ declare -gA cluster_cni
2021-07-16T18:13:31.9296265Z [submariner-operator]$ cluster_cni[cluster2]=weave
2021-07-16T18:13:31.9309991Z [submariner-operator]$ cluster_cni[cluster3]=weave
2021-07-16T18:13:31.9325259Z [submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-16T18:13:31.9340667Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:13:31.9354615Z [submariner-operator]$ script_name=cluster_settings
2021-07-16T18:13:31.9368945Z [submariner-operator]$ exec_name=deploy.sh
2021-07-16T18:13:31.9392910Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker
2021-07-16T18:13:31.9408238Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker
2021-07-16T18:13:31.9440893Z [submariner-operator]$ declare_cidrs
2021-07-16T18:13:31.9452536Z [submariner-operator]$ declare_cidrs
2021-07-16T18:13:31.9467944Z [submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
2021-07-16T18:13:31.9482599Z [submariner-operator]$ i=1
2021-07-16T18:13:31.9498700Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-16T18:13:31.9511913Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-16T18:13:31.9528813Z [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16
2021-07-16T18:13:31.9540207Z [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16
2021-07-16T18:13:31.9555930Z [submariner-operator]$ [cluster1] i=$(("$i"+1))
2021-07-16T18:13:31.9571830Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-16T18:13:31.9586877Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-16T18:13:31.9599581Z [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16
2021-07-16T18:13:31.9614541Z [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16
2021-07-16T18:13:31.9628009Z [submariner-operator]$ [cluster2] i=$(("$i"+1))
2021-07-16T18:13:31.9645956Z [submariner-operator]$ [cluster2] declare_kubeconfig
2021-07-16T18:13:31.9665146Z [submariner-operator]$ [cluster2] declare_kubeconfig
2021-07-16T18:13:31.9679001Z [submariner-operator]$ [cluster2] source /opt/shipyard/scripts/lib/kubecfg
2021-07-16T18:13:31.9708182Z [submariner-operator]$ [cluster2] export KUBECONFIG
2021-07-16T18:13:31.9726409Z [submariner-operator]$ [cluster2] KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)
2021-07-16T18:13:31.9751576Z [submariner-operator]$ [cluster2] find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:
2021-07-16T18:13:31.9792098Z [submariner-operator]$ [cluster2] bash -c curl -Ls https://get.submariner.io | VERSION=devel DESTDIR=/go/bin bash
2021-07-16T18:13:32.0954126Z Installing subctl version devel
2021-07-16T18:13:32.0954794Z   OS detected:           linux
2021-07-16T18:13:32.0955345Z   Architecture detected: amd64
2021-07-16T18:13:32.0962597Z   Download URL:          https://github.com/submariner-io/submariner-operator/releases/download/subctl-devel/subctl-devel-linux-amd64.tar.xz
2021-07-16T18:13:32.0964167Z 
2021-07-16T18:13:32.0964593Z Downloading...
2021-07-16T18:13:33.1959118Z subctl-devel-linux-amd64 has been installed as /go/bin/subctl
2021-07-16T18:13:33.2168602Z This provides subctl version: devel
2021-07-16T18:13:33.2197227Z 
2021-07-16T18:13:33.2198339Z please update your path (and consider adding it to ~/.profile):
2021-07-16T18:13:33.2199119Z     export PATH=$PATH:/go/bin
2021-07-16T18:13:33.2199553Z     
2021-07-16T18:13:33.2312418Z [submariner-operator]$ [cluster2] load_deploytool operator
2021-07-16T18:13:33.2328553Z [submariner-operator]$ [cluster2] load_deploytool operator
2021-07-16T18:13:33.2335419Z Will deploy submariner using operator
2021-07-16T18:13:33.2348831Z [submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/deploy_operator
2021-07-16T18:13:33.2370516Z [submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:13:33.2391837Z [submariner-operator]$ [cluster2] script_name=deploy_operator
2021-07-16T18:13:33.2405408Z [submariner-operator]$ [cluster2] exec_name=deploy.sh
2021-07-16T18:13:33.2426219Z [submariner-operator]$ [cluster2] deploytool_prereqs
2021-07-16T18:13:33.2438926Z [submariner-operator]$ [cluster2] deploytool_prereqs
2021-07-16T18:13:33.2456920Z 
2021-07-16T18:13:33.2473652Z [submariner-operator]$ [cluster2] subctl version
2021-07-16T18:13:33.2715818Z subctl version: devel-0aaed33
2021-07-16T18:13:33.2752250Z [submariner-operator]$ [cluster2] run_if_defined pre_deploy
2021-07-16T18:13:33.2765119Z [submariner-operator]$ [cluster2] run_if_defined pre_deploy
2021-07-16T18:13:33.2778334Z [submariner-operator]$ [cluster2] type -t pre_deploy
2021-07-16T18:13:33.2794655Z [submariner-operator]$ [cluster2] grep -q function
2021-07-16T18:13:33.2826506Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:13:33.2841836Z [submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster
2021-07-16T18:13:33.2856285Z [submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster
2021-07-16T18:13:33.2870704Z [submariner-operator]$ [cluster2] declare -a subm_clusters
2021-07-16T18:13:33.2910911Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 prepare_cluster
2021-07-16T18:13:33.2928616Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 prepare_cluster
2021-07-16T18:13:33.2942007Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-16T18:13:33.2961127Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-16T18:13:33.2984053Z [submariner-operator]$ [cluster1] pids[cluster1]=7180
2021-07-16T18:13:33.2990377Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-16T18:13:33.3003300Z [submariner-operator]$ [cluster1] prepare_cluster
2021-07-16T18:13:33.3025025Z [submariner-operator]$ [cluster2] pids[cluster2]=7184
2021-07-16T18:13:33.3048217Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-16T18:13:33.3070433Z [submariner-operator]$ [cluster2] wait 7184
2021-07-16T18:13:33.3084499Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-16T18:13:33.3112084Z [submariner-operator]$ [cluster2] prepare_cluster
2021-07-16T18:13:33.3140389Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-16T18:13:33.7614128Z [submariner-operator]$ [cluster2] prepare_cluster
2021-07-16T18:13:33.7615327Z [submariner-operator]$ [cluster2] add_subm_gateway_label
2021-07-16T18:13:33.7616417Z [submariner-operator]$ [cluster2] add_subm_gateway_label
2021-07-16T18:13:33.7618013Z [submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
2021-07-16T18:13:33.7619783Z [submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
2021-07-16T18:13:33.7621664Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 label node cluster2-worker submariner.io/gateway=true --overwrite
2021-07-16T18:13:33.7623177Z [cluster2] node/cluster2-worker labeled
2021-07-16T18:13:33.7660814Z [submariner-operator]$ [cluster2] wait 7180
2021-07-16T18:13:33.7957254Z [submariner-operator]$ [cluster1] prepare_cluster
2021-07-16T18:13:33.7958568Z [submariner-operator]$ [cluster1] add_subm_gateway_label
2021-07-16T18:13:33.7959690Z [submariner-operator]$ [cluster1] add_subm_gateway_label
2021-07-16T18:13:33.7961124Z [submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
2021-07-16T18:13:33.7962856Z [submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
2021-07-16T18:13:33.7965510Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 label node cluster1-worker submariner.io/gateway=true --overwrite
2021-07-16T18:13:33.7966877Z [cluster1] node/cluster1-worker labeled
2021-07-16T18:13:33.7990120Z [submariner-operator]$ [cluster2] with_context cluster1 setup_broker
2021-07-16T18:13:33.8001845Z [submariner-operator]$ [cluster2] with_context cluster1 setup_broker
2021-07-16T18:13:33.8014507Z [submariner-operator]$ [cluster1] setup_broker
2021-07-16T18:13:33.8036842Z [submariner-operator]$ [cluster1] setup_broker
2021-07-16T18:13:33.8042013Z Installing broker...
2021-07-16T18:13:33.8054826Z [submariner-operator]$ [cluster1] subctlrepver=
2021-07-16T18:13:33.8066456Z [submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local
2021-07-16T18:13:33.8088189Z [submariner-operator]$ [cluster1] cd /go/src/github.com/submariner-io/submariner-operator/output
2021-07-16T18:13:33.8102181Z [output]$ [cluster1] subctl deploy-broker --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --repository localhost:5000 --version local
2021-07-16T18:13:33.8362528Z  • Setting up broker RBAC  ...
2021-07-16T18:13:33.9145401Z  ✓ Setting up broker RBAC
2021-07-16T18:13:33.9146307Z  • Deploying the Submariner operator  ...
2021-07-16T18:13:45.1761622Z  ✓ Deploying the Submariner operator
2021-07-16T18:13:45.1762653Z  ✓ Created operator CRDs
2021-07-16T18:13:45.1763720Z  ✓ Created operator namespace: submariner-operator
2021-07-16T18:13:45.1764676Z  ✓ Created operator service account and role
2021-07-16T18:13:45.1765532Z  ✓ Created lighthouse service account and role
2021-07-16T18:13:45.1766395Z  ✓ Created Lighthouse service accounts and roles
2021-07-16T18:13:45.1767419Z  ✓ Deployed the operator successfully
2021-07-16T18:13:45.1768147Z  • Deploying the broker  ...
2021-07-16T18:13:45.2561201Z  ✓ Deploying the broker
2021-07-16T18:13:45.2563120Z  ✓ The broker has been deployed
2021-07-16T18:13:45.2564280Z  • Creating broker-info.subm file  ...
2021-07-16T18:13:45.2649316Z  ✓ Creating broker-info.subm file
2021-07-16T18:13:45.2650357Z  ✓ A new IPsec PSK will be generated for broker-info.subm
2021-07-16T18:13:45.2706971Z [submariner-operator]$ [cluster2] install_subm_all_clusters
2021-07-16T18:13:45.2719932Z [submariner-operator]$ [cluster2] install_subm_all_clusters
2021-07-16T18:13:45.2734815Z [submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm
2021-07-16T18:13:45.2749047Z [submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm
2021-07-16T18:13:45.2760638Z [submariner-operator]$ [cluster2] declare -a subm_clusters
2021-07-16T18:13:45.2808454Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 subctl_install_subm
2021-07-16T18:13:45.2835683Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 subctl_install_subm
2021-07-16T18:13:45.2852794Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-16T18:13:45.2877122Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-16T18:13:45.2905920Z [submariner-operator]$ [cluster1] pids[cluster1]=7251
2021-07-16T18:13:45.2914704Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-16T18:13:45.2936605Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-16T18:13:45.2947025Z [submariner-operator]$ [cluster2] pids[cluster2]=7254
2021-07-16T18:13:45.2952734Z [submariner-operator]$ [cluster1] subctl_install_subm
2021-07-16T18:13:45.2969703Z [submariner-operator]$ [cluster2] subctl_install_subm
2021-07-16T18:13:45.2998749Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-16T18:13:45.3002029Z [submariner-operator]$ [cluster2] wait 7254
2021-07-16T18:13:45.3032071Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-16T18:13:52.3284743Z [submariner-operator]$ [cluster1] subctl_install_subm
2021-07-16T18:13:52.3285897Z [submariner-operator]$ [cluster1] subctlrepver=
2021-07-16T18:13:52.3287224Z [submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local
2021-07-16T18:13:52.3288492Z [submariner-operator]$ [cluster1] PASS_CIDR_ARGS=false
2021-07-16T18:13:52.3291706Z [submariner-operator]$ [cluster1] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --clusterid cluster1 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr  --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm
2021-07-16T18:13:52.3294993Z [cluster1] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.8:6443
2021-07-16T18:13:52.3296214Z [cluster1] * There are 1 labeled nodes in the cluster:
2021-07-16T18:13:52.3297032Z [cluster1]   - cluster1-worker
2021-07-16T18:13:52.3297866Z [cluster1]  • Discovering network details  ...
2021-07-16T18:13:52.3298697Z [cluster1]         Network plugin:  weave-net
2021-07-16T18:13:52.3299322Z [cluster1]         Service CIDRs:   [100.1.0.0/16]
2021-07-16T18:13:52.3299890Z [cluster1]         Cluster CIDRs:   [10.1.0.0/16]
2021-07-16T18:13:52.3300755Z [cluster1]  ✓ Discovering network details
2021-07-16T18:13:52.3301617Z [cluster1]  • Discovering multi cluster details  ...
2021-07-16T18:13:52.3302671Z [cluster1]  • Validating Globalnet configurations  ...
2021-07-16T18:13:52.3303642Z [cluster1]  ✓ Validating Globalnet configurations
2021-07-16T18:13:52.3304536Z [cluster1]  ✓ Discovering multi cluster details
2021-07-16T18:13:52.3305434Z [cluster1]  • Deploying the Submariner operator  ...
2021-07-16T18:13:52.3306331Z [cluster1]  ✓ Deploying the Submariner operator
2021-07-16T18:13:52.3307316Z [cluster1]  ✓ Created Lighthouse service accounts and roles
2021-07-16T18:13:52.3308178Z [cluster1]  • Creating SA for cluster  ...
2021-07-16T18:13:52.3308932Z [cluster1]  ✓ Creating SA for cluster
2021-07-16T18:13:52.3309680Z [cluster1]  • Deploying Submariner  ...
2021-07-16T18:13:52.3310462Z [cluster1]  ✓ Deploying Submariner
2021-07-16T18:13:52.3311225Z [cluster1]  ✓ Submariner is up and running
2021-07-16T18:13:58.3708274Z [submariner-operator]$ [cluster2] subctl_install_subm
2021-07-16T18:13:58.3709456Z [submariner-operator]$ [cluster2] subctlrepver=
2021-07-16T18:13:58.3710784Z [submariner-operator]$ [cluster2] subctlrepver=--repository localhost:5000 --version local
2021-07-16T18:13:58.3712051Z [submariner-operator]$ [cluster2] PASS_CIDR_ARGS=false
2021-07-16T18:13:58.3715551Z [submariner-operator]$ [cluster2] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2 --clusterid cluster2 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr  --natt=false --cable-driver libreswan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm
2021-07-16T18:13:58.3718905Z [cluster2] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.8:6443
2021-07-16T18:13:58.3720112Z [cluster2] * There are 1 labeled nodes in the cluster:
2021-07-16T18:13:58.3720954Z [cluster2]   - cluster2-worker
2021-07-16T18:13:58.3721798Z [cluster2]  • Discovering network details  ...
2021-07-16T18:13:58.3722618Z [cluster2]         Network plugin:  weave-net
2021-07-16T18:13:58.3723239Z [cluster2]         Service CIDRs:   [100.2.0.0/16]
2021-07-16T18:13:58.3724012Z [cluster2]         Cluster CIDRs:   [10.2.0.0/16]
2021-07-16T18:13:58.3725885Z [cluster2]  ✓ Discovering network details
2021-07-16T18:13:58.3726742Z [cluster2]  • Discovering multi cluster details  ...
2021-07-16T18:13:58.3728160Z [cluster2]  • Validating Globalnet configurations  ...
2021-07-16T18:13:58.3729124Z [cluster2]  ✓ Validating Globalnet configurations
2021-07-16T18:13:58.3730162Z [cluster2]  ✓ Discovering multi cluster details
2021-07-16T18:13:58.3731066Z [cluster2]  • Deploying the Submariner operator  ...
2021-07-16T18:13:58.3732002Z [cluster2]  ✓ Deploying the Submariner operator
2021-07-16T18:13:58.3732819Z [cluster2]  ✓ Created operator CRDs
2021-07-16T18:13:58.3733808Z [cluster2]  ✓ Created operator namespace: submariner-operator
2021-07-16T18:13:58.3735022Z [cluster2]  ✓ Created operator service account and role
2021-07-16T18:13:58.3735968Z [cluster2]  ✓ Created lighthouse service account and role
2021-07-16T18:13:58.3736961Z [cluster2]  ✓ Created Lighthouse service accounts and roles
2021-07-16T18:13:58.3738057Z [cluster2]  ✓ Deployed the operator successfully
2021-07-16T18:13:58.3738900Z [cluster2]  • Creating SA for cluster  ...
2021-07-16T18:13:58.3739644Z [cluster2]  ✓ Creating SA for cluster
2021-07-16T18:13:58.3740445Z [cluster2]  • Deploying Submariner  ...
2021-07-16T18:13:58.3741231Z [cluster2]  ✓ Deploying Submariner
2021-07-16T18:13:58.3741994Z [cluster2]  ✓ Submariner is up and running
2021-07-16T18:13:58.3792475Z [submariner-operator]$ [cluster2] wait 7251
2021-07-16T18:13:58.3956524Z [submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status
2021-07-16T18:13:58.4002241Z [submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status
2021-07-16T18:13:58.4024960Z [submariner-operator]$ [cluster2] with_retries 30 verify_gw_status
2021-07-16T18:13:58.4059439Z [submariner-operator]$ [cluster2] with_retries
2021-07-16T18:13:58.4096645Z [submariner-operator]$ [cluster2] iteration=1
2021-07-16T18:13:58.4147856Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:13:58.4182477Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:13:58.4190480Z [submariner-operator]$ [cluster2] wait 7295
2021-07-16T18:13:58.4204278Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-16T18:13:58.4271060Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:13:58.4306905Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-16T18:13:58.7242731Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:13:58.7267958Z [submariner-operator]$ [cluster2] grep connected
2021-07-16T18:13:59.1788216Z  • Showing Connections  ...
2021-07-16T18:13:59.4152313Z  ✗ Showing Connections
2021-07-16T18:13:59.4196139Z  ✗ There are no gateways detected
2021-07-16T18:13:59.4412843Z  • Showing Connections  ...
2021-07-16T18:13:59.4525692Z  ✗ Showing Connections
2021-07-16T18:13:59.4530715Z  ✗ There are no gateways detected
2021-07-16T18:13:59.4558111Z iter: 1. Clusters not yet connected. sleeping for 6 secs
2021-07-16T18:13:59.4574265Z [submariner-operator]$ [cluster2] sleep 6
2021-07-16T18:14:05.4614304Z [submariner-operator]$ [cluster2] return 1
2021-07-16T18:14:05.4666633Z [submariner-operator]$ [cluster2] wait 7322
2021-07-16T18:14:05.4717270Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:05.4751168Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:05.4761068Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-16T18:14:05.4809744Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:05.4889148Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-16T18:14:05.9968519Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:05.9987875Z [submariner-operator]$ [cluster2] grep connected
2021-07-16T18:14:06.1052741Z  • Showing Connections  ...
2021-07-16T18:14:06.1113356Z  ✗ Showing Connections
2021-07-16T18:14:06.1114359Z  ✗ There are no gateways detected
2021-07-16T18:14:06.1548706Z  • Showing Connections  ...
2021-07-16T18:14:06.1582750Z  ✗ Showing Connections
2021-07-16T18:14:06.1583488Z  ✗ There are no gateways detected
2021-07-16T18:14:06.1605661Z iter: 2. Clusters not yet connected. sleeping for 6 secs
2021-07-16T18:14:06.1619644Z [submariner-operator]$ [cluster2] sleep 6
2021-07-16T18:14:12.1679487Z [submariner-operator]$ [cluster2] return 1
2021-07-16T18:14:12.1758767Z [submariner-operator]$ [cluster2] wait 7348
2021-07-16T18:14:12.1772136Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:12.1786768Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:12.1813693Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-16T18:14:12.1980207Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:12.2008231Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-16T18:14:12.5518106Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:12.5571235Z [submariner-operator]$ [cluster2] grep connected
2021-07-16T18:14:12.7265939Z  • Showing Connections  ...
2021-07-16T18:14:12.7335410Z  ✗ Showing Connections
2021-07-16T18:14:12.7336138Z  ✗ No connections found
2021-07-16T18:14:12.8774792Z  • Showing Connections  ...
2021-07-16T18:14:12.8885960Z  ✗ Showing Connections
2021-07-16T18:14:12.8890481Z  ✗ There are no gateways detected
2021-07-16T18:14:12.8943022Z iter: 3. Clusters not yet connected. sleeping for 6 secs
2021-07-16T18:14:12.8979533Z [submariner-operator]$ [cluster2] sleep 6
2021-07-16T18:14:18.9012400Z [submariner-operator]$ [cluster2] return 1
2021-07-16T18:14:18.9059761Z [submariner-operator]$ [cluster2] wait 7375
2021-07-16T18:14:18.9085042Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:18.9097019Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:18.9112594Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-16T18:14:18.9129450Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:18.9176015Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-16T18:14:20.3858563Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:20.3873510Z [submariner-operator]$ [cluster2] grep connected
2021-07-16T18:14:20.5256135Z  • Showing Connections  ...
2021-07-16T18:14:20.5358728Z  ✗ Showing Connections
2021-07-16T18:14:20.5359425Z  ✗ No connections found
2021-07-16T18:14:20.6050203Z  • Showing Connections  ...
2021-07-16T18:14:20.6147970Z  ✗ Showing Connections
2021-07-16T18:14:20.6148768Z  ✗ There are no gateways detected
2021-07-16T18:14:20.6173842Z iter: 4. Clusters not yet connected. sleeping for 6 secs
2021-07-16T18:14:20.6187745Z [submariner-operator]$ [cluster2] sleep 6
2021-07-16T18:14:26.6215670Z [submariner-operator]$ [cluster2] return 1
2021-07-16T18:14:26.6262026Z [submariner-operator]$ [cluster2] wait 7402
2021-07-16T18:14:26.6280877Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:26.6294451Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:26.6358355Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-16T18:14:26.6370060Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:26.6385083Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-16T18:14:26.7532074Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:26.7547324Z [submariner-operator]$ [cluster2] grep connected
2021-07-16T18:14:26.9455288Z  • Showing Connections  ...
2021-07-16T18:14:26.9653979Z  ✗ Showing Connections
2021-07-16T18:14:26.9658913Z  ✗ No connections found
2021-07-16T18:14:26.9900535Z  • Showing Connections  ...
2021-07-16T18:14:26.9924051Z  ✗ Showing Connections
2021-07-16T18:14:26.9929306Z  ✗ There are no gateways detected
2021-07-16T18:14:26.9957535Z iter: 5. Clusters not yet connected. sleeping for 6 secs
2021-07-16T18:14:26.9975413Z [submariner-operator]$ [cluster2] sleep 6
2021-07-16T18:14:33.0020121Z [submariner-operator]$ [cluster2] return 1
2021-07-16T18:14:33.0080632Z [submariner-operator]$ [cluster2] wait 7429
2021-07-16T18:14:33.0091363Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:33.0107591Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:33.0122096Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-16T18:14:33.0138726Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:33.0177831Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-16T18:14:33.4337530Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:33.4373728Z [submariner-operator]$ [cluster2] grep connected
2021-07-16T18:14:33.5647588Z  • Showing Connections  ...
2021-07-16T18:14:33.5785677Z  ✗ Showing Connections
2021-07-16T18:14:33.5790542Z  ✗ No connections found
2021-07-16T18:14:33.6097497Z  • Showing Connections  ...
2021-07-16T18:14:33.6792384Z  ✗ Showing Connections
2021-07-16T18:14:33.6796687Z  ✗ No connections found
2021-07-16T18:14:33.6824089Z iter: 6. Clusters not yet connected. sleeping for 6 secs
2021-07-16T18:14:33.6839839Z [submariner-operator]$ [cluster2] sleep 6
2021-07-16T18:14:39.6866742Z [submariner-operator]$ [cluster2] return 1
2021-07-16T18:14:39.6905670Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:39.6907005Z [submariner-operator]$ [cluster2] wait 7455
2021-07-16T18:14:39.6921771Z [submariner-operator]$ [cluster2] verify_gw_status
2021-07-16T18:14:39.6935628Z [submariner-operator]$ [cluster2] sleep_duration=6
2021-07-16T18:14:39.6948638Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:39.6967333Z [submariner-operator]$ [cluster2] grep the server could not find the requested resource
2021-07-16T18:14:39.8025123Z [submariner-operator]$ [cluster2] subctl show connections
2021-07-16T18:14:39.8043383Z [submariner-operator]$ [cluster2] grep connected
2021-07-16T18:14:39.8831698Z  • Showing Connections  ...
2021-07-16T18:14:39.8866206Z  ✓ Showing Connections
2021-07-16T18:14:39.8914238Z  • Showing Connections  ...
2021-07-16T18:14:39.8968448Z  ✓ Showing Connections
2021-07-16T18:14:39.9048199Z cluster1-worker  cluster1  172.18.0.9  no   libreswan     100.1.0.0/16, 10.1.0.0/16  connected  0s        
2021-07-16T18:14:39.9049471Z cluster2-worker  cluster2  172.18.0.5  no   libreswan     100.2.0.0/16, 10.2.0.0/16  connected  588.716µs   
2021-07-16T18:14:39.9063919Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:14:39.9091522Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:14:39.9109761Z [submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1
2021-07-16T18:14:39.9126843Z [submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1
2021-07-16T18:14:39.9146295Z [submariner-operator]$ [cluster2] connectivity_tests cluster1
2021-07-16T18:14:39.9163518Z [submariner-operator]$ [cluster2] connectivity_tests
2021-07-16T18:14:39.9188978Z [submariner-operator]$ [cluster2] target_cluster=cluster1
2021-07-16T18:14:39.9207855Z [submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest
2021-07-16T18:14:39.9225298Z [submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest
2021-07-16T18:14:39.9247523Z [submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local
2021-07-16T18:14:40.2484053Z Error response from daemon: No such image: quay.io/submariner/nettest:dev
2021-07-16T18:14:40.2541619Z [submariner-operator]$ [cluster2] docker pull quay.io/submariner/nettest:devel
2021-07-16T18:14:40.7661695Z devel: Pulling from submariner/nettest
2021-07-16T18:14:40.7663241Z 5843afab3874: Pulling fs layer
2021-07-16T18:14:40.7663807Z 9565bb92f43f: Pulling fs layer
2021-07-16T18:14:40.7664406Z a31c0d5b4ad6: Pulling fs layer
2021-07-16T18:14:40.7664926Z dc3d13650205: Pulling fs layer
2021-07-16T18:14:40.7667493Z dc3d13650205: Waiting
2021-07-16T18:14:40.8456877Z 9565bb92f43f: Download complete
2021-07-16T18:14:40.8843683Z 5843afab3874: Verifying Checksum
2021-07-16T18:14:40.8844339Z 5843afab3874: Download complete
2021-07-16T18:14:40.9319739Z dc3d13650205: Verifying Checksum
2021-07-16T18:14:40.9321133Z dc3d13650205: Download complete
2021-07-16T18:14:40.9551217Z a31c0d5b4ad6: Verifying Checksum
2021-07-16T18:14:40.9551872Z a31c0d5b4ad6: Download complete
2021-07-16T18:14:41.2508220Z 5843afab3874: Pull complete
2021-07-16T18:14:41.3918816Z 9565bb92f43f: Pull complete
2021-07-16T18:14:41.8450340Z a31c0d5b4ad6: Pull complete
2021-07-16T18:14:41.9460730Z dc3d13650205: Pull complete
2021-07-16T18:14:41.9508697Z Digest: sha256:ef019e5e56af44d25e621cd556d8f40edcd641d0acd17c307486b4f4047b848f
2021-07-16T18:14:41.9532628Z Status: Downloaded newer image for quay.io/submariner/nettest:devel
2021-07-16T18:14:41.9542247Z quay.io/submariner/nettest:devel
2021-07-16T18:14:41.9601986Z [submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:devel quay.io/submariner/nettest:dev
2021-07-16T18:14:42.2831182Z [submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local
2021-07-16T18:14:42.6169825Z [submariner-operator]$ [cluster2] docker push localhost:5000/nettest:local
2021-07-16T18:14:42.9169866Z The push refers to repository [localhost:5000/nettest]
2021-07-16T18:14:42.9230199Z 4f2cdf68c81d: Preparing
2021-07-16T18:14:42.9230758Z 1f803d62c8b3: Preparing
2021-07-16T18:14:42.9231277Z 80c9eaab4c9e: Preparing
2021-07-16T18:14:42.9231802Z 72e830a4dff5: Preparing
2021-07-16T18:14:43.5367623Z 80c9eaab4c9e: Pushed
2021-07-16T18:14:43.5394499Z 4f2cdf68c81d: Pushed
2021-07-16T18:14:43.5530583Z 72e830a4dff5: Pushed
2021-07-16T18:14:44.2646793Z 1f803d62c8b3: Pushed
2021-07-16T18:14:44.2838498Z local: digest: sha256:ef019e5e56af44d25e621cd556d8f40edcd641d0acd17c307486b4f4047b848f size: 1155
2021-07-16T18:14:44.2919958Z [submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:44.2930563Z [submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:44.2949251Z [submariner-operator]$ [cluster2] resource_name=$(basename "$resource_file" ".yaml")
2021-07-16T18:14:44.2969852Z [submariner-operator]$ [cluster2] basename /opt/shipyard/scripts/resources/netshoot.yaml .yaml
2021-07-16T18:14:44.2996754Z [submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:44.3011970Z [submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:44.3069930Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:44.8204111Z deployment.apps/netshoot created
2021-07-16T18:14:44.8295742Z Waiting for netshoot pods to be ready.
2021-07-16T18:14:44.8324216Z [submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m
2021-07-16T18:14:44.8337296Z [submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m
2021-07-16T18:14:44.8352033Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 rollout status deploy/netshoot --timeout=5m
2021-07-16T18:14:45.2208989Z Waiting for deployment "netshoot" rollout to finish: 0 of 1 updated replicas are available...
2021-07-16T18:14:46.6794775Z deployment "netshoot" successfully rolled out
2021-07-16T18:14:46.6851550Z [submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:46.6866623Z [submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:46.6886044Z [submariner-operator]$ [cluster1] deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:46.6903357Z [submariner-operator]$ [cluster1] deploy_resource
2021-07-16T18:14:46.6922826Z [submariner-operator]$ [cluster1] resource_name=$(basename "$resource_file" ".yaml")
2021-07-16T18:14:46.6945606Z [submariner-operator]$ [cluster1] basename /opt/shipyard/scripts/resources/nginx-demo.yaml .yaml
2021-07-16T18:14:46.6968947Z [submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:46.6982528Z [submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:46.6998785Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:47.4149569Z deployment.apps/nginx-demo created
2021-07-16T18:14:47.4438196Z service/nginx-demo created
2021-07-16T18:14:47.6067680Z serviceexport.multicluster.x-k8s.io/nginx-demo created
2021-07-16T18:14:47.6133781Z Waiting for nginx-demo pods to be ready.
2021-07-16T18:14:47.6145043Z [submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m
2021-07-16T18:14:47.6156199Z [submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m
2021-07-16T18:14:47.6170719Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 rollout status deploy/nginx-demo --timeout=5m
2021-07-16T18:14:48.3571282Z Waiting for deployment "nginx-demo" rollout to finish: 0 of 2 updated replicas are available...
2021-07-16T18:14:53.9600139Z Waiting for deployment "nginx-demo" rollout to finish: 1 of 2 updated replicas are available...
2021-07-16T18:14:54.4246954Z deployment "nginx-demo" successfully rolled out
2021-07-16T18:14:54.4302830Z [submariner-operator]$ [cluster2] netshoot_pod=$(kubectl get pods -l app=netshoot | awk 'FNR == 2 {print $1}')
2021-07-16T18:14:54.4334024Z [submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot
2021-07-16T18:14:54.4363321Z [submariner-operator]$ [cluster2] awk FNR == 2 {print $1}
2021-07-16T18:14:54.4404795Z [submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot
2021-07-16T18:14:54.4424727Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods -l app=netshoot
2021-07-16T18:14:54.7891005Z [submariner-operator]$ [cluster2] nginx_svc_ip=$(with_context "$target_cluster" get_svc_ip nginx-demo)
2021-07-16T18:14:54.7912075Z [submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo
2021-07-16T18:14:54.7925831Z [submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo
2021-07-16T18:14:54.7940040Z [submariner-operator]$ [cluster1] get_svc_ip nginx-demo
2021-07-16T18:14:54.7954027Z [submariner-operator]$ [cluster1] get_svc_ip
2021-07-16T18:14:54.7967829Z [submariner-operator]$ [cluster1] svc_ip=$(kubectl --context="$cluster" get svc -l "app=${svc_name}" | awk 'FNR == 2 {print $3}')
2021-07-16T18:14:54.7987947Z [submariner-operator]$ [cluster1] kubectl --context=cluster1 get svc -l app=nginx-demo
2021-07-16T18:14:54.8006241Z [submariner-operator]$ [cluster1] awk FNR == 2 {print $3}
2021-07-16T18:14:54.8017041Z [submariner-operator]$ [cluster1] kubectl --context=cluster1 get svc -l app=nginx-demo
2021-07-16T18:14:54.8034058Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc -l app=nginx-demo
2021-07-16T18:14:55.1293865Z [submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-6c8477dd9f-qldwq 100.1.206.190
2021-07-16T18:14:55.1305253Z [submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-6c8477dd9f-qldwq 100.1.206.190
2021-07-16T18:14:55.1329790Z [submariner-operator]$ [cluster2] iteration=1
2021-07-16T18:14:55.1366499Z [submariner-operator]$ [cluster2] test_connection netshoot-6c8477dd9f-qldwq 100.1.206.190
2021-07-16T18:14:55.1372159Z [submariner-operator]$ [cluster2] wait 7609
2021-07-16T18:14:55.1384532Z [submariner-operator]$ [cluster2] test_connection
2021-07-16T18:14:55.1394329Z Attempting connectivity between clusters - netshoot-6c8477dd9f-qldwq --> 100.1.206.190
2021-07-16T18:14:55.1408688Z [submariner-operator]$ [cluster2] wait_time=30
2021-07-16T18:14:55.1420587Z [submariner-operator]$ [cluster2] start_time=$(date +%s)
2021-07-16T18:14:55.1448812Z [submariner-operator]$ [cluster2] date +%s
2021-07-16T18:14:55.1503129Z [submariner-operator]$ [cluster2] kubectl exec netshoot-6c8477dd9f-qldwq -- curl --output /dev/null -m 30 --silent --head --fail 100.1.206.190
2021-07-16T18:14:55.1516585Z [submariner-operator]$ [cluster2] kubectl exec netshoot-6c8477dd9f-qldwq -- curl --output /dev/null -m 30 --silent --head --fail 100.1.206.190
2021-07-16T18:14:55.1528385Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-6c8477dd9f-qldwq -- curl --output /dev/null -m 30 --silent --head --fail 100.1.206.190
2021-07-16T18:14:55.5881736Z Connection test was successful!
2021-07-16T18:14:55.5902193Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:14:55.5919380Z [submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:55.5972280Z [submariner-operator]$ [cluster2] remove_resource /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:55.5997032Z [submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:55.6014722Z [submariner-operator]$ [cluster2] kubectl delete -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:55.6035127Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 delete -f /opt/shipyard/scripts/resources/netshoot.yaml
2021-07-16T18:14:55.9037493Z deployment.apps "netshoot" deleted
2021-07-16T18:14:55.9158930Z [submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:55.9176848Z [submariner-operator]$ [cluster2] with_context cluster1 remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:55.9207259Z [submariner-operator]$ [cluster1] remove_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:55.9253994Z [submariner-operator]$ [cluster1] remove_resource
2021-07-16T18:14:55.9349004Z [submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:55.9380132Z [submariner-operator]$ [cluster1] kubectl delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:55.9438108Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 delete -f /opt/shipyard/scripts/resources/nginx-demo.yaml
2021-07-16T18:14:56.3309694Z deployment.apps "nginx-demo" deleted
2021-07-16T18:14:56.4210816Z service "nginx-demo" deleted
2021-07-16T18:14:56.4724807Z serviceexport.multicluster.x-k8s.io "nginx-demo" deleted
2021-07-16T18:14:56.5438896Z [submariner-operator]$ [cluster2] run_if_defined post_deploy
2021-07-16T18:14:56.5451717Z [submariner-operator]$ [cluster2] run_if_defined post_deploy
2021-07-16T18:14:56.5464418Z [submariner-operator]$ [cluster2] type -t post_deploy
2021-07-16T18:14:56.5482269Z [submariner-operator]$ [cluster2] grep -q function
2021-07-16T18:14:56.5535408Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:14:56.5555832Z   Successfully remade target file 'deploy'.
2021-07-16T18:14:56.5556607Z Must remake target 'e2e'.
2021-07-16T18:14:56.5557279Z scripts/kind-e2e/e2e.sh
2021-07-16T18:14:56.5638491Z [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
2021-07-16T18:14:56.5662373Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:14:56.5673208Z [submariner-operator]$ script_name=utils
2021-07-16T18:14:56.5688722Z [submariner-operator]$ exec_name=e2e.sh
2021-07-16T18:14:56.5748079Z [submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs
2021-07-16T18:14:56.5758037Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:14:56.5788359Z [submariner-operator]$ script_name=deploy_funcs
2021-07-16T18:14:56.5814014Z [submariner-operator]$ exec_name=e2e.sh
2021-07-16T18:14:56.5835331Z [submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_operator
2021-07-16T18:14:56.5870239Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:14:56.5893434Z [submariner-operator]$ script_name=deploy_operator
2021-07-16T18:14:56.5923131Z [submariner-operator]$ exec_name=e2e.sh
2021-07-16T18:14:56.5970287Z [submariner-operator]$ source /opt/shipyard/scripts/lib/cluster_settings
2021-07-16T18:14:56.6007050Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:14:56.6018079Z [submariner-operator]$ script_name=cluster_settings
2021-07-16T18:14:56.6045455Z [submariner-operator]$ exec_name=e2e.sh
2021-07-16T18:14:56.6078629Z [submariner-operator]$ broker=cluster1
2021-07-16T18:14:56.6093098Z [submariner-operator]$ declare -gA cluster_nodes
2021-07-16T18:14:56.6118032Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker
2021-07-16T18:14:56.6145136Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker
2021-07-16T18:14:56.6178551Z [submariner-operator]$ cluster_nodes[cluster3]=control-plane worker worker
2021-07-16T18:14:56.6206991Z [submariner-operator]$ declare -gA cluster_subm
2021-07-16T18:14:56.6243987Z [submariner-operator]$ cluster_subm[cluster1]=true
2021-07-16T18:14:56.6255372Z [submariner-operator]$ cluster_subm[cluster2]=true
2021-07-16T18:14:56.6266008Z [submariner-operator]$ cluster_subm[cluster3]=true
2021-07-16T18:14:56.6284259Z [submariner-operator]$ declare -gA cluster_cni
2021-07-16T18:14:56.6312815Z [submariner-operator]$ cluster_cni[cluster2]=weave
2021-07-16T18:14:56.6337570Z [submariner-operator]$ cluster_cni[cluster3]=weave
2021-07-16T18:14:56.6384665Z [submariner-operator]$ source /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/cluster_settings
2021-07-16T18:14:56.6416123Z [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
2021-07-16T18:14:56.6427184Z [submariner-operator]$ script_name=cluster_settings
2021-07-16T18:14:56.6437887Z [submariner-operator]$ exec_name=e2e.sh
2021-07-16T18:14:56.6475013Z [submariner-operator]$ cluster_nodes[cluster1]=control-plane worker worker
2021-07-16T18:14:56.6532784Z [submariner-operator]$ cluster_nodes[cluster2]=control-plane worker worker
2021-07-16T18:14:56.6606255Z [submariner-operator]$ timeout=2m
2021-07-16T18:14:56.6616418Z [submariner-operator]$ declare_kubeconfig
2021-07-16T18:14:56.6627592Z [submariner-operator]$ declare_kubeconfig
2021-07-16T18:14:56.6638944Z [submariner-operator]$ source /opt/shipyard/scripts/lib/kubecfg
2021-07-16T18:14:56.6653971Z [submariner-operator]$ export KUBECONFIG
2021-07-16T18:14:56.6667556Z [submariner-operator]$ KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)
2021-07-16T18:14:56.6685383Z [submariner-operator]$ find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:
2021-07-16T18:14:56.6785391Z [submariner-operator]$ . /go/src/github.com/submariner-io/submariner-operator/scripts/kind-e2e/lib_operator_verify_subm.sh
2021-07-16T18:14:56.6842595Z [submariner-operator]$ create_subm_vars
2021-07-16T18:14:56.6879631Z [submariner-operator]$ create_subm_vars
2021-07-16T18:14:56.6898601Z [submariner-operator]$ deployment_name=submariner
2021-07-16T18:14:56.6914615Z [submariner-operator]$ operator_deployment_name=submariner-operator
2021-07-16T18:14:56.6929367Z [submariner-operator]$ gateway_deployment_name=submariner-gateway
2021-07-16T18:14:56.6939576Z [submariner-operator]$ routeagent_deployment_name=submariner-routeagent
2021-07-16T18:14:56.6950886Z [submariner-operator]$ broker_deployment_name=submariner-k8s-broker
2021-07-16T18:14:56.6960838Z [submariner-operator]$ globalnet_deployment_name=submariner-globalnet
2021-07-16T18:14:56.6979519Z [submariner-operator]$ declare_cidrs
2021-07-16T18:14:56.6995117Z [submariner-operator]$ declare_cidrs
2021-07-16T18:14:56.7005874Z [submariner-operator]$ declare -gA cluster_CIDRs service_CIDRs global_CIDRs
2021-07-16T18:14:56.7016911Z [submariner-operator]$ i=1
2021-07-16T18:14:56.7075235Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-16T18:14:56.7120395Z [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
2021-07-16T18:14:56.7154298Z [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16
2021-07-16T18:14:56.7165589Z [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16
2021-07-16T18:14:56.7177304Z [submariner-operator]$ [cluster1] i=$(("$i"+1))
2021-07-16T18:14:56.7189789Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-16T18:14:56.7200036Z [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
2021-07-16T18:14:56.7228713Z [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16
2021-07-16T18:14:56.7252970Z [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16
2021-07-16T18:14:56.7268343Z [submariner-operator]$ [cluster2] i=$(("$i"+1))
2021-07-16T18:14:56.7290358Z [submariner-operator]$ [cluster2] natEnabled=false
2021-07-16T18:14:56.7305872Z [submariner-operator]$ [cluster2] subm_gateway_image_repo=localhost:5000
2021-07-16T18:14:56.7317660Z [submariner-operator]$ [cluster2] subm_gateway_image_tag=local
2021-07-16T18:14:56.7331991Z [submariner-operator]$ [cluster2] subm_gateway_size=3
2021-07-16T18:14:56.7345608Z [submariner-operator]$ [cluster2] subm_colorcodes=blue
2021-07-16T18:14:56.7356749Z [submariner-operator]$ [cluster2] subm_debug=false
2021-07-16T18:14:56.7386096Z [submariner-operator]$ [cluster2] subm_broker=k8s
2021-07-16T18:14:56.7400996Z [submariner-operator]$ [cluster2] ce_ipsec_debug=false
2021-07-16T18:14:56.7412900Z [submariner-operator]$ [cluster2] ce_ipsec_ikeport=500
2021-07-16T18:14:56.7423247Z [submariner-operator]$ [cluster2] ce_ipsec_nattport=4500
2021-07-16T18:14:56.7449579Z [submariner-operator]$ [cluster2] subm_ns=submariner-operator
2021-07-16T18:14:56.7465226Z [submariner-operator]$ [cluster2] SUBMARINER_BROKER_NS=submariner-k8s-broker
2021-07-16T18:14:56.7492652Z [submariner-operator]$ [cluster2] with_context cluster1 broker_vars
2021-07-16T18:14:56.7511367Z [submariner-operator]$ [cluster2] with_context cluster1 broker_vars
2021-07-16T18:14:56.7523422Z [submariner-operator]$ [cluster1] broker_vars
2021-07-16T18:14:56.7539961Z [submariner-operator]$ [cluster1] broker_vars
2021-07-16T18:14:56.7560977Z [submariner-operator]$ [cluster1] SUBMARINER_BROKER_URL=$(kubectl -n default get endpoints kubernetes -o jsonpath="{.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}")
2021-07-16T18:14:56.7600760Z [submariner-operator]$ [cluster1] kubectl -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}
2021-07-16T18:14:56.7614348Z [submariner-operator]$ [cluster1] kubectl -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}
2021-07-16T18:14:56.7649463Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n default get endpoints kubernetes -o jsonpath={.subsets[0].addresses[0].ip}:{.subsets[0].ports[?(@.name=='https')].port}
2021-07-16T18:14:57.2340764Z [submariner-operator]$ [cluster1] SUBMARINER_BROKER_CA=$(kubectl -n ${SUBMARINER_BROKER_NS} get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${SUBMARINER_BROKER_NS}-client')].data['ca\.crt']}")
2021-07-16T18:14:57.2382451Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}
2021-07-16T18:14:57.2396608Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}
2021-07-16T18:14:57.2405848Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data['ca\.crt']}
2021-07-16T18:14:57.5794384Z [submariner-operator]$ [cluster1] SUBMARINER_BROKER_TOKEN=$(kubectl -n ${SUBMARINER_BROKER_NS} get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='${SUBMARINER_BROKER_NS}-client')].data.token}"|base64 --decode)
2021-07-16T18:14:57.5817957Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}
2021-07-16T18:14:57.5836650Z [submariner-operator]$ [cluster1] base64 --decode
2021-07-16T18:14:57.5849667Z [submariner-operator]$ [cluster1] kubectl -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}
2021-07-16T18:14:57.5869566Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 -n submariner-k8s-broker get secrets -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].data.token}
2021-07-16T18:14:57.9070489Z [submariner-operator]$ [cluster2] with_context cluster1 verify_subm_broker_secrets
2021-07-16T18:14:57.9087398Z [submariner-operator]$ [cluster2] with_context cluster1 verify_subm_broker_secrets
2021-07-16T18:14:57.9105390Z [submariner-operator]$ [cluster1] verify_subm_broker_secrets
2021-07-16T18:14:57.9121755Z [submariner-operator]$ [cluster1] verify_subm_broker_secrets
2021-07-16T18:14:57.9921651Z [submariner-operator]$ [cluster1] verify_secrets submariner-k8s-broker submariner-k8s-broker-client LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
2021-07-16T18:14:58.1063252Z [submariner-operator]$ [cluster1] verify_secrets submariner-k8s-broker submariner-k8s-broker-client LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
2021-07-16T18:14:58.1611495Z [submariner-operator]$ [cluster1] wait_for_secrets submariner-k8s-broker
2021-07-16T18:14:58.1612986Z [submariner-operator]$ [cluster1] wait_for_secrets submariner-k8s-broker
2021-07-16T18:14:58.1614873Z [submariner-operator]$ [cluster1] kubectl get secret -n submariner-k8s-broker
2021-07-16T18:14:58.1616198Z [submariner-operator]$ [cluster1] grep -q submariner-
2021-07-16T18:14:58.1618061Z [submariner-operator]$ [cluster1] kubectl get secret -n submariner-k8s-broker
2021-07-16T18:14:58.1619733Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secret -n submariner-k8s-broker
2021-07-16T18:14:58.3284026Z [submariner-operator]$ [cluster1] return
2021-07-16T18:14:58.3297252Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker
2021-07-16T18:14:58.3311288Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker
2021-07-16T18:14:58.3325982Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secrets -n submariner-k8s-broker
2021-07-16T18:14:58.6265488Z NAME                                       TYPE                                  DATA   AGE
2021-07-16T18:14:58.6267385Z cluster-cluster1-token-g98zw               kubernetes.io/service-account-token   3      66s
2021-07-16T18:14:58.6269072Z cluster-cluster2-token-6mcch               kubernetes.io/service-account-token   3      61s
2021-07-16T18:14:58.6270775Z default-token-24wff                        kubernetes.io/service-account-token   3      85s
2021-07-16T18:14:58.6272504Z submariner-k8s-broker-admin-token-7lhdr    kubernetes.io/service-account-token   3      85s
2021-07-16T18:14:58.6274701Z submariner-k8s-broker-client-token-qg249   kubernetes.io/service-account-token   3      85s
2021-07-16T18:14:58.6330612Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}
2021-07-16T18:14:58.6345432Z [submariner-operator]$ [cluster1] kubectl get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}
2021-07-16T18:14:58.6362189Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secrets -n submariner-k8s-broker -o jsonpath={.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='submariner-k8s-broker-client')].metadata.name}
2021-07-16T18:14:58.9377802Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner-k8s-broker-client-token-qg249.cluster1.json
2021-07-16T18:14:58.9463798Z [submariner-operator]$ [cluster1] kubectl get secret submariner-k8s-broker-client-token-qg249 -n submariner-k8s-broker -o json
2021-07-16T18:14:58.9497867Z [submariner-operator]$ [cluster1] tee /tmp/submariner-k8s-broker-client-token-qg249.cluster1.json
2021-07-16T18:14:58.9540338Z [submariner-operator]$ [cluster1] kubectl get secret submariner-k8s-broker-client-token-qg249 -n submariner-k8s-broker -o json
2021-07-16T18:14:58.9567138Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get secret submariner-k8s-broker-client-token-qg249 -n submariner-k8s-broker -o json
2021-07-16T18:14:59.2611204Z {
2021-07-16T18:14:59.2611905Z     "apiVersion": "v1",
2021-07-16T18:14:59.2612426Z     "data": {
2021-07-16T18:14:59.3193161Z         "ca.crt": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=",
2021-07-16T18:14:59.3713824Z [submariner-operator]$ [cluster1] validate_equals .kind Secret
2021-07-16T18:14:59.3715077Z [submariner-operator]$ [cluster1] validate_equals .kind Secret
2021-07-16T18:14:59.3717065Z [submariner-operator]$ [cluster1] jq -r -M .kind /tmp/submariner-k8s-broker-client-token-qg249.cluster1.json
2021-07-16T18:14:59.3719282Z [submariner-operator]$ [cluster1] validate_equals .type kubernetes.io/service-account-token
2021-07-16T18:14:59.3721950Z [submariner-operator]$ [cluster1] validate_equals .type kubernetes.io/service-account-token
2021-07-16T18:14:59.3724359Z [submariner-operator]$ [cluster1] jq -r -M .type /tmp/submariner-k8s-broker-client-token-qg249.cluster1.json
2021-07-16T18:14:59.3726128Z         "namespace": "c3VibWFyaW5lci1rOHMtYnJva2Vy",
2021-07-16T18:14:59.3728388Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner-k8s-broker-client-token-qg249
2021-07-16T18:14:59.4349455Z         "token": "ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNkluZzFjRWRNWTBNeloxaFViRWRETXpsM01GZFZkR2RTUTFrdFlVVndaWFI0TkdSS00zUmxjek5aWkZFaWZRLmV5SnBjM01pT2lKcmRXSmxjbTVsZEdWekwzTmxjblpwWTJWaFkyTnZkVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXVZVzFsYzNCaFkyVWlPaUp6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJaUxDSnJkV0psY201bGRHVnpMbWx2TDNObGNuWnBZMlZoWTJOdmRXNTBMM05sWTNKbGRDNXVZVzFsSWpvaWMzVmliV0Z5YVc1bGNpMXJPSE10WW5KdmEyVnlMV05zYVdWdWRDMTBiMnRsYmkxeFp6STBPU0lzSW10MVltVnlibVYwWlhNdWFXOHZjMlZ5ZG1salpXRmpZMjkxYm5RdmMyVnlkbWxqWlMxaFkyTnZkVzUwTG01aGJXVWlPaUp6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJdFkyeHBaVzUwSWl3aWEzVmlaWEp1WlhSbGN5NXBieTl6WlhKMmFXTmxZV05qYjNWdWRDOXpaWEoyYVdObExXRmpZMjkxYm5RdWRXbGtJam9pT0dKaU1HTmlORGt0TURBeE1TMDBPRGxpTFRnNE1qSXRaREl6TkRReVpHUXpZbU5oSWl3aWMzVmlJam9pYzNsemRHVnRPbk5sY25acFkyVmhZMk52ZFc1ME9uTjFZbTFoY21sdVpYSXRhemh6TFdKeWIydGxjanB6ZFdKdFlYSnBibVZ5TFdzNGN5MWljbTlyWlhJdFkyeHBaVzUwSW4wLmhlRkZSUkZycjhaNU9tWkpwZmVwMGVraXVOTFpOaWxrVkZDenhyRFVMUC1OdGxwMlJ0UkR0VEZ3NVowNkdjRGV2aFd3ZUJUcHl2WDVtZk9memZVSTh5MEp5VjhBS3RuYVp4cjJCYXZ6X3pXZGxyRjVYYUQ2ZldrVVhxZkRtVHE4by1EanZ3aVNXN1ZQdHBscFd5QmZvekstMHdzRElwSGJsTmZhTmZmZEFuV0JUYVhxQVV4c2RTMzFxeEtPZVIyam41ZDEwMVNKOUVlVEItTUY2ZU52ZjBsR295bk40ZS1uV3FWek9CYlJnZEpyUVRuNlRhMmFKcm9oRkJNRnRPSUJxbHZ3eUlWMm5Ia09STXBVUkdQWlFlS1ZaNUxGRHZKVVFYeVJzTXBmQ2NOT3J3cE1MQ2NZaXJvUkk4VkU5Zk9aYlE0WWVISFI5dC1xSTJwczViSFd3dw=="
2021-07-16T18:14:59.4916236Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner-k8s-broker-client-token-qg249
2021-07-16T18:14:59.4918885Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariner-k8s-broker-client-token-qg249.cluster1.json
2021-07-16T18:14:59.4921189Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-k8s-broker
2021-07-16T18:14:59.4922911Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-k8s-broker
2021-07-16T18:14:59.4925429Z [submariner-operator]$ [cluster1] jq -r -M .metadata.namespace /tmp/submariner-k8s-broker-client-token-qg249.cluster1.json
2021-07-16T18:14:59.4926987Z     },
2021-07-16T18:14:59.4927552Z     "kind": "Secret",
2021-07-16T18:14:59.4928129Z     "metadata": {
2021-07-16T18:14:59.4928601Z         "annotations": {
2021-07-16T18:14:59.4929790Z             "kubernetes.io/service-account.name": "submariner-k8s-broker-client",
2021-07-16T18:14:59.4931538Z             "kubernetes.io/service-account.uid": "8bb0cb49-0011-489b-8822-d23442dd3bca"
2021-07-16T18:14:59.4932416Z         },
2021-07-16T18:14:59.4933168Z         "creationTimestamp": "2021-07-16T18:13:33Z",
2021-07-16T18:14:59.4933823Z         "managedFields": [
2021-07-16T18:14:59.4934284Z             {
2021-07-16T18:14:59.4934862Z                 "apiVersion": "v1",
2021-07-16T18:14:59.4935427Z                 "fieldsType": "FieldsV1",
2021-07-16T18:14:59.4935956Z                 "fieldsV1": {
2021-07-16T18:14:59.4936365Z                     "f:data": {
2021-07-16T18:14:59.4936788Z                         ".": {},
2021-07-16T18:14:59.4937592Z                         "f:ca.crt": {},
2021-07-16T18:14:59.4938209Z                         "f:namespace": {},
2021-07-16T18:14:59.4938688Z                         "f:token": {}
2021-07-16T18:14:59.4956184Z                     },
2021-07-16T18:14:59.4956726Z                     "f:metadata": {
2021-07-16T18:14:59.4957322Z                         "f:annotations": {
2021-07-16T18:14:59.4957905Z                             ".": {},
2021-07-16T18:14:59.4958907Z                             "f:kubernetes.io/service-account.name": {},
2021-07-16T18:14:59.4960191Z                             "f:kubernetes.io/service-account.uid": {}
2021-07-16T18:14:59.4960944Z                         }
2021-07-16T18:14:59.4961397Z                     },
2021-07-16T18:14:59.4961814Z                     "f:type": {}
2021-07-16T18:14:59.4962201Z                 },
2021-07-16T18:14:59.4963004Z                 "manager": "kube-controller-manager",
2021-07-16T18:14:59.4964627Z                 "operation": "Update",
2021-07-16T18:14:59.4965421Z                 "time": "2021-07-16T18:13:33Z"
2021-07-16T18:14:59.4965933Z             }
2021-07-16T18:14:59.4966358Z         ],
2021-07-16T18:14:59.4967434Z         "name": "submariner-k8s-broker-client-token-qg249",
2021-07-16T18:14:59.5007262Z         "namespace": "submariner-k8s-broker",
2021-07-16T18:14:59.5008226Z         "resourceVersion": "1640",
2021-07-16T18:14:59.5009913Z         "selfLink": "/api/v1/namespaces/submariner-k8s-broker/secrets/submariner-k8s-broker-client-token-qg249",
2021-07-16T18:14:59.5011593Z         "uid": "e32bf4e8-9478-45b0-94bf-03333e7674e7"
2021-07-16T18:14:59.5012236Z     },
2021-07-16T18:14:59.5013132Z     "type": "kubernetes.io/service-account-token"
2021-07-16T18:14:59.5013836Z }
2021-07-16T18:14:59.5202131Z [submariner-operator]$ [cluster1] jq -r -M .data["ca.crt"] /tmp/submariner-k8s-broker-client-token-qg249.cluster1.json
2021-07-16T18:14:59.5635430Z [submariner-operator]$ [cluster1] one_ca_crt_ok=true
2021-07-16T18:14:59.5650043Z [submariner-operator]$ [cluster2] run_subm_clusters verify_subm_deployed
2021-07-16T18:14:59.5663335Z [submariner-operator]$ [cluster2] run_subm_clusters verify_subm_deployed
2021-07-16T18:14:59.5680707Z [submariner-operator]$ [cluster2] declare -a subm_clusters
2021-07-16T18:14:59.5717313Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 verify_subm_deployed
2021-07-16T18:14:59.5741842Z [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 verify_subm_deployed
2021-07-16T18:14:59.5762278Z [submariner-operator]$ [cluster2] declare -A pids
2021-07-16T18:14:59.5789692Z [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
2021-07-16T18:14:59.5819966Z [submariner-operator]$ [cluster1] pids[cluster1]=7857
2021-07-16T18:14:59.5827201Z [submariner-operator]$ [cluster1] set -o pipefail
2021-07-16T18:14:59.5839780Z [submariner-operator]$ [cluster1] verify_subm_deployed
2021-07-16T18:14:59.5856212Z [submariner-operator]$ [cluster2] set -o pipefail
2021-07-16T18:14:59.5858413Z [submariner-operator]$ [cluster2] pids[cluster2]=7861
2021-07-16T18:14:59.5879889Z [submariner-operator]$ [cluster2] verify_subm_deployed
2021-07-16T18:14:59.5881208Z [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
2021-07-16T18:14:59.5900352Z [submariner-operator]$ [cluster2] wait 7861
2021-07-16T18:14:59.5913013Z [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
2021-07-16T18:15:00.4237909Z [submariner-operator]$ [cluster2] verify_subm_deployed
2021-07-16T18:15:00.4239082Z [submariner-operator]$ [cluster2] verify_endpoints_crd
2021-07-16T18:15:00.4240202Z [submariner-operator]$ [cluster2] verify_endpoints_crd
2021-07-16T18:15:00.4241569Z [submariner-operator]$ [cluster2] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-16T18:15:00.4243158Z [submariner-operator]$ [cluster2] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-16T18:15:00.4245328Z [submariner-operator]$ [cluster2] kubectl get crds endpoints.submariner.io
2021-07-16T18:15:00.4246811Z [submariner-operator]$ [cluster2] kubectl get crds endpoints.submariner.io
2021-07-16T18:15:00.4248451Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds endpoints.submariner.io
2021-07-16T18:15:00.4249527Z [cluster2] NAME                      CREATED AT
2021-07-16T18:15:00.4250445Z [cluster2] endpoints.submariner.io   2021-07-16T18:13:58Z
2021-07-16T18:15:00.4251979Z [submariner-operator]$ [cluster2] json_file=/tmp/endpoints.submariner.io.cluster2.json
2021-07-16T18:15:00.4253705Z [submariner-operator]$ [cluster2] kubectl get crd endpoints.submariner.io -o json
2021-07-16T18:15:00.4255419Z [submariner-operator]$ [cluster2] tee /tmp/endpoints.submariner.io.cluster2.json
2021-07-16T18:15:00.4257348Z [submariner-operator]$ [cluster2] kubectl get crd endpoints.submariner.io -o json
2021-07-16T18:15:00.4259033Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd endpoints.submariner.io -o json
2021-07-16T18:15:00.4260029Z [cluster2] {
2021-07-16T18:15:00.4260679Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:00.4261590Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-16T18:15:00.4262292Z [cluster2]     "metadata": {
2021-07-16T18:15:00.4262827Z [cluster2]         "annotations": {
2021-07-16T18:15:00.4263875Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-16T18:15:00.4264659Z [cluster2]         },
2021-07-16T18:15:00.4265435Z [cluster2]         "creationTimestamp": "2021-07-16T18:13:58Z",
2021-07-16T18:15:00.4266096Z [cluster2]         "generation": 1,
2021-07-16T18:15:00.4266685Z [cluster2]         "managedFields": [
2021-07-16T18:15:00.4267192Z [cluster2]             {
2021-07-16T18:15:00.4267999Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:00.4268772Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:00.4269367Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:00.4269892Z [cluster2]                     "f:metadata": {
2021-07-16T18:15:00.4270465Z [cluster2]                         "f:annotations": {
2021-07-16T18:15:00.4270991Z [cluster2]                             ".": {},
2021-07-16T18:15:00.4272025Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-16T18:15:00.4272820Z [cluster2]                         }
2021-07-16T18:15:00.4273273Z [cluster2]                     },
2021-07-16T18:15:00.4273750Z [cluster2]                     "f:spec": {
2021-07-16T18:15:00.4274282Z [cluster2]                         "f:conversion": {
2021-07-16T18:15:00.4274813Z [cluster2]                             ".": {},
2021-07-16T18:15:00.4275316Z [cluster2]                             "f:strategy": {}
2021-07-16T18:15:00.4275823Z [cluster2]                         },
2021-07-16T18:15:00.4276306Z [cluster2]                         "f:group": {},
2021-07-16T18:15:00.4276823Z [cluster2]                         "f:names": {
2021-07-16T18:15:00.4277320Z [cluster2]                             "f:kind": {},
2021-07-16T18:15:00.4277864Z [cluster2]                             "f:listKind": {},
2021-07-16T18:15:00.4278407Z [cluster2]                             "f:plural": {},
2021-07-16T18:15:00.4279083Z [cluster2]                             "f:singular": {}
2021-07-16T18:15:00.4279593Z [cluster2]                         },
2021-07-16T18:15:00.4280077Z [cluster2]                         "f:scope": {},
2021-07-16T18:15:00.4280620Z [cluster2]                         "f:versions": {}
2021-07-16T18:15:00.4281110Z [cluster2]                     },
2021-07-16T18:15:00.4281594Z [cluster2]                     "f:status": {
2021-07-16T18:15:00.4282172Z [cluster2]                         "f:acceptedNames": {
2021-07-16T18:15:00.4282759Z [cluster2]                             "f:kind": {},
2021-07-16T18:15:00.4283275Z [cluster2]                             "f:plural": {}
2021-07-16T18:15:00.4284209Z [cluster2]                         },
2021-07-16T18:15:00.4284733Z [cluster2]                         "f:conditions": {},
2021-07-16T18:15:00.4285349Z [cluster2]                         "f:storedVersions": {}
2021-07-16T18:15:00.4285906Z [cluster2]                     }
2021-07-16T18:15:00.4286339Z [cluster2]                 },
2021-07-16T18:15:00.4287231Z [cluster2]                 "manager": "submariner-operator",
2021-07-16T18:15:00.4287939Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:00.4288695Z [cluster2]                 "time": "2021-07-16T18:14:08Z"
2021-07-16T18:15:00.4289167Z [cluster2]             }
2021-07-16T18:15:00.4289596Z [cluster2]         ],
2021-07-16T18:15:00.4290238Z [cluster2]         "name": "endpoints.submariner.io",
2021-07-16T18:15:00.4291002Z [cluster2]         "resourceVersion": "1859",
2021-07-16T18:15:00.4292378Z [cluster2]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/endpoints.submariner.io",
2021-07-16T18:15:00.4293852Z [cluster2]         "uid": "e3918938-47b8-48ad-8b00-97ade2d70b76"
2021-07-16T18:15:00.4294451Z [cluster2]     },
2021-07-16T18:15:00.4294874Z [cluster2]     "spec": {
2021-07-16T18:15:00.4295380Z [cluster2]         "conversion": {
2021-07-16T18:15:00.4295923Z [cluster2]             "strategy": "None"
2021-07-16T18:15:00.4296418Z [cluster2]         },
2021-07-16T18:15:00.4296935Z [cluster2]         "group": "submariner.io",
2021-07-16T18:15:00.4297490Z [cluster2]         "names": {
2021-07-16T18:15:00.4297992Z [cluster2]             "kind": "Endpoint",
2021-07-16T18:15:00.4334746Z [cluster2]             "listKind": "EndpointList",
2021-07-16T18:15:00.4335432Z [cluster2]             "plural": "endpoints",
2021-07-16T18:15:00.4336033Z [cluster2]             "singular": "endpoint"
2021-07-16T18:15:00.4336553Z [cluster2]         },
2021-07-16T18:15:00.4337069Z [cluster2]         "scope": "Namespaced",
2021-07-16T18:15:00.4337623Z [cluster2]         "versions": [
2021-07-16T18:15:00.4338081Z [cluster2]             {
2021-07-16T18:15:00.4338543Z [cluster2]                 "name": "v1",
2021-07-16T18:15:00.4339041Z [cluster2]                 "schema": {
2021-07-16T18:15:00.4339623Z [cluster2]                     "openAPIV3Schema": {
2021-07-16T18:15:00.4340265Z [cluster2]                         "properties": {
2021-07-16T18:15:00.4340852Z [cluster2]                             "apiVersion": {
2021-07-16T18:15:00.4343616Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-16T18:15:00.4345609Z [cluster2]                                 "type": "string"
2021-07-16T18:15:00.4346120Z [cluster2]                             },
2021-07-16T18:15:00.4346611Z [cluster2]                             "kind": {
2021-07-16T18:15:00.4349138Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-16T18:15:00.4351113Z [cluster2]                                 "type": "string"
2021-07-16T18:15:00.4351633Z [cluster2]                             },
2021-07-16T18:15:00.4352132Z [cluster2]                             "metadata": {
2021-07-16T18:15:00.4352699Z [cluster2]                                 "type": "object"
2021-07-16T18:15:00.4353196Z [cluster2]                             },
2021-07-16T18:15:00.4353684Z [cluster2]                             "spec": {
2021-07-16T18:15:00.4354238Z [cluster2]                                 "properties": {
2021-07-16T18:15:00.4354821Z [cluster2]                                     "backend": {
2021-07-16T18:15:00.4355392Z [cluster2]                                         "type": "string"
2021-07-16T18:15:00.4356068Z [cluster2]                                     },
2021-07-16T18:15:00.4356614Z [cluster2]                                     "backend_config": {
2021-07-16T18:15:00.4357336Z [cluster2]                                         "additionalProperties": {
2021-07-16T18:15:00.4358065Z [cluster2]                                             "type": "string"
2021-07-16T18:15:00.4358588Z [cluster2]                                         },
2021-07-16T18:15:00.4359136Z [cluster2]                                         "type": "object"
2021-07-16T18:15:00.4359664Z [cluster2]                                     },
2021-07-16T18:15:00.4360172Z [cluster2]                                     "cable_name": {
2021-07-16T18:15:00.4360839Z [cluster2]                                         "type": "string"
2021-07-16T18:15:00.4361357Z [cluster2]                                     },
2021-07-16T18:15:00.4361884Z [cluster2]                                     "cluster_id": {
2021-07-16T18:15:00.4362451Z [cluster2]                                         "type": "string"
2021-07-16T18:15:00.4362975Z [cluster2]                                     },
2021-07-16T18:15:00.4363727Z [cluster2]                                     "healthCheckIP": {
2021-07-16T18:15:00.4364359Z [cluster2]                                         "type": "string"
2021-07-16T18:15:00.4364885Z [cluster2]                                     },
2021-07-16T18:15:00.4365396Z [cluster2]                                     "hostname": {
2021-07-16T18:15:00.4365972Z [cluster2]                                         "type": "string"
2021-07-16T18:15:00.4366480Z [cluster2]                                     },
2021-07-16T18:15:00.4367011Z [cluster2]                                     "nat_enabled": {
2021-07-16T18:15:00.4367590Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:00.4368122Z [cluster2]                                     },
2021-07-16T18:15:00.4368653Z [cluster2]                                     "private_ip": {
2021-07-16T18:15:00.4369225Z [cluster2]                                         "type": "string"
2021-07-16T18:15:00.4369747Z [cluster2]                                     },
2021-07-16T18:15:00.4370262Z [cluster2]                                     "public_ip": {
2021-07-16T18:15:00.4370835Z [cluster2]                                         "type": "string"
2021-07-16T18:15:00.4371344Z [cluster2]                                     },
2021-07-16T18:15:00.4371858Z [cluster2]                                     "subnets": {
2021-07-16T18:15:00.4372393Z [cluster2]                                         "items": {
2021-07-16T18:15:00.4372952Z [cluster2]                                             "type": "string"
2021-07-16T18:15:00.4373486Z [cluster2]                                         },
2021-07-16T18:15:00.4374009Z [cluster2]                                         "type": "array"
2021-07-16T18:15:00.4374525Z [cluster2]                                     }
2021-07-16T18:15:00.4374978Z [cluster2]                                 },
2021-07-16T18:15:00.4375491Z [cluster2]                                 "required": [
2021-07-16T18:15:00.4376035Z [cluster2]                                     "backend",
2021-07-16T18:15:00.5170716Z [cluster2]                                   [submariner-operator]$ [cluster1] verify_subm_deployed
2021-07-16T18:15:00.5171989Z [submariner-operator]$ [cluster1] verify_endpoints_crd
2021-07-16T18:15:00.5173115Z [submariner-operator]$ [cluster1] verify_endpoints_crd
2021-07-16T18:15:00.5174457Z [submariner-operator]$ [cluster1] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-16T18:15:00.5176029Z [submariner-operator]$ [cluster1] validate_crd endpoints.submariner.io v1 Endpoint
2021-07-16T18:15:00.5177544Z [submariner-operator]$ [cluster1] kubectl get crds endpoints.submariner.io
2021-07-16T18:15:00.5179031Z [submariner-operator]$ [cluster1] kubectl get crds endpoints.submariner.io
2021-07-16T18:15:00.5180666Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds endpoints.submariner.io
2021-07-16T18:15:00.5181709Z [cluster1] NAME                      CREATED AT
2021-07-16T18:15:00.5182627Z [cluster1] endpoints.submariner.io   2021-07-16T18:13:43Z
2021-07-16T18:15:00.5184142Z [submariner-operator]$ [cluster1] json_file=/tmp/endpoints.submariner.io.cluster1.json
2021-07-16T18:15:00.5185885Z [submariner-operator]$ [cluster1] kubectl get crd endpoints.submariner.io -o json
2021-07-16T18:15:00.5187598Z [submariner-operator]$ [cluster1] tee /tmp/endpoints.submariner.io.cluster1.json
2021-07-16T18:15:00.5189286Z [submariner-operator]$ [cluster1] kubectl get crd endpoints.submariner.io -o json
2021-07-16T18:15:00.5191079Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd endpoints.submariner.io -o json
2021-07-16T18:15:00.5192055Z [cluster1] {
2021-07-16T18:15:00.5192719Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:00.5193614Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-16T18:15:00.5194323Z [cluster1]     "metadata": {
2021-07-16T18:15:00.5194846Z [cluster1]         "annotations": {
2021-07-16T18:15:00.5195897Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-16T18:15:00.5196687Z [cluster1]         },
2021-07-16T18:15:00.5197456Z [cluster1]         "creationTimestamp": "2021-07-16T18:13:43Z",
2021-07-16T18:15:00.5198118Z [cluster1]         "generation": 1,
2021-07-16T18:15:00.5198689Z [cluster1]         "managedFields": [
2021-07-16T18:15:00.5199214Z [cluster1]             {
2021-07-16T18:15:00.5199875Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:00.5200662Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:00.5201242Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:00.5201777Z [cluster1]                     "f:metadata": {
2021-07-16T18:15:00.5202342Z [cluster1]                         "f:annotations": {
2021-07-16T18:15:00.5202878Z [cluster1]                             ".": {},
2021-07-16T18:15:00.5204205Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-16T18:15:00.5205002Z [cluster1]                         }
2021-07-16T18:15:00.5205458Z [cluster1]                     },
2021-07-16T18:15:00.5205923Z [cluster1]                     "f:spec": {
2021-07-16T18:15:00.5206473Z [cluster1]                         "f:conversion": {
2021-07-16T18:15:00.5206992Z [cluster1]                             ".": {},
2021-07-16T18:15:00.5207512Z [cluster1]                             "f:strategy": {}
2021-07-16T18:15:00.5208007Z [cluster1]                         },
2021-07-16T18:15:00.5208509Z [cluster1]                         "f:group": {},
2021-07-16T18:15:00.5209013Z [cluster1]                         "f:names": {
2021-07-16T18:15:00.5209531Z [cluster1]                             "f:kind": {},
2021-07-16T18:15:00.5210074Z [cluster1]                             "f:listKind": {},
2021-07-16T18:15:00.5210609Z [cluster1]                             "f:plural": {},
2021-07-16T18:15:00.5211159Z [cluster1]                             "f:singular": {}
2021-07-16T18:15:00.5211748Z [cluster1]                         },
2021-07-16T18:15:00.5212251Z [cluster1]                         "f:scope": {},
2021-07-16T18:15:00.5212776Z [cluster1]                         "f:versions": {}
2021-07-16T18:15:00.5213281Z [cluster1]                     },
2021-07-16T18:15:00.5213754Z [cluster1]                     "f:status": {
2021-07-16T18:15:00.5214383Z [cluster1]                         "f:acceptedNames": {
2021-07-16T18:15:00.5214971Z [cluster1]                             "f:kind": {},
2021-07-16T18:15:00.5215490Z [cluster1]                             "f:plural": {}
2021-07-16T18:15:00.5215986Z [cluster1]                         },
2021-07-16T18:15:00.5216496Z [cluster1]                         "f:conditions": {},
2021-07-16T18:15:00.5217260Z [cluster1]                         "f:storedVersions": {}
2021-07-16T18:15:00.5217811Z [cluster1]                     }
2021-07-16T18:15:00.5218257Z [cluster1]                 },
2021-07-16T18:15:00.5219144Z [cluster1]                 "manager": "submariner-operator",
2021-07-16T18:15:00.5219865Z [cluster1]                 "operation": "Update",
2021-07-16T18:15:00.5220597Z [cluster1]                 "time": "2021-07-16T18:13:48Z"
2021-07-16T18:15:00.5221084Z [cluster1]             }
2021-07-16T18:15:00.5221516Z [cluster1]         ],
2021-07-16T18:15:00.5222156Z [cluster1]         "name": "endpoints.submariner.io",
2021-07-16T18:15:00.5222916Z [cluster1]         "resourceVersion": "1830",
2021-07-16T18:15:00.5224122Z [cluster1]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/endpoints.submariner.io",
2021-07-16T18:15:00.5225843Z [cluster1]         "uid": "cd54cdc8-baed-4f92-af2c-b83fb7e08921"
2021-07-16T18:15:00.5226567Z [cluster1]     },
2021-07-16T18:15:00.5227007Z [cluster1]     "spec": {
2021-07-16T18:15:00.5227503Z [cluster1]         "conversion": {
2021-07-16T18:15:00.5228059Z [cluster1]             "strategy": "None"
2021-07-16T18:15:00.5228532Z [cluster1]         },
2021-07-16T18:15:00.5229076Z [cluster1]         "group": "submariner.io",
2021-07-16T18:15:00.5229633Z [cluster1]         "names": {
2021-07-16T18:15:00.5230257Z [cluster1]             "kind": "Endpoint",
2021-07-16T18:15:00.5230876Z [cluster1]             "listKind": "EndpointList",
2021-07-16T18:15:00.5231501Z [cluster1]             "plural": "endpoints",
2021-07-16T18:15:00.5232109Z [cluster1]             "singular": "endpoint"
2021-07-16T18:15:00.5232610Z [cluster1]         },
2021-07-16T18:15:00.5233122Z [cluster1]         "scope": "Namespaced",
2021-07-16T18:15:00.5233671Z [cluster1]         "versions": [
2021-07-16T18:15:00.5234145Z [cluster1]             {
2021-07-16T18:15:00.5234595Z [cluster1]                 "name": "v1",
2021-07-16T18:15:00.5235092Z [cluster1]                 "schema": {
2021-07-16T18:15:00.5235685Z [cluster1]                     "openAPIV3Schema": {
2021-07-16T18:15:00.5236314Z [cluster1]                         "properties": {
2021-07-16T18:15:00.5236902Z [cluster1]                             "apiVersion": {
2021-07-16T18:15:00.5239400Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-16T18:15:00.5241379Z [cluster1]                                 "type": "string"
2021-07-16T18:15:00.5241899Z [cluster1]                             },
2021-07-16T18:15:00.5242377Z [cluster1]                             "kind": {
2021-07-16T18:15:00.5285515Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-16T18:15:00.5287819Z [cluster1]                                 "type": "string"
2021-07-16T18:15:00.5288343Z [cluster1]                             },
2021-07-16T18:15:00.5288865Z [cluster1]                             "metadata": {
2021-07-16T18:15:00.5289417Z [cluster1]                                 "type": "object"
2021-07-16T18:15:00.5289932Z [cluster1]                             },
2021-07-16T18:15:00.5290407Z [cluster1]                             "spec": {
2021-07-16T18:15:00.5291110Z [cluster1]                                 "properties": {
2021-07-16T18:15:00.5291690Z [cluster1]                                     "backend": {
2021-07-16T18:15:00.5292263Z [cluster1]                                         "type": "string"
2021-07-16T18:15:00.5292796Z [cluster1]                                     },
2021-07-16T18:15:00.5293324Z [cluster1]                                     "backend_config": {
2021-07-16T18:15:00.5294053Z [cluster1]                                         "additionalProperties": {
2021-07-16T18:15:00.5294761Z [cluster1]                                             "type": "string"
2021-07-16T18:15:00.5295304Z [cluster1]                                         },
2021-07-16T18:15:00.5295831Z [cluster1]                                         "type": "object"
2021-07-16T18:15:00.5296413Z [cluster1]                                     },
2021-07-16T18:15:00.5296938Z [cluster1]                                     "cable_name": {
2021-07-16T18:15:00.5297497Z [cluster1]                                         "type": "string"
2021-07-16T18:15:00.5298117Z [cluster1]                                     },
2021-07-16T18:15:00.5298625Z [cluster1]                                     "cluster_id": {
2021-07-16T18:15:00.5299200Z [cluster1]                                         "type": "string"
2021-07-16T18:15:00.5299709Z [cluster1]                                     },
2021-07-16T18:15:00.5300283Z [cluster1]                                     "healthCheckIP": {
2021-07-16T18:15:00.5300915Z [cluster1]                                         "type": "string"
2021-07-16T18:15:00.5301425Z [cluster1]                                     },
2021-07-16T18:15:00.5302132Z [cluster1]                                     "hostname": {
2021-07-16T18:15:00.5302699Z [cluster1]                                         "type": "string"
2021-07-16T18:15:00.5303220Z [cluster1]                                     },
2021-07-16T18:15:00.5303732Z [cluster1]                                     "nat_enabled": {
2021-07-16T18:15:00.5304326Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:00.5304849Z [cluster1]                                     },
2021-07-16T18:15:00.5305378Z [cluster1]                                     "private_ip": {
2021-07-16T18:15:00.5305962Z [cluster1]                                         "type": "string"
2021-07-16T18:15:00.5306473Z [cluster1]                                     },
2021-07-16T18:15:00.5306992Z [cluster1]                                     "public_ip": {
2021-07-16T18:15:00.5307553Z [cluster1]                                         "type": "string"
2021-07-16T18:15:00.5308079Z [cluster1]                                     },
2021-07-16T18:15:00.5308589Z [cluster1]                                     "subnets": {
2021-07-16T18:15:00.5309136Z [cluster1]                                         "items": {
2021-07-16T18:15:00.5309686Z [cluster1]                                             "type": "string"
2021-07-16T18:15:00.5310222Z [cluster1]                                         },
2021-07-16T18:15:00.5310755Z [cluster1]                                         "type": "array"
2021-07-16T18:15:00.5311263Z [cluster1]                                     }
2021-07-16T18:15:00.5311737Z [cluster1]                                 },
2021-07-16T18:15:00.5312243Z [cluster1]                                 "required": [
2021-07-16T18:15:00.5312797Z [cluster1]                                     "backend",
2021-07-16T18:15:00.6520012Z [cluster1]                                     "cable_name",
2021-07-16T18:15:00.6520624Z [cluster2]                                     "cluster_id",
2021-07-16T18:15:00.6521237Z [cluster2]                                     "hostname",
2021-07-16T18:15:00.6521795Z [cluster2]                                     "nat_enabled",
2021-07-16T18:15:00.6522371Z [cluster2]                                     "private_ip",
2021-07-16T18:15:00.6522923Z [cluster2]                                     "public_ip",
2021-07-16T18:15:00.6523490Z [cluster2]                                     "subnets"
2021-07-16T18:15:00.6524132Z [cluster2]                                 ],
2021-07-16T18:15:00.6524650Z [cluster2]                                 "type": "object"
2021-07-16T18:15:00.6525160Z [cluster2]                             }
2021-07-16T18:15:00.6525608Z [cluster2]                         },
2021-07-16T18:15:00.6526115Z [cluster2]                         "required": [
2021-07-16T18:15:00.6526629Z [cluster2]                             "spec"
2021-07-16T18:15:00.6527199Z [cluster2]                         ],
2021-07-16T18:15:00.6527698Z [cluster2]                         "type": "object"
2021-07-16T18:15:00.6528196Z [cluster2]                     }
2021-07-16T18:15:00.6528624Z [cluster2]                 },
2021-07-16T18:15:00.6529123Z [cluster2]                 "served": true,
2021-07-16T18:15:00.6529664Z [cluster2]                 "storage": true
2021-07-16T18:15:00.6530226Z [cluster2]             }
2021-07-16T18:15:00.6530654Z [cluster2]         ]
2021-07-16T18:15:00.6531163Z [cluster2]     },
2021-07-16T18:15:00.6531617Z [cluster2]     "status": {
2021-07-16T18:15:00.6532150Z [cluster2]         "acceptedNames": {
2021-07-16T18:15:00.6532743Z [cluster2]             "kind": "Endpoint",
2021-07-16T18:15:00.6533342Z [cluster2]             "listKind": "EndpointList",
2021-07-16T18:15:00.6533980Z [cluster2]             "plural": "endpoints",
2021-07-16T18:15:00.6534664Z [cluster2]             "singular": "endpoint"
2021-07-16T18:15:00.6535184Z [cluster2]         },
2021-07-16T18:15:00.6535656Z [cluster2]         "conditions": [
2021-07-16T18:15:00.6536148Z [cluster2]             {
2021-07-16T18:15:00.6537391Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:58Z",
2021-07-16T18:15:00.6538131Z [cluster2]                 "message": "no conflicts found",
2021-07-16T18:15:00.6538789Z [cluster2]                 "reason": "NoConflicts",
2021-07-16T18:15:00.6539357Z [cluster2]                 "status": "True",
2021-07-16T18:15:00.6539969Z [cluster2]                 "type": "NamesAccepted"
2021-07-16T18:15:00.6540506Z [cluster2]             },
2021-07-16T18:15:00.6541022Z [cluster2]             {
2021-07-16T18:15:00.6541840Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:58Z",
2021-07-16T18:15:00.6542668Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-16T18:15:00.6543510Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-16T18:15:00.6544194Z [cluster2]                 "status": "True",
2021-07-16T18:15:00.6544779Z [cluster2]                 "type": "Established"
2021-07-16T18:15:00.6545283Z [cluster2]             }
2021-07-16T18:15:00.6545709Z [cluster2]         ],
2021-07-16T18:15:00.6546227Z [cluster2]         "storedVersions": [
2021-07-16T18:15:00.6546767Z [cluster2]             "v1"
2021-07-16T18:15:00.6547186Z [cluster2]         ]
2021-07-16T18:15:00.6547609Z [cluster2]     }
2021-07-16T18:15:00.6548007Z [cluster2] }
2021-07-16T18:15:00.6549188Z [submariner-operator]$ [cluster2] validate_equals .metadata.name endpoints.submariner.io
2021-07-16T18:15:00.6550888Z [submariner-operator]$ [cluster2] validate_equals .metadata.name endpoints.submariner.io
2021-07-16T18:15:00.6552797Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/endpoints.submariner.io.cluster2.json
2021-07-16T18:15:00.6554494Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-16T18:15:00.6555907Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-16T18:15:00.6557645Z [submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/endpoints.submariner.io.cluster2.json
2021-07-16T18:15:00.6559343Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-16T18:15:00.6560713Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-16T18:15:00.6562438Z [submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/endpoints.submariner.io.cluster2.json
2021-07-16T18:15:00.6564320Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Endpoint
2021-07-16T18:15:00.6565707Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Endpoint
2021-07-16T18:15:00.6567466Z [submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/endpoints.submariner.io.cluster2.json
2021-07-16T18:15:00.6569361Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Endpoint
2021-07-16T18:15:00.6571015Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Endpoint
2021-07-16T18:15:00.6573068Z [submariner-operator]$ [cluster2] jq -r -M .status.acceptedNames.kind /tmp/endpoints.submariner.io.cluster2.json
2021-07-16T18:15:00.6574890Z [submariner-operator]$ [cluster2] verify_clusters_crd
2021-07-16T18:15:00.6576051Z [submariner-operator]$ [cluster2] verify_clusters_crd
2021-07-16T18:15:00.6577485Z [submariner-operator]$ [cluster2] validate_crd clusters.submariner.io v1 Cluster
2021-07-16T18:15:00.6578979Z [submariner-operator]$ [cluster2] validate_crd clusters.submariner.io v1 Cluster
2021-07-16T18:15:00.6580455Z [submariner-operator]$ [cluster2] kubectl get crds clusters.submariner.io
2021-07-16T18:15:00.6581898Z [submariner-operator]$ [cluster2] kubectl get crds clusters.submariner.io
2021-07-16T18:15:00.7391882Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd  "cable_name",
2021-07-16T18:15:00.7392822Z [cluster1]                                     "cluster_id",
2021-07-16T18:15:00.7393389Z [cluster1]                                     "hostname",
2021-07-16T18:15:00.7393942Z [cluster1]                                     "nat_enabled",
2021-07-16T18:15:00.7394519Z [cluster1]                                     "private_ip",
2021-07-16T18:15:00.7395070Z [cluster1]                                     "public_ip",
2021-07-16T18:15:00.7395631Z [cluster1]                                     "subnets"
2021-07-16T18:15:00.7396125Z [cluster1]                                 ],
2021-07-16T18:15:00.7396655Z [cluster1]                                 "type": "object"
2021-07-16T18:15:00.7397150Z [cluster1]                             }
2021-07-16T18:15:00.7397611Z [cluster1]                         },
2021-07-16T18:15:00.7398119Z [cluster1]                         "required": [
2021-07-16T18:15:00.7398764Z [cluster1]                             "spec"
2021-07-16T18:15:00.7399240Z [cluster1]                         ],
2021-07-16T18:15:00.7399740Z [cluster1]                         "type": "object"
2021-07-16T18:15:00.7400241Z [cluster1]                     }
2021-07-16T18:15:00.7400665Z [cluster1]                 },
2021-07-16T18:15:00.7401156Z [cluster1]                 "served": true,
2021-07-16T18:15:00.7401686Z [cluster1]                 "storage": true
2021-07-16T18:15:00.7402169Z [cluster1]             }
2021-07-16T18:15:00.7402581Z [cluster1]         ]
2021-07-16T18:15:00.7403008Z [cluster1]     },
2021-07-16T18:15:00.7403457Z [cluster1]     "status": {
2021-07-16T18:15:00.7404102Z [cluster1]         "acceptedNames": {
2021-07-16T18:15:00.7404694Z [cluster1]             "kind": "Endpoint",
2021-07-16T18:15:00.7405290Z [cluster1]             "listKind": "EndpointList",
2021-07-16T18:15:00.7405936Z [cluster1]             "plural": "endpoints",
2021-07-16T18:15:00.7406529Z [cluster1]             "singular": "endpoint"
2021-07-16T18:15:00.7407220Z [cluster1]         },
2021-07-16T18:15:00.7407699Z [cluster1]         "conditions": [
2021-07-16T18:15:00.7408190Z [cluster1]             {
2021-07-16T18:15:00.7409074Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:43Z",
2021-07-16T18:15:00.7409830Z [cluster1]                 "message": "no conflicts found",
2021-07-16T18:15:00.7410482Z [cluster1]                 "reason": "NoConflicts",
2021-07-16T18:15:00.7411053Z [cluster1]                 "status": "True",
2021-07-16T18:15:00.7411660Z [cluster1]                 "type": "NamesAccepted"
2021-07-16T18:15:00.7412190Z [cluster1]             },
2021-07-16T18:15:00.7412628Z [cluster1]             {
2021-07-16T18:15:00.7413431Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:43Z",
2021-07-16T18:15:00.7414255Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-16T18:15:00.7415079Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-16T18:15:00.7415775Z [cluster1]                 "status": "True",
2021-07-16T18:15:00.7416335Z [cluster1]                 "type": "Established"
2021-07-16T18:15:00.7416852Z [cluster1]             }
2021-07-16T18:15:00.7417275Z [cluster1]         ],
2021-07-16T18:15:00.7417789Z [cluster1]         "storedVersions": [
2021-07-16T18:15:00.7418331Z [cluster1]             "v1"
2021-07-16T18:15:00.7418751Z [cluster1]         ]
2021-07-16T18:15:00.7419175Z [cluster1]     }
2021-07-16T18:15:00.7419573Z [cluster1] }
2021-07-16T18:15:00.7420734Z [submariner-operator]$ [cluster1] validate_equals .metadata.name endpoints.submariner.io
2021-07-16T18:15:00.7422520Z [submariner-operator]$ [cluster1] validate_equals .metadata.name endpoints.submariner.io
2021-07-16T18:15:00.7424436Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/endpoints.submariner.io.cluster1.json
2021-07-16T18:15:00.7426129Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-16T18:15:00.7427457Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-16T18:15:00.7429163Z [submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/endpoints.submariner.io.cluster1.json
2021-07-16T18:15:00.7430839Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-16T18:15:00.7432196Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-16T18:15:00.7433887Z [submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/endpoints.submariner.io.cluster1.json
2021-07-16T18:15:00.7435580Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Endpoint
2021-07-16T18:15:00.7436951Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Endpoint
2021-07-16T18:15:00.7438711Z [submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/endpoints.submariner.io.cluster1.json
2021-07-16T18:15:00.7440589Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Endpoint
2021-07-16T18:15:00.7442200Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Endpoint
2021-07-16T18:15:00.7444431Z [submariner-operator]$ [cluster1] jq -r -M .status.acceptedNames.kind /tmp/endpoints.submariner.io.cluster1.json
2021-07-16T18:15:00.7446149Z [submariner-operator]$ [cluster1] verify_clusters_crd
2021-07-16T18:15:00.7447235Z [submariner-operator]$ [cluster1] verify_clusters_crd
2021-07-16T18:15:00.7448549Z [submariner-operator]$ [cluster1] validate_crd clusters.submariner.io v1 Cluster
2021-07-16T18:15:00.7450039Z [submariner-operator]$ [cluster1] validate_crd clusters.submariner.io v1 Cluster
2021-07-16T18:15:00.7451507Z [submariner-operator]$ [cluster1] kubectl get crds clusters.submariner.io
2021-07-16T18:15:00.7452936Z [submariner-operator]$ [cluster1] kubectl get crds clusters.submariner.io
2021-07-16T18:15:01.5844682Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds clusters.submariner.io
2021-07-16T18:15:01.5845808Z [cluster1] NAME                     CREATED AT
2021-07-16T18:15:01.5846774Z [cluster1] clusters.submariner.io   2021-07-16T18:13:43Z
2021-07-16T18:15:01.5848245Z [submariner-operator]$ [cluster1] json_file=/tmp/clusters.submariner.io.cluster1.json
2021-07-16T18:15:01.5849936Z [submariner-operator]$ [cluster1] kubectl get crd clusters.submariner.io -o json
2021-07-16T18:15:01.5851607Z [submariner-operator]$ [cluster1] tee /tmp/clusters.submariner.io.cluster1.json
2021-07-16T18:15:01.5853269Z [submariner-operator]$ [cluster1] kubectl get crd clusters.submariner.io -o json
2021-07-16T18:15:01.5854904Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd clusters.submariner.io -o json
2021-07-16T18:15:01.5855868Z [cluster1] {
2021-07-16T18:15:01.5856529Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:01.5857426Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-16T18:15:01.5858139Z [cluster1]     "metadata": {
2021-07-16T18:15:01.5858659Z [cluster1]         "annotations": {
2021-07-16T18:15:01.5859715Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-16T18:15:01.5860486Z [cluster1]         },
2021-07-16T18:15:01.5861257Z [cluster1]         "creationTimestamp": "2021-07-16T18:13:43Z",
2021-07-16T18:15:01.5861922Z [cluster1]         "generation": 1,
2021-07-16T18:15:01.5862493Z [cluster1]         "managedFields": [
2021-07-16T18:15:01.5863179Z [cluster1]             {
2021-07-16T18:15:01.5863845Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:01.5864630Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:01.5865211Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:01.5865750Z [cluster1]                     "f:metadata": {
2021-07-16T18:15:01.5866317Z [cluster1]                         "f:annotations": {
2021-07-16T18:15:01.5866859Z [cluster1]                             ".": {},
2021-07-16T18:15:01.5867899Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-16T18:15:01.5868691Z [cluster1]                         }
2021-07-16T18:15:01.5869142Z [cluster1]                     },
2021-07-16T18:15:01.5869609Z [cluster1]                     "f:spec": {
2021-07-16T18:15:01.5870155Z [cluster1]                         "f:conversion": {
2021-07-16T18:15:01.5870673Z [cluster1]                             ".": {},
2021-07-16T18:15:01.5871199Z [cluster1]                             "f:strategy": {}
2021-07-16T18:15:01.5871691Z [cluster1]                         },
2021-07-16T18:15:01.5872180Z [cluster1]                         "f:group": {},
2021-07-16T18:15:01.5872699Z [cluster1]                         "f:names": {
2021-07-16T18:15:01.5873199Z [cluster1]                             "f:kind": {},
2021-07-16T18:15:01.5873734Z [cluster1]                             "f:listKind": {},
2021-07-16T18:15:01.5874277Z [cluster1]                             "f:plural": {},
2021-07-16T18:15:01.5874830Z [cluster1]                             "f:singular": {}
2021-07-16T18:15:01.5875322Z [cluster1]                         },
2021-07-16T18:15:01.5875813Z [cluster1]                         "f:scope": {},
2021-07-16T18:15:01.5876340Z [cluster1]                         "f:versions": {}
2021-07-16T18:15:01.5876840Z [cluster1]                     },
2021-07-16T18:15:01.5877476Z [cluster1]                     "f:status": {
2021-07-16T18:15:01.5878071Z [cluster1]                         "f:acceptedNames": {
2021-07-16T18:15:01.5878657Z [cluster1]                             "f:kind": {},
2021-07-16T18:15:01.5879172Z [cluster1]                             "f:plural": {}
2021-07-16T18:15:01.5879671Z [cluster1]                         },
2021-07-16T18:15:01.5880180Z [cluster1]                         "f:conditions": {},
2021-07-16T18:15:01.5880808Z [cluster1]                         "f:storedVersions": {}
2021-07-16T18:15:01.5881445Z [cluster1]                     }
2021-07-16T18:15:01.5881899Z [cluster1]                 },
2021-07-16T18:15:01.5882746Z [cluster1]                 "manager": "submariner-operator",
2021-07-16T18:15:01.5883466Z [cluster1]                 "operation": "Update",
2021-07-16T18:15:01.5884507Z [cluster1]                 "time": "2021-07-16T18:13:48Z"
2021-07-16T18:15:01.5884982Z [cluster1]             }
2021-07-16T18:15:01.5885412Z [cluster1]         ],
2021-07-16T18:15:01.5886037Z [cluster1]         "name": "clusters.submariner.io",
2021-07-16T18:15:01.5886787Z [cluster1]         "resourceVersion": "1829",
2021-07-16T18:15:01.5887976Z [cluster1]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.submariner.io",
2021-07-16T18:15:01.5889488Z [cluster1]         "uid": "f0e0e233-793e-4d76-b1ec-e1c8e072af80"
2021-07-16T18:15:01.5890132Z [cluster1]     },
2021-07-16T18:15:01.5890575Z [cluster1]     "spec": {
2021-07-16T18:15:01.5891071Z [cluster1]         "conversion": {
2021-07-16T18:15:01.5891633Z [cluster1]             "strategy": "None"
2021-07-16T18:15:01.5892123Z [cluster1]         },
2021-07-16T18:15:01.5892650Z [cluster1]         "group": "submariner.io",
2021-07-16T18:15:01.5893285Z [cluster1]         "names": {
2021-07-16T18:15:01.5893785Z [cluster1]             "kind": "Cluster",
2021-07-16T18:15:01.5894383Z [cluster1]             "listKind": "ClusterList",
2021-07-16T18:15:01.5894985Z [cluster1]             "plural": "clusters",
2021-07-16T18:15:01.5895565Z [cluster1]             "singular": "cluster"
2021-07-16T18:15:01.5896159Z [cluster1]         },
2021-07-16T18:15:01.5896678Z [cluster1]         "scope": "Namespaced",
2021-07-16T18:15:01.5897216Z [cluster1]         "versions": [
2021-07-16T18:15:01.5897690Z [cluster1]             {
2021-07-16T18:15:01.5898142Z [cluster1]                 "name": "v1",
2021-07-16T18:15:01.5898644Z [cluster1]                 "schema": {
2021-07-16T18:15:01.5899238Z [cluster1]                     "openAPIV3Schema": {
2021-07-16T18:15:01.5899869Z [cluster1]                         "properties": {
2021-07-16T18:15:01.5900463Z [cluster1]                             "apiVersion": {
2021-07-16T18:15:01.5946684Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-16T18:15:01.5948715Z [cluster1]                                 "type": "string"
2021-07-16T18:15:01.5949239Z [cluster1]                             },
2021-07-16T18:15:01.5949732Z [cluster1]                             "kind": {
2021-07-16T18:15:01.5952133Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-16T18:15:01.5954216Z [cluster1]                                 "type": "string"
2021-07-16T18:15:01.5954734Z [cluster1]                             },
2021-07-16T18:15:01.5955239Z [cluster1]                             "metadata": {
2021-07-16T18:15:01.5955810Z [cluster1]                                 "type": "object"
2021-07-16T18:15:01.5956311Z [cluster1]                             },
2021-07-16T18:15:01.5956802Z [cluster1]                             "spec": {
2021-07-16T18:15:01.5957351Z [cluster1]                                 "properties": {
2021-07-16T18:15:01.5957953Z [cluster1]                                     "cluster_cidr": {
2021-07-16T18:15:01.5958513Z [cluster1]                                         "items": {
2021-07-16T18:15:01.5959082Z [cluster1]                                             "type": "string"
2021-07-16T18:15:01.5959779Z [cluster1]                                         },
2021-07-16T18:15:01.5960303Z [cluster1]                                         "type": "array"
2021-07-16T18:15:01.5960825Z [cluster1]                                     },
2021-07-16T18:15:01.5961340Z [cluster1]                                     "cluster_id": {
2021-07-16T18:15:01.5961922Z [cluster1]                                         "type": "string"
2021-07-16T18:15:01.5962435Z [cluster1]                                     },
2021-07-16T18:15:01.5962954Z [cluster1]                                     "color_codes": {
2021-07-16T18:15:01.5964257Z [cluster1]                                         "items": {
2021-07-16T18:15:01.5964820Z [cluster1]                                             "type": "string"
2021-07-16T18:15:01.5965356Z [cluster1]                                         },
2021-07-16T18:15:01.5965874Z [cluster1]                                         "type": "array"
2021-07-16T18:15:01.5966394Z [cluster1]                                     },
2021-07-16T18:15:01.5966907Z [cluster1]                                     "global_cidr": {
2021-07-16T18:15:01.5967457Z [cluster1]                                         "items": {
2021-07-16T18:15:01.5968004Z [cluster1]                                             "type": "string"
2021-07-16T18:15:01.5968534Z [cluster1]                                         },
2021-07-16T18:15:01.5969068Z [cluster1]                                         "type": "array"
2021-07-16T18:15:01.5969576Z [cluster1]                                     },
2021-07-16T18:15:01.5970214Z [cluster1]                                     "service_cidr": {
2021-07-16T18:15:01.5970766Z [cluster1]                                         "items": {
2021-07-16T18:15:01.5971329Z [cluster1]                                             "type": "string"
2021-07-16T18:15:01.5971846Z [cluster1]                                         },
2021-07-16T18:15:01.5972373Z [cluster1]                                         "type": "array"
2021-07-16T18:15:01.5972883Z [cluster1]                                     }
2021-07-16T18:15:01.5973355Z [cluster1]                                 },
2021-07-16T18:15:01.5973879Z [cluster1]                                 "required": [
2021-07-16T18:15:01.5974444Z [cluster1]                                     "cluster_cidr",
2021-07-16T18:15:01.5975011Z [cluster1]                                     "cluster_id",
2021-07-16T18:15:01.5975565Z [cluster1]                                     "color_codes",
2021-07-16T18:15:01.5976129Z [cluster1]                                     "global_cidr",
2021-07-16T18:15:01.5976682Z [cluster1]                                     "service_cidr"
2021-07-16T18:15:01.5977203Z [cluster1]                                 ],
2021-07-16T18:15:01.5977713Z [cluster1]                                 "type": "object"
2021-07-16T18:15:01.5978222Z [cluster1]                             }
2021-07-16T18:15:01.5978683Z [cluster1]                         },
2021-07-16T18:15:01.5979183Z [cluster1]                         "required": [
2021-07-16T18:15:01.5979709Z [cluster1]                             "spec"
2021-07-16T18:15:01.5980170Z [cluster1]                         ],
2021-07-16T18:15:01.5980681Z [cluster1]                         "type": "object"
2021-07-16T18:15:01.5981164Z [cluster1]                     }
2021-07-16T18:15:01.5981605Z [cluster1]                 },
2021-07-16T18:15:01.5982087Z [cluster1]                 "served": true,
2021-07-16T18:15:01.5982628Z [cluster1]                 "storage": true
2021-07-16T18:15:01.5983095Z [cluster1]             }
2021-07-16T18:15:01.5983521Z [cluster1]         ]
2021-07-16T18:15:01.5983923Z [cluster1]     },
2021-07-16T18:15:01.5984373Z [cluster1]     "status": {
2021-07-16T18:15:01.5984922Z [cluster1]         "acceptedNames": {
2021-07-16T18:15:01.5985493Z [cluster1]             "kind": "Cluster",
2021-07-16T18:15:01.5986095Z [cluster1]             "listKind": "ClusterList",
2021-07-16T18:15:01.5986696Z [cluster1]             "plural": "clusters",
2021-07-16T18:15:01.5987383Z [cluster1]             "singular": "cluster"
2021-07-16T18:15:01.5987875Z [cluster1]         },
2021-07-16T18:15:01.5988475Z [cluster1]         "conditions": [
2021-07-16T18:15:01.5988951Z [cluster1]             {
2021-07-16T18:15:01.6182332Z [cluster1]                 "lastTransitionTime": "2021-07-16Ts clusters.submariner.io
2021-07-16T18:15:01.6183260Z [cluster2] NAME                     CREATED AT
2021-07-16T18:15:01.6184296Z [cluster2] clusters.submariner.io   2021-07-16T18:13:58Z
2021-07-16T18:15:01.6185815Z [submariner-operator]$ [cluster2] json_file=/tmp/clusters.submariner.io.cluster2.json
2021-07-16T18:15:01.6187494Z [submariner-operator]$ [cluster2] kubectl get crd clusters.submariner.io -o json
2021-07-16T18:15:01.6189144Z [submariner-operator]$ [cluster2] tee /tmp/clusters.submariner.io.cluster2.json
2021-07-16T18:15:01.6190810Z [submariner-operator]$ [cluster2] kubectl get crd clusters.submariner.io -o json
2021-07-16T18:15:01.6192442Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd clusters.submariner.io -o json
2021-07-16T18:15:01.6193418Z [cluster2] {
2021-07-16T18:15:01.6194071Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:01.6194981Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-16T18:15:01.6195694Z [cluster2]     "metadata": {
2021-07-16T18:15:01.6196211Z [cluster2]         "annotations": {
2021-07-16T18:15:01.6197249Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-16T18:15:01.6198188Z [cluster2]         },
2021-07-16T18:15:01.6198981Z [cluster2]         "creationTimestamp": "2021-07-16T18:13:58Z",
2021-07-16T18:15:01.6199620Z [cluster2]         "generation": 1,
2021-07-16T18:15:01.6200194Z [cluster2]         "managedFields": [
2021-07-16T18:15:01.6200699Z [cluster2]             {
2021-07-16T18:15:01.6201383Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:01.6202167Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:01.6202746Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:01.6203291Z [cluster2]                     "f:metadata": {
2021-07-16T18:15:01.6204752Z [cluster2]                         "f:annotations": {
2021-07-16T18:15:01.6205294Z [cluster2]                             ".": {},
2021-07-16T18:15:01.6206328Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-16T18:15:01.6207133Z [cluster2]                         }
2021-07-16T18:15:01.6207578Z [cluster2]                     },
2021-07-16T18:15:01.6208058Z [cluster2]                     "f:spec": {
2021-07-16T18:15:01.6208592Z [cluster2]                         "f:conversion": {
2021-07-16T18:15:01.6209121Z [cluster2]                             ".": {},
2021-07-16T18:15:01.6209643Z [cluster2]                             "f:strategy": {}
2021-07-16T18:15:01.6210135Z [cluster2]                         },
2021-07-16T18:15:01.6210633Z [cluster2]                         "f:group": {},
2021-07-16T18:15:01.6211136Z [cluster2]                         "f:names": {
2021-07-16T18:15:01.6211650Z [cluster2]                             "f:kind": {},
2021-07-16T18:15:01.6212178Z [cluster2]                             "f:listKind": {},
2021-07-16T18:15:01.6212739Z [cluster2]                             "f:plural": {},
2021-07-16T18:15:01.6213281Z [cluster2]                             "f:singular": {}
2021-07-16T18:15:01.6213782Z [cluster2]                         },
2021-07-16T18:15:01.6214264Z [cluster2]                         "f:scope": {},
2021-07-16T18:15:01.6214800Z [cluster2]                         "f:versions": {}
2021-07-16T18:15:01.6215300Z [cluster2]                     },
2021-07-16T18:15:01.6215772Z [cluster2]                     "f:status": {
2021-07-16T18:15:01.6216361Z [cluster2]                         "f:acceptedNames": {
2021-07-16T18:15:01.6216928Z [cluster2]                             "f:kind": {},
2021-07-16T18:15:01.6217577Z [cluster2]                             "f:plural": {}
2021-07-16T18:15:01.6218067Z [cluster2]                         },
2021-07-16T18:15:01.6218596Z [cluster2]                         "f:conditions": {},
2021-07-16T18:15:01.6219210Z [cluster2]                         "f:storedVersions": {}
2021-07-16T18:15:01.6219763Z [cluster2]                     }
2021-07-16T18:15:01.6220204Z [cluster2]                 },
2021-07-16T18:15:01.6221036Z [cluster2]                 "manager": "submariner-operator",
2021-07-16T18:15:01.6221750Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:01.6222500Z [cluster2]                 "time": "2021-07-16T18:14:08Z"
2021-07-16T18:15:01.6222981Z [cluster2]             }
2021-07-16T18:15:01.6223396Z [cluster2]         ],
2021-07-16T18:15:01.6224032Z [cluster2]         "name": "clusters.submariner.io",
2021-07-16T18:15:01.6224760Z [cluster2]         "resourceVersion": "1858",
2021-07-16T18:15:01.6225964Z [cluster2]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/clusters.submariner.io",
2021-07-16T18:15:01.6227443Z [cluster2]         "uid": "1db03093-6aff-47ee-a605-1c56ce5f8469"
2021-07-16T18:15:01.6228094Z [cluster2]     },
2021-07-16T18:15:01.6228533Z [cluster2]     "spec": {
2021-07-16T18:15:01.6229029Z [cluster2]         "conversion": {
2021-07-16T18:15:01.6229584Z [cluster2]             "strategy": "None"
2021-07-16T18:15:01.6230059Z [cluster2]         },
2021-07-16T18:15:01.6230589Z [cluster2]         "group": "submariner.io",
2021-07-16T18:15:01.6231125Z [cluster2]         "names": {
2021-07-16T18:15:01.6231733Z [cluster2]             "kind": "Cluster",
2021-07-16T18:15:01.6232319Z [cluster2]             "listKind": "ClusterList",
2021-07-16T18:15:01.6232933Z [cluster2]             "plural": "clusters",
2021-07-16T18:15:01.6233502Z [cluster2]             "singular": "cluster"
2021-07-16T18:15:01.6234003Z [cluster2]         },
2021-07-16T18:15:01.6234517Z [cluster2]         "scope": "Namespaced",
2021-07-16T18:15:01.6235057Z [cluster2]         "versions": [
2021-07-16T18:15:01.6235535Z [cluster2]             {
2021-07-16T18:15:01.6235984Z [cluster2]                 "name": "v1",
2021-07-16T18:15:01.6236482Z [cluster2]                 "schema": {
2021-07-16T18:15:01.6237060Z [cluster2]                     "openAPIV3Schema": {
2021-07-16T18:15:01.6237691Z [cluster2]                         "properties": {
2021-07-16T18:15:01.6238275Z [cluster2]                             "apiVersion": {
2021-07-16T18:15:01.6260061Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-16T18:15:01.6266520Z [cluster2]                                 "type": "string"
2021-07-16T18:15:01.6267199Z [cluster2]                             },
2021-07-16T18:15:01.6267812Z [cluster2]                             "kind": {
2021-07-16T18:15:01.6270501Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-16T18:15:01.6275454Z [cluster2]                                 "type": "string"
2021-07-16T18:15:01.6276153Z [cluster2]                             },
2021-07-16T18:15:01.6276779Z [cluster2]                             "metadata": {
2021-07-16T18:15:01.6277438Z [cluster2]                                 "type": "object"
2021-07-16T18:15:01.6278059Z [cluster2]                             },
2021-07-16T18:15:01.6278652Z [cluster2]                             "spec": {
2021-07-16T18:15:01.6279321Z [cluster2]                                 "properties": {
2021-07-16T18:15:01.6280189Z [cluster2]                                     "cluster_cidr": {
2021-07-16T18:15:01.6281009Z [cluster2]                                         "items": {
2021-07-16T18:15:01.6282050Z [cluster2]                                             "type": "string"
2021-07-16T18:15:01.6284011Z [cluster2]                                         },
2021-07-16T18:15:01.6284788Z [cluster2]                                         "type": "array"
2021-07-16T18:15:01.6285418Z [cluster2]                                     },
2021-07-16T18:15:01.6309713Z [cluster2]                                     "cluster_id": {
2021-07-16T18:15:01.6310407Z [cluster2]                                         "type": "string"
2021-07-16T18:15:01.6311044Z [cluster2]                                     },
2021-07-16T18:15:01.6311680Z [cluster2]                                     "color_codes": {
2021-07-16T18:15:01.6312472Z [cluster2]                                         "items": {
2021-07-16T18:15:01.6313140Z [cluster2]                                             "type": "string"
2021-07-16T18:15:01.6313784Z [cluster2]                                         },
2021-07-16T18:15:01.6314395Z [cluster2]                                         "type": "array"
2021-07-16T18:15:01.6315011Z [cluster2]                                     },
2021-07-16T18:15:01.6315651Z [cluster2]                                     "global_cidr": {
2021-07-16T18:15:01.6316282Z [cluster2]                                         "items": {
2021-07-16T18:15:01.6316953Z [cluster2]                                             "type": "string"
2021-07-16T18:15:01.6317743Z [cluster2]                                         },
2021-07-16T18:15:01.6318369Z [cluster2]                                         "type": "array"
2021-07-16T18:15:01.6318973Z [cluster2]                                     },
2021-07-16T18:15:01.6319615Z [cluster2]                                     "service_cidr": {
2021-07-16T18:15:01.6320281Z [cluster2]                                         "items": {
2021-07-16T18:15:01.6320951Z [cluster2]                                             "type": "string"
2021-07-16T18:15:01.6321567Z [cluster2]                                         },
2021-07-16T18:15:01.6322200Z [cluster2]                                         "type": "array"
2021-07-16T18:15:01.6322805Z [cluster2]                                     }
2021-07-16T18:15:01.6323388Z [cluster2]                                 },
2021-07-16T18:15:01.6325342Z [cluster2]                                 "required": [
2021-07-16T18:15:01.6325936Z [cluster2]                                     "cluster_cidr",
2021-07-16T18:15:01.6326486Z [cluster2]                                     "cluster_id",
2021-07-16T18:15:01.6327052Z [cluster2]                                     "color_codes",
2021-07-16T18:15:01.6327609Z [cluster2]                                     "global_cidr",
2021-07-16T18:15:01.6328168Z [cluster2]                                     "service_cidr"
2021-07-16T18:15:01.6328792Z [cluster2]                                 ],
2021-07-16T18:15:01.6329314Z [cluster2]                                 "type": "object"
2021-07-16T18:15:01.6329906Z [cluster2]                             }
2021-07-16T18:15:01.6330349Z [cluster2]                         },
2021-07-16T18:15:01.6330854Z [cluster2]                         "required": [
2021-07-16T18:15:01.6331366Z [cluster2]                             "spec"
2021-07-16T18:15:01.6331838Z [cluster2]                         ],
2021-07-16T18:15:01.6332355Z [cluster2]                         "type": "object"
2021-07-16T18:15:01.6332844Z [cluster2]                     }
2021-07-16T18:15:01.6333286Z [cluster2]                 },
2021-07-16T18:15:01.6333770Z [cluster2]                 "served": true,
2021-07-16T18:15:01.6334319Z [cluster2]                 "storage": true
2021-07-16T18:15:01.6334789Z [cluster2]             }
2021-07-16T18:15:01.6335216Z [cluster2]         ]
2021-07-16T18:15:01.6335623Z [cluster2]     },
2021-07-16T18:15:01.6336071Z [cluster2]     "status": {
2021-07-16T18:15:01.6336730Z [cluster2]         "acceptedNames": {
2021-07-16T18:15:01.6337327Z [cluster2]             "kind": "Cluster",
2021-07-16T18:15:01.6337905Z [cluster2]             "listKind": "ClusterList",
2021-07-16T18:15:01.6338524Z [cluster2]             "plural": "clusters",
2021-07-16T18:15:01.6339109Z [cluster2]             "singular": "cluster"
2021-07-16T18:15:01.6339596Z [cluster2]         },
2021-07-16T18:15:01.6340080Z [cluster2]         "conditions": [
2021-07-16T18:15:01.6340554Z [cluster2]             {
2021-07-16T18:15:02.7284334Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:43Z",
2021-07-16T18:15:02.7286168Z [cluster1]                 "message": "no conflicts found",
2021-07-16T18:15:02.7287051Z [cluster1]                 "reason": "NoConflicts",
2021-07-16T18:15:02.7287807Z [cluster1]                 "status": "True",
2021-07-16T18:15:02.7288589Z [cluster1]                 "type": "NamesAccepted"
2021-07-16T18:15:02.7289296Z [cluster1]             },
2021-07-16T18:15:02.7289927Z [cluster1]             {
2021-07-16T18:15:02.7290991Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:43Z",
2021-07-16T18:15:02.7292023Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-16T18:15:02.7293034Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-16T18:15:02.7294013Z [cluster1]                 "status": "True",
2021-07-16T18:15:02.7294779Z [cluster1]                 "type": "Established"
2021-07-16T18:15:02.7295738Z [cluster1]             }
2021-07-16T18:15:02.7296344Z [cluster1]         ],
2021-07-16T18:15:02.7297691Z [cluster1]         "storedVersions": [
2021-07-16T18:15:02.7298378Z [cluster1]             "v1"
2021-07-16T18:15:02.7298905Z [cluster1]         ]
2021-07-16T18:15:02.7299474Z [cluster1]     }
2021-07-16T18:15:02.7299970Z [cluster1] }
2021-07-16T18:15:02.7301333Z [submariner-operator]$ [cluster1] validate_equals .metadata.name clusters.submariner.io
2021-07-16T18:15:02.7303137Z [submariner-operator]$ [cluster1] validate_equals .metadata.name clusters.submariner.io
2021-07-16T18:15:02.7305136Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/clusters.submariner.io.cluster1.json
2021-07-16T18:15:02.7306937Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-16T18:15:02.7308400Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-16T18:15:02.7310194Z [submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/clusters.submariner.io.cluster1.json
2021-07-16T18:15:02.7312018Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-16T18:15:02.7313523Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-16T18:15:02.7315328Z [submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/clusters.submariner.io.cluster1.json
2021-07-16T18:15:02.7317132Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Cluster
2021-07-16T18:15:02.7319223Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Cluster
2021-07-16T18:15:02.7338740Z [submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/clusters.submariner.io.cluster1.json
2021-07-16T18:15:02.7341012Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Cluster
2021-07-16T18:15:02.7342661Z [submariner-operator]$ [cluster1] validate_equals .status.acceptedNames.kind Cluster
2021-07-16T18:15:02.7344708Z [submariner-operator]$ [cluster1] jq -r -M .status.acceptedNames.kind /tmp/clusters.submariner.io.cluster1.json
2021-07-16T18:15:02.7346359Z [submariner-operator]$ [cluster1] verify_subm_crd
2021-07-16T18:15:02.7347394Z [submariner-operator]$ [cluster1] verify_subm_crd
2021-07-16T18:15:02.7348806Z [submariner-operator]$ [cluster1] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-16T18:15:02.7350718Z [submariner-operator]$ [cluster1] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-16T18:15:02.7352376Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-16T18:15:02.7354058Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-16T18:15:02.7355756Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds submariners.submariner.io
2021-07-16T18:15:02.7356904Z [cluster1] NAME                        CREATED AT
2021-07-16T18:15:02.7357868Z [cluster1] submariners.submariner.io   2021-07-16T18:13:33Z
2021-07-16T18:15:02.7359463Z [submariner-operator]$ [cluster1] json_file=/tmp/submariners.submariner.io.cluster1.json
2021-07-16T18:15:02.7361234Z [submariner-operator]$ [cluster1] kubectl get crd submariners.submariner.io -o json
2021-07-16T18:15:02.7422975Z [submariner-operator]$ [cluster1] tee /tmp/submariners.submariner.io.cluster1.json
2021-07-16T18:15:02.7424820Z [submariner-operator]$ [cluster1] kubectl get crd submariners.submariner.io -o json
2021-07-16T18:15:02.7426774Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crd submariners.submariner.io -o json
2021-07-16T18:15:02.7427802Z [cluster1] {
2021-07-16T18:15:02.7428457Z [cluster1]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:02.7429370Z [cluster1]     "kind": "CustomResourceDefinition",
2021-07-16T18:15:02.7430070Z [cluster1]     "metadata": {
2021-07-16T18:15:02.7430824Z [cluster1]         "annotations": {
2021-07-16T18:15:02.7431892Z [cluster1]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-16T18:15:02.7432690Z [cluster1]         },
2021-07-16T18:15:02.7433467Z [cluster1]         "creationTimestamp": "2021-07-16T18:13:33Z",
2021-07-16T18:15:02.7434136Z [cluster1]         "generation": 1,
2021-07-16T18:15:02.7434707Z [cluster1]         "managedFields": [
2021-07-16T18:15:02.7435226Z [cluster1]             {
2021-07-16T18:15:02.7435896Z [cluster1]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:02.7436679Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:02.7437269Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:02.7437798Z [cluster1]                     "f:metadata": {
2021-07-16T18:15:02.7438372Z [cluster1]                         "f:annotations": {
2021-07-16T18:15:02.7438901Z [cluster1]                             ".": {},
2021-07-16T18:15:02.7439926Z [cluster1]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-16T18:15:02.7440719Z [cluster1]                         }
2021-07-16T18:15:02.7441172Z [cluster1]                     },
2021-07-16T18:15:02.7441638Z [cluster1]                     "f:spec": {
2021-07-16T18:15:02.7442184Z [cluster1]                         "f:conversion": {
2021-07-16T18:15:02.7442715Z [cluster1]                             ".": {},
2021-07-16T18:15:02.7443219Z [cluster1]                             "f:strategy": {}
2021-07-16T18:15:02.7444799Z [cluster1]                         },
2021-07-16T18:15:02.7445289Z [cluster1]                         "f:group": {},
2021-07-16T18:15:02.7445814Z [cluster1]                         "f:names": {
2021-07-16T18:15:02.7446321Z [cluster1]                             "f:kind": {},
2021-07-16T18:15:02.7446867Z [cluster1]                             "f:listKind": {},
2021-07-16T18:15:02.7447414Z [cluster1]                             "f:plural": {},
2021-07-16T18:15:02.7447973Z [cluster1]                             "f:singular": {}
2021-07-16T18:15:02.7448483Z [cluster1]                         },
2021-07-16T18:15:02.7448966Z [cluster1]                         "f:scope": {},
2021-07-16T18:15:02.7449506Z [cluster1]                         "f:versions": {}
2021-07-16T18:15:02.7449995Z [cluster1]                     },
2021-07-16T18:15:02.7450482Z [cluster1]                     "f:status": {
2021-07-16T18:15:02.7451058Z [cluster1]                         "f:acceptedNames": {
2021-07-16T18:15:02.7451776Z [cluster1]                             "f:kind": {},
2021-07-16T18:15:02.7452304Z [cluster1]                             "f:plural": {}
2021-07-16T18:15:02.7452801Z [cluster1]                         },
2021-07-16T18:15:02.7453315Z [cluster1]                         "f:conditions": {},
2021-07-16T18:15:02.7453956Z [cluster1]                         "f:storedVersions": {}
2021-07-16T18:15:02.7454514Z [cluster1]                     }
2021-07-16T18:15:02.7454948Z [cluster1]                 },
2021-07-16T18:15:02.7455470Z [cluster1]                 "manager": "subctl",
2021-07-16T18:15:02.7456044Z [cluster1]                 "operation": "Update",
2021-07-16T18:15:02.7456895Z [cluster1]                 "time": "2021-07-16T18:13:45Z"
2021-07-16T18:15:02.7457376Z [cluster1]             }
2021-07-16T18:15:02.7457812Z [cluster1]         ],
2021-07-16T18:15:02.7458481Z [cluster1]         "name": "submariners.submariner.io",
2021-07-16T18:15:02.7459268Z [cluster1]         "resourceVersion": "1791",
2021-07-16T18:15:02.7460516Z [cluster1]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/submariners.submariner.io",
2021-07-16T18:15:02.7461975Z [cluster1]         "uid": "45f337f6-8324-4847-9a33-94266b1b11e1"
2021-07-16T18:15:02.7462525Z [cluster1]     },
2021-07-16T18:15:02.7462955Z [cluster1]     "spec": {
2021-07-16T18:15:02.7463470Z [cluster1]         "conversion": {
2021-07-16T18:15:02.7464015Z [cluster1]             "strategy": "None"
2021-07-16T18:15:02.7464514Z [cluster1]         },
2021-07-16T18:15:02.7465132Z [cluster1]         "group": "submariner.io",
2021-07-16T18:15:02.7465688Z [cluster1]         "names": {
2021-07-16T18:15:02.7466204Z [cluster1]             "kind": "Submariner",
2021-07-16T18:15:02.7466856Z [cluster1]             "listKind": "SubmarinerList",
2021-07-16T18:15:02.7467509Z [cluster1]             "plural": "submariners",
2021-07-16T18:15:02.7468132Z [cluster1]             "singular": "submariner"
2021-07-16T18:15:02.7468658Z [cluster1]         },
2021-07-16T18:15:02.7469166Z [cluster1]         "scope": "Namespaced",
2021-07-16T18:15:02.7469722Z [cluster1]         "versions": [
2021-07-16T18:15:02.7470180Z [cluster1]             {
2021-07-16T18:15:02.7470679Z [cluster1]                 "name": "v1alpha1",
2021-07-16T18:15:02.7471190Z [cluster1]                 "schema": {
2021-07-16T18:15:02.7471778Z [cluster1]                     "openAPIV3Schema": {
2021-07-16T18:15:02.7472622Z [cluster1]                         "description": "Submariner is the Schema for the submariners API",
2021-07-16T18:15:02.7473424Z [cluster1]                         "properties": {
2021-07-16T18:15:02.7474011Z [cluster1]                             "apiVersion": {
2021-07-16T18:15:02.7476535Z [cluster1]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-16T18:15:02.7478515Z [cluster1]                                 "type": "string"
2021-07-16T18:15:02.7479021Z [cluster1]                             },
2021-07-16T18:15:02.7479514Z [cluster1]                             "kind": {
2021-07-16T18:15:02.7481911Z [cluster1]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-16T18:15:02.7550068Z [cluster1]                                 "type": "string"
2021-07-16T18:15:02.7550629Z [cluster1]                             },
2021-07-16T18:15:02.7551162Z [cluster1]                             "metadata": {
2021-07-16T18:15:02.7551719Z [cluster1]                                 "type": "object"
2021-07-16T18:15:02.7552452Z [cluster1]                             },
2021-07-16T18:15:02.7552942Z [cluster1]                             "spec": {
2021-07-16T18:15:02.7553791Z [cluster1]                                 "description": "SubmarinerSpec defines the desired state of Submariner",
2021-07-16T18:15:02.7554661Z [cluster1]                                 "properties": {
2021-07-16T18:15:02.7555248Z [cluster1]                                     "broker": {
2021-07-16T18:15:02.7556328Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7556856Z [cluster1]                                     },
2021-07-16T18:15:02.7557490Z [cluster1]                                     "brokerK8sApiServer": {
2021-07-16T18:15:02.7558170Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7558700Z [cluster1]                                     },
2021-07-16T18:15:02.7559383Z [cluster1]                                     "brokerK8sApiServerToken": {
2021-07-16T18:15:02.7600460Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7601018Z [cluster1]                                     },
2021-07-16T18:15:02.7601571Z [cluster1]                                     "brokerK8sCA": {
2021-07-16T18:15:02.7602184Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7602830Z [cluster1]                                     },
2021-07-16T18:15:02.7603738Z [cluster1]                                     "brokerK8sRemoteNamespace": {
2021-07-16T18:15:02.7604692Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7605225Z [cluster1]                                     },
2021-07-16T18:15:02.7605763Z [cluster1]                                     "cableDriver": {
2021-07-16T18:15:02.7606378Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7606906Z [cluster1]                                     },
2021-07-16T18:15:02.7607559Z [cluster1]                                     "ceIPSecDebug": {
2021-07-16T18:15:02.7608190Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.7608719Z [cluster1]                                     },
2021-07-16T18:15:02.7609402Z [cluster1]                                     "ceIPSecForceUDPEncaps": {
2021-07-16T18:15:02.7610140Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.7610677Z [cluster1]                                     },
2021-07-16T18:15:02.7611262Z [cluster1]                                     "ceIPSecIKEPort": {
2021-07-16T18:15:02.7611923Z [cluster1]                                         "type": "integer"
2021-07-16T18:15:02.7612466Z [cluster1]                                     },
2021-07-16T18:15:02.7613049Z [cluster1]                                     "ceIPSecNATTPort": {
2021-07-16T18:15:02.7613712Z [cluster1]                                         "type": "integer"
2021-07-16T18:15:02.7614242Z [cluster1]                                     },
2021-07-16T18:15:02.7614790Z [cluster1]                                     "ceIPSecPSK": {
2021-07-16T18:15:02.7615383Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7615908Z [cluster1]                                     },
2021-07-16T18:15:02.7616591Z [cluster1]                                     "ceIPSecPreferredServer": {
2021-07-16T18:15:02.7617353Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.7617891Z [cluster1]                                     },
2021-07-16T18:15:02.7618430Z [cluster1]                                     "clusterCIDR": {
2021-07-16T18:15:02.7619033Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7619543Z [cluster1]                                     },
2021-07-16T18:15:02.7620070Z [cluster1]                                     "clusterID": {
2021-07-16T18:15:02.7620745Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7621370Z [cluster1]                                     },
2021-07-16T18:15:02.7621918Z [cluster1]                                     "colorCodes": {
2021-07-16T18:15:02.7622497Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7623019Z [cluster1]                                     },
2021-07-16T18:15:02.7623676Z [cluster1]                                     "connectionHealthCheck": {
2021-07-16T18:15:02.7624422Z [cluster1]                                         "properties": {
2021-07-16T18:15:02.7644451Z [cluster1]                                             "enabled": {
2021-07-16T18:15:02.7645122Z [cluster1]                                                 "type": "boolean"
2021-07-16T18:15:02.7645670Z [cluster1]                                             },
2021-07-16T18:15:02.7646292Z [cluster1]                                             "intervalSeconds": {
2021-07-16T18:15:02.7647305Z [cluster1]                                                 "description": "The interval at which health check pings are sent.",
2021-07-16T18:15:02.7648093Z [cluster1]                                                 "format": "int64",
2021-07-16T18:15:02.7648705Z [cluster1]                                                 "type": "integer"
2021-07-16T18:15:02.7649247Z [cluster1]                                             },
2021-07-16T18:15:02.7649898Z [cluster1]                                             "maxPacketLossCount": {
2021-07-16T18:15:02.7651127Z [cluster1]                                                 "description": "The maximum number of packets lost at which the health checker will mark the connection as down.",
2021-07-16T18:15:02.7652102Z [cluster1]                                                 "format": "int64",
2021-07-16T18:15:02.7652717Z [cluster1]                                                 "type": "integer"
2021-07-16T18:15:02.7653255Z [cluster1]                                             }
2021-07-16T18:15:02.7653754Z [cluster1]                                         },
2021-07-16T18:15:02.7654283Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.7654816Z [cluster1]                                     },
2021-07-16T18:15:02.7655458Z [cluster1]                                     "coreDNSCustomConfig": {
2021-07-16T18:15:02.7656159Z [cluster1]                                         "properties": {
2021-07-16T18:15:02.7656813Z [cluster1]                                             "configMapName": {
2021-07-16T18:15:02.7657447Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.7657992Z [cluster1]                                             },
2021-07-16T18:15:02.7658530Z [cluster1]                                             "namespace": {
2021-07-16T18:15:02.7659130Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.7659667Z [cluster1]                                             }
2021-07-16T18:15:02.7660146Z [cluster1]                                         },
2021-07-16T18:15:02.7660686Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.7661203Z [cluster1]                                     },
2021-07-16T18:15:02.7661774Z [cluster1]                                     "customDomains": {
2021-07-16T18:15:02.7662353Z [cluster1]                                         "items": {
2021-07-16T18:15:02.7662912Z [cluster1]                                             "type": "string"
2021-07-16T18:15:02.7663431Z [cluster1]                                         },
2021-07-16T18:15:02.7663965Z [cluster1]                                         "type": "array",
2021-07-16T18:15:02.7665311Z [cluster1]                                         "x-kubernetes-list-type": "set"
2021-07-16T18:15:02.7665982Z [cluster1]                                     },
2021-07-16T18:15:02.7666491Z [cluster1]                                     "debug": {
2021-07-16T18:15:02.7667146Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.7667695Z [cluster1]                                     },
2021-07-16T18:15:02.7668229Z [cluster1]                                     "globalCIDR": {
2021-07-16T18:15:02.7668829Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7669350Z [cluster1]                                     },
2021-07-16T18:15:02.7669933Z [cluster1]                                     "imageOverrides": {
2021-07-16T18:15:02.7670706Z [cluster1]                                         "additionalProperties": {
2021-07-16T18:15:02.7671420Z [cluster1]                                             "type": "string"
2021-07-16T18:15:02.7671958Z [cluster1]                                         },
2021-07-16T18:15:02.7672495Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.7673020Z [cluster1]                                     },
2021-07-16T18:15:02.7673649Z [cluster1]                                     "loadBalancerEnabled": {
2021-07-16T18:15:02.7674354Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.7674893Z [cluster1]                                     },
2021-07-16T18:15:02.7675419Z [cluster1]                                     "namespace": {
2021-07-16T18:15:02.7676012Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7676525Z [cluster1]                                     },
2021-07-16T18:15:02.7677159Z [cluster1]                                     "natEnabled": {
2021-07-16T18:15:02.7677753Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.7678292Z [cluster1]                                     },
2021-07-16T18:15:02.7678821Z [cluster1]                                     "repository": {
2021-07-16T18:15:02.7679414Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7679947Z [cluster1]                                     },
2021-07-16T18:15:02.7680490Z [cluster1]                                     "serviceCIDR": {
2021-07-16T18:15:02.7681104Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7682921Z [cluster1]                                     },
2021-07-16T18:15:02.7683813Z [cluster1]                                     "serviceDiscoveryEnabled": {
2021-07-16T18:15:02.7684577Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.7685123Z [cluster1]                                     },
2021-07-16T18:15:02.7685624Z [cluster1]                                     "version": {
2021-07-16T18:15:02.7686201Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7706146Z [cluster1]                                     }
2021-07-16T18:15:02.7706640Z [cluster1]                                 },
2021-07-16T18:15:02.7707177Z [cluster1]                                 "required": [
2021-07-16T18:15:02.7707741Z [cluster1]                                     "broker",
2021-07-16T18:15:02.7708402Z [cluster1]                                     "brokerK8sApiServer",
2021-07-16T18:15:02.7709219Z [cluster1]                                     "brokerK8sApiServerToken",
2021-07-16T18:15:02.7709975Z [cluster1]                                     "brokerK8sCA",
2021-07-16T18:15:02.7710731Z [cluster1]                                     "brokerK8sRemoteNamespace",
2021-07-16T18:15:02.7711525Z [cluster1]                                     "ceIPSecDebug",
2021-07-16T18:15:02.7712158Z [cluster1]                                     "ceIPSecPSK",
2021-07-16T18:15:02.7712755Z [cluster1]                                     "clusterCIDR",
2021-07-16T18:15:02.7713347Z [cluster1]                                     "clusterID",
2021-07-16T18:15:02.7713880Z [cluster1]                                     "debug",
2021-07-16T18:15:02.7714441Z [cluster1]                                     "namespace",
2021-07-16T18:15:02.7715210Z [cluster1]                                     "natEnabled",
2021-07-16T18:15:02.7716011Z [cluster1]                                     "serviceCIDR"
2021-07-16T18:15:02.7716542Z [cluster1]                                 ],
2021-07-16T18:15:02.7717079Z [cluster1]                                 "type": "object"
2021-07-16T18:15:02.7717593Z [cluster1]                             },
2021-07-16T18:15:02.7718076Z [cluster1]                             "status": {
2021-07-16T18:15:02.7718960Z [cluster1]                                 "description": "SubmarinerStatus defines the observed state of Submariner",
2021-07-16T18:15:02.7719851Z [cluster1]                                 "properties": {
2021-07-16T18:15:02.7720463Z [cluster1]                                     "clusterCIDR": {
2021-07-16T18:15:02.7721057Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7721589Z [cluster1]                                     },
2021-07-16T18:15:02.7722128Z [cluster1]                                     "clusterID": {
2021-07-16T18:15:02.7722701Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7723227Z [cluster1]                                     },
2021-07-16T18:15:02.7724106Z [cluster1]                                     "colorCodes": {
2021-07-16T18:15:02.7725576Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.7726320Z [cluster1]                                     },
2021-07-16T18:15:02.7726910Z [cluster1]                                     "deploymentInfo": {
2021-07-16T18:15:02.7727684Z [cluster1]                                         "properties": {
2021-07-16T18:15:02.7728343Z [cluster1]                                             "cloudProvider": {
2021-07-16T18:15:02.7729493Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.7730046Z [cluster1]                                             },
2021-07-16T18:15:02.7730661Z [cluster1]                                             "kubernetesType": {
2021-07-16T18:15:02.7731303Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.7731847Z [cluster1]                                             },
2021-07-16T18:15:02.7733487Z [cluster1]                                             "kubernetesTypeVersion": {
2021-07-16T18:15:02.7734392Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.7735090Z [cluster1]                                             },
2021-07-16T18:15:02.7735863Z [cluster1]                                             "kubernetesVersion": {
2021-07-16T18:15:02.7736695Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.7737370Z [cluster1]                                             }
2021-07-16T18:15:02.7738010Z [cluster1]                                         },
2021-07-16T18:15:02.7738685Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.7739362Z [cluster1]                                     },
2021-07-16T18:15:02.7740182Z [cluster1]                                     "gatewayDaemonSetStatus": {
2021-07-16T18:15:02.7741654Z [cluster1]                                         "properties": {
2021-07-16T18:15:02.7742544Z [cluster1]                                             "lastResourceVersion": {
2021-07-16T18:15:02.7743399Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.7744094Z [cluster1]                                             },
2021-07-16T18:15:02.7744983Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-16T18:15:02.7746064Z [cluster1]                                                 "type": "boolean"
2021-07-16T18:15:02.7747135Z [cluster1]                                             },
2021-07-16T18:15:02.7748110Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-16T18:15:02.7749330Z [cluster1]                                                 "items": {
2021-07-16T18:15:02.7751017Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-16T18:15:02.7752426Z [cluster1]                                                     "properties": {
2021-07-16T18:15:02.7753049Z [cluster1]                                                         "running": {
2021-07-16T18:15:02.7753815Z [cluster1]                                                             "description": "Details about a running container",
2021-07-16T18:15:02.7754602Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.7755235Z [cluster1]                                                                 "startedAt": {
2021-07-16T18:15:02.7764209Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-16T18:15:02.7765551Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.7766203Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7766791Z [cluster1]                                                                 }
2021-07-16T18:15:02.7767310Z [cluster1]                                                             },
2021-07-16T18:15:02.7768067Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.7768639Z [cluster1]                                                         },
2021-07-16T18:15:02.7769205Z [cluster1]                                                         "terminated": {
2021-07-16T18:15:02.7770027Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-16T18:15:02.7770823Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.7771493Z [cluster1]                                                                 "containerID": {
2021-07-16T18:15:02.7772706Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-16T18:15:02.7773585Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7774167Z [cluster1]                                                                 },
2021-07-16T18:15:02.7774739Z [cluster1]                                                                 "exitCode": {
2021-07-16T18:15:02.7775604Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-16T18:15:02.7776455Z [cluster1]                                                                     "format": "int32",
2021-07-16T18:15:02.7777079Z [cluster1]                                                                     "type": "integer"
2021-07-16T18:15:02.7777665Z [cluster1]                                                                 },
2021-07-16T18:15:02.7778237Z [cluster1]                                                                 "finishedAt": {
2021-07-16T18:15:02.7779075Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-16T18:15:02.7780158Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.7780801Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7781372Z [cluster1]                                                                 },
2021-07-16T18:15:02.7782009Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.7782909Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-16T18:15:02.7783778Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7784334Z [cluster1]                                                                 },
2021-07-16T18:15:02.7784889Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.7785731Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-16T18:15:02.7786582Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7787155Z [cluster1]                                                                 },
2021-07-16T18:15:02.7787700Z [cluster1]                                                                 "signal": {
2021-07-16T18:15:02.7788529Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-16T18:15:02.7789343Z [cluster1]                                                                     "format": "int32",
2021-07-16T18:15:02.7789965Z [cluster1]                                                                     "type": "integer"
2021-07-16T18:15:02.7790593Z [cluster1]                                                                 },
2021-07-16T18:15:02.7791168Z [cluster1]                                                                 "startedAt": {
2021-07-16T18:15:02.7792061Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-16T18:15:02.7793189Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.7793850Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7794411Z [cluster1]                                                                 }
2021-07-16T18:15:02.7794938Z [cluster1]                                                             },
2021-07-16T18:15:02.7795506Z [cluster1]                                                             "required": [
2021-07-16T18:15:02.7796116Z [cluster1]                                                                 "exitCode"
2021-07-16T18:15:02.7796676Z [cluster1]                                                             ],
2021-07-16T18:15:02.7798583Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.7799215Z [cluster1]                                                         },
2021-07-16T18:15:02.7799759Z [cluster1]                                                         "waiting": {
2021-07-16T18:15:02.7800540Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-16T18:15:02.7802053Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.7802673Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.7804353Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-16T18:15:02.7805227Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7805805Z [cluster1]                                                                 },
2021-07-16T18:15:02.7806374Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.7807308Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-16T18:15:02.7808129Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7808693Z [cluster1]                                                                 }
2021-07-16T18:15:02.7809230Z [cluster1]                                                             },
2021-07-16T18:15:02.7809790Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.7810364Z [cluster1]                                                         }
2021-07-16T18:15:02.7810872Z [cluster1]                                                     },
2021-07-16T18:15:02.7811415Z [cluster1]                                                     "type": "object"
2021-07-16T18:15:02.7811966Z [cluster1]                                                 },
2021-07-16T18:15:02.7812964Z [cluster1]                                                 "type": "array"
2021-07-16T18:15:02.7813535Z [cluster1]                                             },
2021-07-16T18:15:02.7814047Z [cluster1]                                             "status": {
2021-07-16T18:15:02.7814954Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-16T18:15:02.7815877Z [cluster1]                                                 "properties": {
2021-07-16T18:15:02.7816532Z [cluster1]                                                     "collisionCount": {
2021-07-16T18:15:02.7818125Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-16T18:15:02.7819539Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.7820151Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.7820726Z [cluster1]                                                     },
2021-07-16T18:15:02.7821280Z [cluster1]                                                     "conditions": {
2021-07-16T18:15:02.7822731Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-16T18:15:02.7823682Z [cluster1]                                                         "items": {
2021-07-16T18:15:02.7824643Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-16T18:15:02.7825645Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.7826368Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-16T18:15:02.7827379Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-16T18:15:02.7828536Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.7829182Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7829761Z [cluster1]                                                                 },
2021-07-16T18:15:02.7830356Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.7831331Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-16T18:15:02.7832385Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7833051Z [cluster1]                                                                 },
2021-07-16T18:15:02.7833706Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.7834830Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-16T18:15:02.7835726Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7836399Z [cluster1]                                                                 },
2021-07-16T18:15:02.7836983Z [cluster1]                                                                 "status": {
2021-07-16T18:15:02.7837912Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-16T18:15:02.7838771Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7839421Z [cluster1]                                                                 },
2021-07-16T18:15:02.7957039Z [cluster1]                                                                 "type": {
2021-07-16T18:15:02.7957824Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-16T18:15:02.7958586Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.7959168Z [cluster1]                                                                 }
2021-07-16T18:15:02.7959876Z [cluster1]                                                             },
2021-07-16T18:15:02.7960446Z [cluster1]                                                             "required": [
2021-07-16T18:15:02.7967435Z [cluster1]                                                                 "status",
2021-07-16T18:15:02.7968035Z [cluster1]                                                                 "type"
2021-07-16T18:15:02.7971747Z [cluster1]                                                             ],
2021-07-16T18:15:02.7972581Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.7973312Z [cluster1]                                                         },
2021-07-16T18:15:02.7974013Z [cluster1]                                                         "type": "array"
2021-07-16T18:15:02.7974588Z [cluster1]                                                     },
2021-07-16T18:15:02.7975299Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-16T18:15:02.7976882Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.7983372Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.7984021Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.7984808Z [cluster1]                                                     },
2021-07-16T18:15:02.7985695Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-16T18:15:02.7987778Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.7989262Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.7989886Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.7990441Z [cluster1]                                                     },
2021-07-16T18:15:02.7991243Z [cluster1]                                                     "numberAvailable": {
2021-07-16T18:15:02.7992578Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.7993850Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.7994469Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.7995023Z [cluster1]                                                     },
2021-07-16T18:15:02.7995675Z [cluster1]                                                     "numberMisscheduled": {
2021-07-16T18:15:02.8020459Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8055720Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8056379Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8056947Z [cluster1]                                                     },
2021-07-16T18:15:02.8057537Z [cluster1]                                                     "numberReady": {
2021-07-16T18:15:02.8059229Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-16T18:15:02.8060242Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8060866Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8061433Z [cluster1]                                                     },
2021-07-16T18:15:02.8062082Z [cluster1]                                                     "numberUnavailable": {
2021-07-16T18:15:02.8063820Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.8065079Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8065701Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8066255Z [cluster1]                                                     },
2021-07-16T18:15:02.8066917Z [cluster1]                                                     "observedGeneration": {
2021-07-16T18:15:02.8067914Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-16T18:15:02.8068815Z [cluster1]                                                         "format": "int64",
2021-07-16T18:15:02.8069422Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8069993Z [cluster1]                                                     },
2021-07-16T18:15:02.8070702Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-16T18:15:02.8071702Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-16T18:15:02.8075232Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8075857Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8076434Z [cluster1]                                                     }
2021-07-16T18:15:02.8077121Z [cluster1]                                                 },
2021-07-16T18:15:02.8077844Z [cluster1]                                                 "required": [
2021-07-16T18:15:02.8078601Z [cluster1]                                                     "currentNumberScheduled",
2021-07-16T18:15:02.8079494Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-16T18:15:02.8080340Z [cluster1]                                                     "numberMisscheduled",
2021-07-16T18:15:02.8081038Z [cluster1]                                                     "numberReady"
2021-07-16T18:15:02.8081624Z [cluster1]                                                 ],
2021-07-16T18:15:02.8082178Z [cluster1]                                                 "type": "object"
2021-07-16T18:15:02.8082711Z [cluster1]                                             }
2021-07-16T18:15:02.8083198Z [cluster1]                                         },
2021-07-16T18:15:02.8083872Z [cluster1]                                         "required": [
2021-07-16T18:15:02.8084670Z [cluster1]                                             "mismatchedContainerImages"
2021-07-16T18:15:02.8085389Z [cluster1]                                         ],
2021-07-16T18:15:02.8085936Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.8086453Z [cluster1]                                     },
2021-07-16T18:15:02.8086984Z [cluster1]                                     "gateways": {
2021-07-16T18:15:02.8087546Z [cluster1]                                         "items": {
2021-07-16T18:15:02.8088223Z [cluster1]                                             "properties": {
2021-07-16T18:15:02.8088860Z [cluster1]                                                 "connections": {
2021-07-16T18:15:02.8089444Z [cluster1]                                                     "items": {
2021-07-16T18:15:02.8090053Z [cluster1]                                                         "properties": {
2021-07-16T18:15:02.8090681Z [cluster1]                                                             "endpoint": {
2021-07-16T18:15:02.8091328Z [cluster1]                                                                 "properties": {
2021-07-16T18:15:02.8091966Z [cluster1]                                                                     "backend": {
2021-07-16T18:15:02.8092580Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8093160Z [cluster1]                                                                     },
2021-07-16T18:15:02.8093745Z [cluster1]                                                                     "backend_config": {
2021-07-16T18:15:02.8094520Z [cluster1]                                                                         "additionalProperties": {
2021-07-16T18:15:02.8096090Z [cluster1]                                                                             "type": "string"
2021-07-16T18:15:02.8096768Z [cluster1]                                                                         },
2021-07-16T18:15:02.8097365Z [cluster1]                                                                         "type": "object"
2021-07-16T18:15:02.8097929Z [cluster1]                                                                     },
2021-07-16T18:15:02.8098509Z [cluster1]                                                                     "cable_name": {
2021-07-16T18:15:02.8099122Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8099737Z [cluster1]                                                                     },
2021-07-16T18:15:02.8100318Z [cluster1]                                                                     "cluster_id": {
2021-07-16T18:15:02.8100934Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8101511Z [cluster1]                                                                     },
2021-07-16T18:15:02.8102225Z [cluster1]                                                                     "healthCheckIP": {
2021-07-16T18:15:02.8102912Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8103489Z [cluster1]                                                                     },
2021-07-16T18:15:02.8104056Z [cluster1]                                                                     "hostname": {
2021-07-16T18:15:02.8104681Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8105246Z [cluster1]                                                                     },
2021-07-16T18:15:02.8106261Z [cluster1]                                                                     "nat_enabled": {
2021-07-16T18:15:02.8106911Z [cluster1]                                                                         "type": "boolean"
2021-07-16T18:15:02.8107489Z [cluster1]                                                                     },
2021-07-16T18:15:02.8108072Z [cluster1]                                                                     "private_ip": {
2021-07-16T18:15:02.8108687Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8109260Z [cluster1]                                                                     },
2021-07-16T18:15:02.8109820Z [cluster1]                                                                     "public_ip": {
2021-07-16T18:15:02.8110970Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8111552Z [cluster1]                                                                     },
2021-07-16T18:15:02.8112103Z [cluster1]                                                                     "subnets": {
2021-07-16T18:15:02.8112695Z [cluster1]                                                                         "items": {
2021-07-16T18:15:02.8113303Z [cluster1]                                                                             "type": "string"
2021-07-16T18:15:02.8113886Z [cluster1]                                                                         },
2021-07-16T18:15:02.8114461Z [cluster1]                                                                         "type": "array"
2021-07-16T18:15:02.8115023Z [cluster1]                                                                     }
2021-07-16T18:15:02.8115558Z [cluster1]                                                                 },
2021-07-16T18:15:02.8116121Z [cluster1]                                                                 "required": [
2021-07-16T18:15:02.8117139Z [cluster1]                                                                     "backend",
2021-07-16T18:15:02.8117911Z [cluster1]                                                                     "cable_name",
2021-07-16T18:15:02.8118692Z [cluster1]                                                                     "cluster_id",
2021-07-16T18:15:02.8119473Z [cluster1]                                                                     "hostname",
2021-07-16T18:15:02.8120218Z [cluster1]                                                                     "nat_enabled",
2021-07-16T18:15:02.8120848Z [cluster1]                                                                     "private_ip",
2021-07-16T18:15:02.8121444Z [cluster1]                                                                     "public_ip",
2021-07-16T18:15:02.8122051Z [cluster1]                                                                     "subnets"
2021-07-16T18:15:02.8122600Z [cluster1]                                                                 ],
2021-07-16T18:15:02.8123179Z [cluster1]                                                                 "type": "object"
2021-07-16T18:15:02.8123905Z [cluster1]                                                             },
2021-07-16T18:15:02.8124592Z [cluster1]                                                             "latencyRTT": {
2021-07-16T18:15:02.8125715Z [cluster1]                                                                 "description": "LatencySpec describes the round trip time information for a packet between the gateway pods of two clusters.",
2021-07-16T18:15:02.8126809Z [cluster1]                                                                 "properties": {
2021-07-16T18:15:02.8127444Z [cluster1]                                                                     "average": {
2021-07-16T18:15:02.8128070Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8128631Z [cluster1]                                                                     },
2021-07-16T18:15:02.8129534Z [cluster1]                                                                     "last": {
2021-07-16T18:15:02.8130128Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8130700Z [cluster1]                                                                     },
2021-07-16T18:15:02.8131229Z [cluster1]                                                                     "max": {
2021-07-16T18:15:02.8131829Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8132403Z [cluster1]                                                                     },
2021-07-16T18:15:02.8133020Z [cluster1]                                                                     "min": {
2021-07-16T18:15:02.8133613Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8134173Z [cluster1]                                                                     },
2021-07-16T18:15:02.8134736Z [cluster1]                                                                     "stdDev": {
2021-07-16T18:15:02.8135359Z [cluster1]                                                                         "type": "string"
2021-07-16T18:15:02.8135918Z [cluster1]                                                                     }
2021-07-16T18:15:02.8136447Z [cluster1]                                                                 },
2021-07-16T18:15:02.8137009Z [cluster1]                                                                 "type": "object"
2021-07-16T18:15:02.8138132Z [cluster1]                                                             },
2021-07-16T18:15:02.8138679Z [cluster1]                                                             "status": {
2021-07-16T18:15:02.8139278Z [cluster1]                                                                 "type": "string"
2021-07-16T18:15:02.8139851Z [cluster1]                                                             },
2021-07-16T18:15:02.8142179Z [cluster1]                                                             "statusMessage": {
2021-07-16T18:15:02.8143036Z [cluster1]                                                                 "type": "string"
2021-07-16T18:15:02.8143596Z [cluster1]                                                             },
2021-07-16T18:15:02.8144159Z [cluster1]                                                             "usingIP": {
2021-07-16T18:15:02.8145254Z [cluster1]                                                                 "type": "string"
2021-07-16T18:15:02.8145820Z [cluster1]                                                             },
2021-07-16T18:15:02.8146400Z [cluster1]                                                             "usingNAT": {
2021-07-16T18:15:02.8147010Z [cluster1]                                                                 "type": "boolean"
2021-07-16T18:15:02.8147585Z [cluster1]                                                             }
2021-07-16T18:15:02.8148089Z [cluster1]                                                         },
2021-07-16T18:15:02.8148759Z [cluster1]                                                         "required": [
2021-07-16T18:15:02.8149361Z [cluster1]                                                             "endpoint",
2021-07-16T18:15:02.8149949Z [cluster1]                                                             "status",
2021-07-16T18:15:02.8150584Z [cluster1]                                                             "statusMessage"
2021-07-16T18:15:02.8151171Z [cluster1]                                                         ],
2021-07-16T18:15:02.8151734Z [cluster1]                                                         "type": "object"
2021-07-16T18:15:02.8152277Z [cluster1]                                                     },
2021-07-16T18:15:02.8152823Z [cluster1]                                                     "type": "array"
2021-07-16T18:15:02.8153349Z [cluster1]                                                 },
2021-07-16T18:15:02.8153897Z [cluster1]                                                 "haStatus": {
2021-07-16T18:15:02.8154498Z [cluster1]                                                     "type": "string"
2021-07-16T18:15:02.8155035Z [cluster1]                                                 },
2021-07-16T18:15:02.8155628Z [cluster1]                                                 "localEndpoint": {
2021-07-16T18:15:02.8156507Z [cluster1]                                                     "properties": {
2021-07-16T18:15:02.8157122Z [cluster1]                                                         "backend": {
2021-07-16T18:15:02.8157714Z [cluster1]                                                             "type": "string"
2021-07-16T18:15:02.8158357Z [cluster1]                                                         },
2021-07-16T18:15:02.8158929Z [cluster1]                                                         "backend_config": {
2021-07-16T18:15:02.8160069Z [cluster1]                                                             "additionalProperties": {
2021-07-16T18:15:02.8160832Z [cluster1]                                                                 "type": "string"
2021-07-16T18:15:02.8161398Z [cluster1]                                                             },
2021-07-16T18:15:02.8161965Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8162514Z [cluster1]                                                         },
2021-07-16T18:15:02.8163070Z [cluster1]                                                         "cable_name": {
2021-07-16T18:15:02.8165915Z [cluster1]                                                             "type": "string"
2021-07-16T18:15:02.8166487Z [cluster1]                                                         },
2021-07-16T18:15:02.8167667Z [cluster1]                                                         "cluster_id": {
2021-07-16T18:15:02.8168289Z [cluster1]                                                             "type": "string"
2021-07-16T18:15:02.8176556Z [cluster1]                                                         },
2021-07-16T18:15:02.8177590Z [cluster1]                                                         "healthCheckIP": {
2021-07-16T18:15:02.8178754Z [cluster1]                                                             "type": "string"
2021-07-16T18:15:02.8179327Z [cluster1]                                                         },
2021-07-16T18:15:02.8179873Z [cluster1]                                                         "hostname": {
2021-07-16T18:15:02.8180486Z [cluster1]                                                             "type": "string"
2021-07-16T18:15:02.8187389Z [cluster1]                                                         },
2021-07-16T18:15:02.8188030Z [cluster1]                                                         "nat_enabled": {
2021-07-16T18:15:02.8188653Z [cluster1]                                                             "type": "boolean"
2021-07-16T18:15:02.8189233Z [cluster1]                                                         },
2021-07-16T18:15:02.8189982Z [cluster1]                                                         "private_ip": {
2021-07-16T18:15:02.8190594Z [cluster1]                                                             "type": "string"
2021-07-16T18:15:02.8191163Z [cluster1]                                                         },
2021-07-16T18:15:02.8191700Z [cluster1]                                                         "public_ip": {
2021-07-16T18:15:02.8192306Z [cluster1]                                                             "type": "string"
2021-07-16T18:15:02.8192859Z [cluster1]                                                         },
2021-07-16T18:15:02.8193408Z [cluster1]                                                         "subnets": {
2021-07-16T18:15:02.8193985Z [cluster1]                                                             "items": {
2021-07-16T18:15:02.8194568Z [cluster1]                                                                 "type": "string"
2021-07-16T18:15:02.8195127Z [cluster1]                                                             },
2021-07-16T18:15:02.8195676Z [cluster1]                                                             "type": "array"
2021-07-16T18:15:02.8196229Z [cluster1]                                                         }
2021-07-16T18:15:02.8196724Z [cluster1]                                                     },
2021-07-16T18:15:02.8197276Z [cluster1]                                                     "required": [
2021-07-16T18:15:02.8197889Z [cluster1]                                                         "backend",
2021-07-16T18:15:02.8198547Z [cluster1]                                                         "cable_name",
2021-07-16T18:15:02.8199137Z [cluster1]                                                         "cluster_id",
2021-07-16T18:15:02.8199780Z [cluster1]                                                         "hostname",
2021-07-16T18:15:02.8200377Z [cluster1]                                                         "nat_enabled",
2021-07-16T18:15:02.8200969Z [cluster1]                                                         "private_ip",
2021-07-16T18:15:02.8201566Z [cluster1]                                                         "public_ip",
2021-07-16T18:15:02.8202153Z [cluster1]                                                         "subnets"
2021-07-16T18:15:02.8202679Z [cluster1]                                                     ],
2021-07-16T18:15:02.8203441Z [cluster1]                                                     "type": "object"
2021-07-16T18:15:02.8204297Z [cluster1]                                                 },
2021-07-16T18:15:02.8204976Z [cluster1]                                                 "statusFailure": {
2021-07-16T18:15:02.8205605Z [cluster1]                                                     "type": "string"
2021-07-16T18:15:02.8206150Z [cluster1]                                                 },
2021-07-16T18:15:02.8206688Z [cluster1]                                                 "version": {
2021-07-16T18:15:02.8207264Z [cluster1]                                                     "type": "string"
2021-07-16T18:15:02.8207811Z [cluster1]                                                 }
2021-07-16T18:15:02.8208297Z [cluster1]                                             },
2021-07-16T18:15:02.8208841Z [cluster1]                                             "required": [
2021-07-16T18:15:02.8209442Z [cluster1]                                                 "connections",
2021-07-16T18:15:02.8210041Z [cluster1]                                                 "haStatus",
2021-07-16T18:15:02.8210670Z [cluster1]                                                 "localEndpoint",
2021-07-16T18:15:02.8211585Z [cluster1]                                                 "statusFailure",
2021-07-16T18:15:02.8212798Z [cluster1]                                                 "version"
2021-07-16T18:15:02.8213328Z [cluster1]                                             ],
2021-07-16T18:15:02.8213872Z [cluster1]                                             "type": "object"
2021-07-16T18:15:02.8214498Z [cluster1]                                         },
2021-07-16T18:15:02.8215040Z [cluster1]                                         "type": "array"
2021-07-16T18:15:02.8215788Z [cluster1]                                     },
2021-07-16T18:15:02.8216500Z [cluster1]                                     "globalCIDR": {
2021-07-16T18:15:02.8217102Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.8217612Z [cluster1]                                     },
2021-07-16T18:15:02.8218343Z [cluster1]                                     "globalnetDaemonSetStatus": {
2021-07-16T18:15:02.8219122Z [cluster1]                                         "properties": {
2021-07-16T18:15:02.8219848Z [cluster1]                                             "lastResourceVersion": {
2021-07-16T18:15:02.8220720Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.8221418Z [cluster1]                                             },
2021-07-16T18:15:02.8222166Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-16T18:15:02.8222965Z [cluster1]                                                 "type": "boolean"
2021-07-16T18:15:02.8223516Z [cluster1]                                             },
2021-07-16T18:15:02.8224212Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-16T18:15:02.8224963Z [cluster1]                                                 "items": {
2021-07-16T18:15:02.8226415Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-16T18:15:02.8227820Z [cluster1]                                                     "properties": {
2021-07-16T18:15:02.8228430Z [cluster1]                                                         "running": {
2021-07-16T18:15:02.8229389Z [cluster1]                                                             "description": "Details about a running container",
2021-07-16T18:15:02.8230340Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8230967Z [cluster1]                                                                 "startedAt": {
2021-07-16T18:15:02.8232509Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-16T18:15:02.8233590Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8234231Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8234804Z [cluster1]                                                                 }
2021-07-16T18:15:02.8235314Z [cluster1]                                                             },
2021-07-16T18:15:02.8235893Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8236453Z [cluster1]                                                         },
2021-07-16T18:15:02.8237036Z [cluster1]                                                         "terminated": {
2021-07-16T18:15:02.8237848Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-16T18:15:02.8238638Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8239303Z [cluster1]                                                                 "containerID": {
2021-07-16T18:15:02.8240480Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-16T18:15:02.8241454Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8242034Z [cluster1]                                                                 },
2021-07-16T18:15:02.8242604Z [cluster1]                                                                 "exitCode": {
2021-07-16T18:15:02.8244518Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-16T18:15:02.8245378Z [cluster1]                                                                     "format": "int32",
2021-07-16T18:15:02.8246019Z [cluster1]                                                                     "type": "integer"
2021-07-16T18:15:02.8246606Z [cluster1]                                                                 },
2021-07-16T18:15:02.8247182Z [cluster1]                                                                 "finishedAt": {
2021-07-16T18:15:02.8248028Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-16T18:15:02.8249181Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8249834Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8250390Z [cluster1]                                                                 },
2021-07-16T18:15:02.8250965Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.8251965Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-16T18:15:02.8252827Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8253396Z [cluster1]                                                                 },
2021-07-16T18:15:02.8253941Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.8254798Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-16T18:15:02.8255647Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8256202Z [cluster1]                                                                 },
2021-07-16T18:15:02.8256756Z [cluster1]                                                                 "signal": {
2021-07-16T18:15:02.8257570Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-16T18:15:02.8258404Z [cluster1]                                                                     "format": "int32",
2021-07-16T18:15:02.8259033Z [cluster1]                                                                     "type": "integer"
2021-07-16T18:15:02.8259606Z [cluster1]                                                                 },
2021-07-16T18:15:02.8260181Z [cluster1]                                                                 "startedAt": {
2021-07-16T18:15:02.8261065Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-16T18:15:02.8262198Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8262856Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8263414Z [cluster1]                                                                 }
2021-07-16T18:15:02.8263940Z [cluster1]                                                             },
2021-07-16T18:15:02.8264494Z [cluster1]                                                             "required": [
2021-07-16T18:15:02.8265188Z [cluster1]                                                                 "exitCode"
2021-07-16T18:15:02.8265744Z [cluster1]                                                             ],
2021-07-16T18:15:02.8266315Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8266873Z [cluster1]                                                         },
2021-07-16T18:15:02.8267401Z [cluster1]                                                         "waiting": {
2021-07-16T18:15:02.8268170Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-16T18:15:02.8268944Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8269570Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.8270418Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-16T18:15:02.8271268Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8271840Z [cluster1]                                                                 },
2021-07-16T18:15:02.8272380Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.8273186Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-16T18:15:02.8274037Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8274607Z [cluster1]                                                                 }
2021-07-16T18:15:02.8275125Z [cluster1]                                                             },
2021-07-16T18:15:02.8275685Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8276249Z [cluster1]                                                         }
2021-07-16T18:15:02.8276744Z [cluster1]                                                     },
2021-07-16T18:15:02.8277305Z [cluster1]                                                     "type": "object"
2021-07-16T18:15:02.8277839Z [cluster1]                                                 },
2021-07-16T18:15:02.8278387Z [cluster1]                                                 "type": "array"
2021-07-16T18:15:02.8278923Z [cluster1]                                             },
2021-07-16T18:15:02.8279432Z [cluster1]                                             "status": {
2021-07-16T18:15:02.8280332Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-16T18:15:02.8281232Z [cluster1]                                                 "properties": {
2021-07-16T18:15:02.8281900Z [cluster1]                                                     "collisionCount": {
2021-07-16T18:15:02.8283384Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-16T18:15:02.8285111Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8285729Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8286290Z [cluster1]                                                     },
2021-07-16T18:15:02.8286855Z [cluster1]                                                     "conditions": {
2021-07-16T18:15:02.8288168Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-16T18:15:02.8289227Z [cluster1]                                                         "items": {
2021-07-16T18:15:02.8290205Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-16T18:15:02.8291185Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8291914Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-16T18:15:02.8292898Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-16T18:15:02.8294054Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8294707Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8295260Z [cluster1]                                                                 },
2021-07-16T18:15:02.8295833Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.8296735Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-16T18:15:02.8297660Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8298232Z [cluster1]                                                                 },
2021-07-16T18:15:02.8298871Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.8299997Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-16T18:15:02.8300806Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8301383Z [cluster1]                                                                 },
2021-07-16T18:15:02.8301933Z [cluster1]                                                                 "status": {
2021-07-16T18:15:02.8302752Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-16T18:15:02.8303591Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8304147Z [cluster1]                                                                 },
2021-07-16T18:15:02.8304700Z [cluster1]                                                                 "type": {
2021-07-16T18:15:02.8305441Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-16T18:15:02.8306203Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8306779Z [cluster1]                                                                 }
2021-07-16T18:15:02.8307287Z [cluster1]                                                             },
2021-07-16T18:15:02.8307853Z [cluster1]                                                             "required": [
2021-07-16T18:15:02.8308430Z [cluster1]                                                                 "status",
2021-07-16T18:15:02.8309003Z [cluster1]                                                                 "type"
2021-07-16T18:15:02.8309521Z [cluster1]                                                             ],
2021-07-16T18:15:02.8310099Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8310658Z [cluster1]                                                         },
2021-07-16T18:15:02.8311199Z [cluster1]                                                         "type": "array"
2021-07-16T18:15:02.8311744Z [cluster1]                                                     },
2021-07-16T18:15:02.8312550Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-16T18:15:02.8314137Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8316030Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8316661Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8317226Z [cluster1]                                                     },
2021-07-16T18:15:02.8317928Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-16T18:15:02.8319571Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8321038Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8321650Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8322211Z [cluster1]                                                     },
2021-07-16T18:15:02.8322818Z [cluster1]                                                     "numberAvailable": {
2021-07-16T18:15:02.8324550Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.8325818Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8326421Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8326987Z [cluster1]                                                     },
2021-07-16T18:15:02.8327619Z [cluster1]                                                     "numberMisscheduled": {
2021-07-16T18:15:02.8329114Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8330483Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8331089Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8331651Z [cluster1]                                                     },
2021-07-16T18:15:02.8332211Z [cluster1]                                                     "numberReady": {
2021-07-16T18:15:02.8333265Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-16T18:15:02.8334274Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8334878Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8335448Z [cluster1]                                                     },
2021-07-16T18:15:02.8336074Z [cluster1]                                                     "numberUnavailable": {
2021-07-16T18:15:02.8337408Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.8338648Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8339353Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8339920Z [cluster1]                                                     },
2021-07-16T18:15:02.8340560Z [cluster1]                                                     "observedGeneration": {
2021-07-16T18:15:02.8341567Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-16T18:15:02.8342448Z [cluster1]                                                         "format": "int64",
2021-07-16T18:15:02.8343759Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8344333Z [cluster1]                                                     },
2021-07-16T18:15:02.8345037Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-16T18:15:02.8346054Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-16T18:15:02.8346877Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8347485Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8348049Z [cluster1]                                                     }
2021-07-16T18:15:02.8348544Z [cluster1]                                                 },
2021-07-16T18:15:02.8349245Z [cluster1]                                                 "required": [
2021-07-16T18:15:02.8349982Z [cluster1]                                                     "currentNumberScheduled",
2021-07-16T18:15:02.8350878Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-16T18:15:02.8351702Z [cluster1]                                                     "numberMisscheduled",
2021-07-16T18:15:02.8352412Z [cluster1]                                                     "numberReady"
2021-07-16T18:15:02.8352981Z [cluster1]                                                 ],
2021-07-16T18:15:02.8353518Z [cluster1]                                                 "type": "object"
2021-07-16T18:15:02.8354068Z [cluster1]                                             }
2021-07-16T18:15:02.8354549Z [cluster1]                                         },
2021-07-16T18:15:02.8355086Z [cluster1]                                         "required": [
2021-07-16T18:15:02.8355855Z [cluster1]                                             "mismatchedContainerImages"
2021-07-16T18:15:02.8356584Z [cluster1]                                         ],
2021-07-16T18:15:02.8357114Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.8357639Z [cluster1]                                     },
2021-07-16T18:15:02.8358264Z [cluster1]                                     "loadBalancerStatus": {
2021-07-16T18:15:02.8358954Z [cluster1]                                         "properties": {
2021-07-16T18:15:02.8359541Z [cluster1]                                             "status": {
2021-07-16T18:15:02.8362085Z [cluster1]                                                 "description": "LoadBalancerStatus represents the status of a load-balancer.",
2021-07-16T18:15:02.8363063Z [cluster1]                                                 "properties": {
2021-07-16T18:15:02.8363756Z [cluster1]                                                     "ingress": {
2021-07-16T18:15:02.8365313Z [cluster1]                                                         "description": "Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points.",
2021-07-16T18:15:02.8366479Z [cluster1]                                                         "items": {
2021-07-16T18:15:02.8368222Z [cluster1]                                                             "description": "LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.",
2021-07-16T18:15:02.8369536Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8370167Z [cluster1]                                                                 "hostname": {
2021-07-16T18:15:02.8371543Z [cluster1]                                                                     "description": "Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers)",
2021-07-16T18:15:02.8372614Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8373172Z [cluster1]                                                                 },
2021-07-16T18:15:02.8373713Z [cluster1]                                                                 "ip": {
2021-07-16T18:15:02.8375107Z [cluster1]                                                                     "description": "IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers)",
2021-07-16T18:15:02.8376186Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8376755Z [cluster1]                                                                 }
2021-07-16T18:15:02.8377267Z [cluster1]                                                             },
2021-07-16T18:15:02.8377844Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8378495Z [cluster1]                                                         },
2021-07-16T18:15:02.8379056Z [cluster1]                                                         "type": "array"
2021-07-16T18:15:02.8379605Z [cluster1]                                                     }
2021-07-16T18:15:02.8380095Z [cluster1]                                                 },
2021-07-16T18:15:02.8380650Z [cluster1]                                                 "type": "object"
2021-07-16T18:15:02.8381179Z [cluster1]                                             }
2021-07-16T18:15:02.8381665Z [cluster1]                                         },
2021-07-16T18:15:02.8382188Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.8382717Z [cluster1]                                     },
2021-07-16T18:15:02.8383242Z [cluster1]                                     "natEnabled": {
2021-07-16T18:15:02.8383848Z [cluster1]                                         "type": "boolean"
2021-07-16T18:15:02.8384389Z [cluster1]                                     },
2021-07-16T18:15:02.8384941Z [cluster1]                                     "networkPlugin": {
2021-07-16T18:15:02.8385571Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.8386078Z [cluster1]                                     },
2021-07-16T18:15:02.8386816Z [cluster1]                                     "routeAgentDaemonSetStatus": {
2021-07-16T18:15:02.8387614Z [cluster1]                                         "properties": {
2021-07-16T18:15:02.8388337Z [cluster1]                                             "lastResourceVersion": {
2021-07-16T18:15:02.8389047Z [cluster1]                                                 "type": "string"
2021-07-16T18:15:02.8389602Z [cluster1]                                             },
2021-07-16T18:15:02.8390342Z [cluster1]                                             "mismatchedContainerImages": {
2021-07-16T18:15:02.8391140Z [cluster1]                                                 "type": "boolean"
2021-07-16T18:15:02.8391692Z [cluster1]                                             },
2021-07-16T18:15:02.8392391Z [cluster1]                                             "nonReadyContainerStates": {
2021-07-16T18:15:02.8393139Z [cluster1]                                                 "items": {
2021-07-16T18:15:02.8394586Z [cluster1]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-16T18:15:02.8395984Z [cluster1]                                                     "properties": {
2021-07-16T18:15:02.8396589Z [cluster1]                                                         "running": {
2021-07-16T18:15:02.8397332Z [cluster1]                                                             "description": "Details about a running container",
2021-07-16T18:15:02.8398114Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8398751Z [cluster1]                                                                 "startedAt": {
2021-07-16T18:15:02.8399914Z [cluster1]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-16T18:15:02.8401485Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8402147Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8402722Z [cluster1]                                                                 }
2021-07-16T18:15:02.8403238Z [cluster1]                                                             },
2021-07-16T18:15:02.8403985Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8404664Z [cluster1]                                                         },
2021-07-16T18:15:02.8405222Z [cluster1]                                                         "terminated": {
2021-07-16T18:15:02.8406029Z [cluster1]                                                             "description": "Details about a terminated container",
2021-07-16T18:15:02.8406822Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8407483Z [cluster1]                                                                 "containerID": {
2021-07-16T18:15:02.8409676Z [cluster1]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-16T18:15:02.8410727Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8411304Z [cluster1]                                                                 },
2021-07-16T18:15:02.8411877Z [cluster1]                                                                 "exitCode": {
2021-07-16T18:15:02.8412740Z [cluster1]                                                                     "description": "Exit status from the last termination of the container",
2021-07-16T18:15:02.8413583Z [cluster1]                                                                     "format": "int32",
2021-07-16T18:15:02.8414213Z [cluster1]                                                                     "type": "integer"
2021-07-16T18:15:02.8414793Z [cluster1]                                                                 },
2021-07-16T18:15:02.8415362Z [cluster1]                                                                 "finishedAt": {
2021-07-16T18:15:02.8416197Z [cluster1]                                                                     "description": "Time at which the container last terminated",
2021-07-16T18:15:02.8417291Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8417954Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8418527Z [cluster1]                                                                 },
2021-07-16T18:15:02.8419078Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.8420075Z [cluster1]                                                                     "description": "Message regarding the last termination of the container",
2021-07-16T18:15:02.8420946Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8421516Z [cluster1]                                                                 },
2021-07-16T18:15:02.8422066Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.8422902Z [cluster1]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-16T18:15:02.8423753Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8424308Z [cluster1]                                                                 },
2021-07-16T18:15:02.8424863Z [cluster1]                                                                 "signal": {
2021-07-16T18:15:02.8425686Z [cluster1]                                                                     "description": "Signal from the last termination of the container",
2021-07-16T18:15:02.8426500Z [cluster1]                                                                     "format": "int32",
2021-07-16T18:15:02.8427126Z [cluster1]                                                                     "type": "integer"
2021-07-16T18:15:02.8427695Z [cluster1]                                                                 },
2021-07-16T18:15:02.8428273Z [cluster1]                                                                 "startedAt": {
2021-07-16T18:15:02.8429222Z [cluster1]                                                                     "description": "Time at which previous execution of the container started",
2021-07-16T18:15:02.8430353Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8431009Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8431658Z [cluster1]                                                                 }
2021-07-16T18:15:02.8432184Z [cluster1]                                                             },
2021-07-16T18:15:02.8432739Z [cluster1]                                                             "required": [
2021-07-16T18:15:02.8433354Z [cluster1]                                                                 "exitCode"
2021-07-16T18:15:02.8433915Z [cluster1]                                                             ],
2021-07-16T18:15:02.8434473Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8435037Z [cluster1]                                                         },
2021-07-16T18:15:02.8435568Z [cluster1]                                                         "waiting": {
2021-07-16T18:15:02.8436334Z [cluster1]                                                             "description": "Details about a waiting container",
2021-07-16T18:15:02.8437105Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8437723Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.8438581Z [cluster1]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-16T18:15:02.8439418Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8439991Z [cluster1]                                                                 },
2021-07-16T18:15:02.8440524Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.8441328Z [cluster1]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-16T18:15:02.8442203Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8442767Z [cluster1]                                                                 }
2021-07-16T18:15:02.8443285Z [cluster1]                                                             },
2021-07-16T18:15:02.8444602Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8445165Z [cluster1]                                                         }
2021-07-16T18:15:02.8445661Z [cluster1]                                                     },
2021-07-16T18:15:02.8446227Z [cluster1]                                                     "type": "object"
2021-07-16T18:15:02.8448006Z [cluster1]                                                 },
2021-07-16T18:15:02.8448721Z [cluster1]                                                 "type": "array"
2021-07-16T18:15:02.8449268Z [cluster1]                                             },
2021-07-16T18:15:02.8449777Z [cluster1]                                             "status": {
2021-07-16T18:15:02.8450667Z [cluster1]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-16T18:15:02.8451572Z [cluster1]                                                 "properties": {
2021-07-16T18:15:02.8452238Z [cluster1]                                                     "collisionCount": {
2021-07-16T18:15:02.8454135Z [cluster1]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-16T18:15:02.8455689Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8456554Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8457279Z [cluster1]                                                     },
2021-07-16T18:15:02.8457848Z [cluster1]                                                     "conditions": {
2021-07-16T18:15:02.8459284Z [cluster1]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-16T18:15:02.8460209Z [cluster1]                                                         "items": {
2021-07-16T18:15:02.8461176Z [cluster1]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-16T18:15:02.8462158Z [cluster1]                                                             "properties": {
2021-07-16T18:15:02.8462888Z [cluster1]                                                                 "lastTransitionTime": {
2021-07-16T18:15:02.8463885Z [cluster1]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-16T18:15:02.8465005Z [cluster1]                                                                     "format": "date-time",
2021-07-16T18:15:02.8465655Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8466212Z [cluster1]                                                                 },
2021-07-16T18:15:02.8466778Z [cluster1]                                                                 "message": {
2021-07-16T18:15:02.8467701Z [cluster1]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-16T18:15:02.8468602Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8469171Z [cluster1]                                                                 },
2021-07-16T18:15:02.8469712Z [cluster1]                                                                 "reason": {
2021-07-16T18:15:02.8470900Z [cluster1]                                                                     "description": "The reason for the condition's last transition.",
2021-07-16T18:15:02.8471725Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8472276Z [cluster1]                                                                 },
2021-07-16T18:15:02.8472826Z [cluster1]                                                                 "status": {
2021-07-16T18:15:02.8473649Z [cluster1]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-16T18:15:02.8474477Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8475035Z [cluster1]                                                                 },
2021-07-16T18:15:02.8476110Z [cluster1]                                                                 "type": {
2021-07-16T18:15:02.8477035Z [cluster1]                                                                     "description": "Type of DaemonSet condition.",
2021-07-16T18:15:02.8477795Z [cluster1]                                                                     "type": "string"
2021-07-16T18:15:02.8478368Z [cluster1]                                                                 }
2021-07-16T18:15:02.8478878Z [cluster1]                                                             },
2021-07-16T18:15:02.8479542Z [cluster1]                                                             "required": [
2021-07-16T18:15:02.8480121Z [cluster1]                                                                 "status",
2021-07-16T18:15:02.8480693Z [cluster1]                                                                 "type"
2021-07-16T18:15:02.8481230Z [cluster1]                                                             ],
2021-07-16T18:15:02.8481795Z [cluster1]                                                             "type": "object"
2021-07-16T18:15:02.8482358Z [cluster1]                                                         },
2021-07-16T18:15:02.8482905Z [cluster1]                                                         "type": "array"
2021-07-16T18:15:02.8483454Z [cluster1]                                                     },
2021-07-16T18:15:02.8484334Z [cluster1]                                                     "currentNumberScheduled": {
2021-07-16T18:15:02.8485915Z [cluster1]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8487306Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8495529Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8496136Z [cluster1]                                                     },
2021-07-16T18:15:02.8496845Z [cluster1]                                                     "desiredNumberScheduled": {
2021-07-16T18:15:02.8498513Z [cluster1]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8500049Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8500658Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8501228Z [cluster1]                                                     },
2021-07-16T18:15:02.8501843Z [cluster1]                                                     "numberAvailable": {
2021-07-16T18:15:02.8503355Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.8504629Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8505229Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8505793Z [cluster1]                                                     },
2021-07-16T18:15:02.8506431Z [cluster1]                                                     "numberMisscheduled": {
2021-07-16T18:15:02.8507923Z [cluster1]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8509300Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8509905Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8510468Z [cluster1]                                                     },
2021-07-16T18:15:02.8511036Z [cluster1]                                                     "numberReady": {
2021-07-16T18:15:02.8512529Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-16T18:15:02.8513640Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8514238Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8514805Z [cluster1]                                                     },
2021-07-16T18:15:02.8515432Z [cluster1]                                                     "numberUnavailable": {
2021-07-16T18:15:02.8516770Z [cluster1]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.8518008Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8518604Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8519163Z [cluster1]                                                     },
2021-07-16T18:15:02.8519808Z [cluster1]                                                     "observedGeneration": {
2021-07-16T18:15:02.8520818Z [cluster1]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-16T18:15:02.8521706Z [cluster1]                                                         "format": "int64",
2021-07-16T18:15:02.8522304Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8522869Z [cluster1]                                                     },
2021-07-16T18:15:02.8524162Z [cluster1]                                                     "updatedNumberScheduled": {
2021-07-16T18:15:02.8525194Z [cluster1]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-16T18:15:02.8526033Z [cluster1]                                                         "format": "int32",
2021-07-16T18:15:02.8526641Z [cluster1]                                                         "type": "integer"
2021-07-16T18:15:02.8527213Z [cluster1]                                                     }
2021-07-16T18:15:02.8527707Z [cluster1]                                                 },
2021-07-16T18:15:02.8528259Z [cluster1]                                                 "required": [
2021-07-16T18:15:02.8529090Z [cluster1]                                                     "currentNumberScheduled",
2021-07-16T18:15:02.8530932Z [cluster1]                                                     "desiredNumberScheduled",
2021-07-16T18:15:02.8531775Z [cluster1]                                                     "numberMisscheduled",
2021-07-16T18:15:02.8532478Z [cluster1]                                                     "numberReady"
2021-07-16T18:15:02.8533045Z [cluster1]                                                 ],
2021-07-16T18:15:02.8533584Z [cluster1]                                                 "type": "object"
2021-07-16T18:15:02.8534134Z [cluster1]                                             }
2021-07-16T18:15:02.8534608Z [cluster1]                                         },
2021-07-16T18:15:02.8535143Z [cluster1]                                         "required": [
2021-07-16T18:15:02.8536577Z [cluster1]                                             "mismatchedContainerImages"
2021-07-16T18:15:02.8537332Z [cluster1]                                         ],
2021-07-16T18:15:02.8537870Z [cluster1]                                         "type": "object"
2021-07-16T18:15:02.8538384Z [cluster1]                                     },
2021-07-16T18:15:02.8538935Z [cluster1]                                     "serviceCIDR": {
2021-07-16T18:15:02.8539530Z [cluster1]                                         "type": "string"
2021-07-16T18:15:02.8548042Z [cluster1]                                     }
2021-07-16T18:15:02.8548709Z [cluster1]                                 },
2021-07-16T18:15:02.8549240Z [cluster1]                                 "required": [
2021-07-16T18:15:02.8549810Z [cluster1]                                     "clusterID",
2021-07-16T18:15:02.8550395Z [cluster1]                                     "natEnabled"
2021-07-16T18:15:02.8550926Z [cluster1]                                 ],
2021-07-16T18:15:02.8551439Z [cluster1]                                 "type": "object"
2021-07-16T18:15:02.8551953Z [cluster1]                             }
2021-07-16T18:15:02.8552392Z [cluster1]                         },
2021-07-16T18:15:02.8552896Z [cluster1]                         "type": "object"
2021-07-16T18:15:02.8559798Z [cluster1]                     }
2021-07-16T18:15:02.8560259Z [cluster1]                 },
2021-07-16T18:15:02.8560746Z [cluster1]                 "served": true,
2021-07-16T18:15:02.8561303Z [cluster1]                 "storage": true,
2021-07-16T18:15:02.8561861Z [cluster1]                 "subresources": {
2021-07-16T18:15:02.8562431Z [cluster1]                     "status": {}
2021-07-16T18:15:02.8562905Z [cluster1]                 }
2021-07-16T18:15:02.8563324Z [cluster1]             }
2021-07-16T18:15:02.8564002Z [cluster1]         ]
2021-07-16T18:15:02.8564413Z [cluster1]     },
2021-07-16T18:15:02.8564865Z [cluster1]     "status": {
2021-07-16T18:15:02.8565404Z [cluster1]         "acceptedNames": {
2021-07-16T18:15:02.8566017Z [cluster1]             "kind": "Submariner",
2021-07-16T18:15:02.8566476Z [18:13:58Z",
2021-07-16T18:15:02.8566994Z [cluster2]                 "message": "no conflicts found",
2021-07-16T18:15:02.8567627Z [cluster2]                 "reason": "NoConflicts",
2021-07-16T18:15:02.8568210Z [cluster2]                 "status": "True",
2021-07-16T18:15:02.8568810Z [cluster2]                 "type": "NamesAccepted"
2021-07-16T18:15:02.8570069Z [cluster2]             },
2021-07-16T18:15:02.8570506Z [cluster2]             {
2021-07-16T18:15:02.8571577Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:58Z",
2021-07-16T18:15:02.8572420Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-16T18:15:02.8573509Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-16T18:15:02.8574382Z [cluster2]                 "status": "True",
2021-07-16T18:15:02.8574947Z [cluster2]                 "type": "Established"
2021-07-16T18:15:02.8575467Z [cluster2]             }
2021-07-16T18:15:02.8575887Z [cluster2]         ],
2021-07-16T18:15:02.8576588Z [cluster2]         "storedVersions": [
2021-07-16T18:15:02.8577134Z [cluster2]             "v1"
2021-07-16T18:15:02.8577554Z [cluster2]         ]
2021-07-16T18:15:02.8577977Z [cluster2]     }
2021-07-16T18:15:02.8578379Z [cluster2] }
2021-07-16T18:15:02.8579597Z [submariner-operator]$ [cluster2] validate_equals .metadata.name clusters.submariner.io
2021-07-16T18:15:02.8581647Z [submariner-operator]$ [cluster2] validate_equals .metadata.name clusters.submariner.io
2021-07-16T18:15:02.8583569Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/clusters.submariner.io.cluster2.json
2021-07-16T18:15:02.8585254Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-16T18:15:02.8586589Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-16T18:15:02.8588270Z [submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/clusters.submariner.io.cluster2.json
2021-07-16T18:15:02.8589924Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-16T18:15:02.8591297Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-16T18:15:02.8592991Z [submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/clusters.submariner.io.cluster2.json
2021-07-16T18:15:02.8594644Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Cluster
2021-07-16T18:15:02.8595992Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Cluster
2021-07-16T18:15:02.8597859Z [submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/clusters.submariner.io.cluster2.json
2021-07-16T18:15:02.8599755Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Cluster
2021-07-16T18:15:02.8601390Z [submariner-operator]$ [cluster2] validate_equals .status.acceptedNames.kind Cluster
2021-07-16T18:15:02.8603421Z [submariner-operator]$ [cluster2] jq -r -M .status.acceptedNames.kind /tmp/clusters.submariner.io.cluster2.json
2021-07-16T18:15:02.8605767Z [submariner-operator]$ [cluster2] verify_subm_crd
2021-07-16T18:15:02.8606785Z [submariner-operator]$ [cluster2] verify_subm_crd
2021-07-16T18:15:02.8608167Z [submariner-operator]$ [cluster2] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-16T18:15:02.8609896Z [submariner-operator]$ [cluster2] validate_crd submariners.submariner.io v1alpha1 Submariner
2021-07-16T18:15:02.8611536Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-16T18:15:02.8613065Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-16T18:15:02.8614757Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds submariners.submariner.io
2021-07-16T18:15:02.8615832Z [cluster2] NAME                        CREATED AT
2021-07-16T18:15:02.8616774Z [cluster2] submariners.submariner.io   2021-07-16T18:13:45Z
2021-07-16T18:15:02.8618954Z [submariner-operator]$ [cluster2] json_file=/tmp/submariners.submariner.io.cluster2.json
2021-07-16T18:15:02.8620733Z [submariner-operator]$ [cluster2] kubectl get crd submariners.submariner.io -o json
2021-07-16T18:15:02.8622307Z [submariner-operator]$ [cluster2] kubectl get crd submariners.submariner.io -o json
2021-07-16T18:15:02.8624021Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crd submariners.submariner.io -o json
2021-07-16T18:15:02.8626351Z [submariner-operator]$ [cluster2] tee /tmp/submariners.submariner.io.cluster2.json
2021-07-16T18:15:02.8627437Z [cluster2] {
2021-07-16T18:15:02.8628095Z [cluster2]     "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:02.8629005Z [cluster2]     "kind": "CustomResourceDefinition",
2021-07-16T18:15:02.8629703Z [cluster2]     "metadata": {
2021-07-16T18:15:02.8630234Z [cluster2]         "annotations": {
2021-07-16T18:15:02.8634939Z [cluster2]             "controller-gen.kubebuilder.io/version": "v0.4.1"
2021-07-16T18:15:02.8635948Z [cluster2]         },
2021-07-16T18:15:02.8636829Z [cluster2]         "creationTimestamp": "2021-07-16T18:13:45Z",
2021-07-16T18:15:02.8637503Z [cluster2]         "generation": 1,
2021-07-16T18:15:02.8638076Z [cluster2]         "managedFields": [
2021-07-16T18:15:02.8638600Z [cluster2]             {
2021-07-16T18:15:02.8639268Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:02.8640063Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:02.8640658Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:02.8641167Z [cluster2]                     "f:status": {
2021-07-16T18:15:02.8641762Z [cluster2]                         "f:acceptedNames": {
2021-07-16T18:15:02.8642334Z [cluster2]                             "f:kind": {},
2021-07-16T18:15:02.8642875Z [cluster2]                             "f:listKind": {},
2021-07-16T18:15:02.8643421Z [cluster2]                             "f:plural": {},
2021-07-16T18:15:02.8644178Z [cluster2]                             "f:singular": {}
2021-07-16T18:15:02.8644669Z [cluster2]                         },
2021-07-16T18:15:02.8645194Z [cluster2]                         "f:conditions": {}
2021-07-16T18:15:02.8645693Z [cluster2]                     }
2021-07-16T18:15:02.8646138Z [cluster2]                 },
2021-07-16T18:15:02.8646913Z [cluster2]                 "manager": "kube-apiserver",
2021-07-16T18:15:02.8647675Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:02.8648440Z [cluster2]                 "time": "2021-07-16T18:13:45Z"
2021-07-16T18:15:02.8649145Z [cluster2]             },
2021-07-16T18:15:02.8649589Z [cluster2]             {
2021-07-16T18:15:02.8650552Z [cluster2]                 "apiVersion": "apiextensions.k8s.io/v1",
2021-07-16T18:15:02.8651355Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:02.8651932Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:02.8652476Z [cluster2]                     "f:metadata": {
2021-07-16T18:15:02.8653036Z [cluster2]                         "f:annotations": {
2021-07-16T18:15:02.8653577Z [cluster2]                             ".": {},
2021-07-16T18:15:02.8654634Z [cluster2]                             "f:controller-gen.kubebuilder.io/version": {}
2021-07-16T18:15:02.8655430Z [cluster2]                         }
2021-07-16T18:15:02.8655886Z [cluster2]                     },
2021-07-16T18:15:02.8656351Z [cluster2]                     "f:spec": {
2021-07-16T18:15:02.8656903Z [cluster2]                         "f:conversion": {
2021-07-16T18:15:02.8657422Z [cluster2]                             ".": {},
2021-07-16T18:15:02.8657944Z [cluster2]                             "f:strategy": {}
2021-07-16T18:15:02.8658438Z [cluster2]                         },
2021-07-16T18:15:02.8658931Z [cluster2]                         "f:group": {},
2021-07-16T18:15:02.8659450Z [cluster2]                         "f:names": {
2021-07-16T18:15:02.8659959Z [cluster2]                             "f:kind": {},
2021-07-16T18:15:02.8660502Z [cluster2]                             "f:listKind": {},
2021-07-16T18:15:02.8661044Z [cluster2]                             "f:plural": {},
2021-07-16T18:15:02.8661595Z [cluster2]                             "f:singular": {}
2021-07-16T18:15:02.8662087Z [cluster2]                         },
2021-07-16T18:15:02.8662575Z [cluster2]                         "f:scope": {},
2021-07-16T18:15:02.8663099Z [cluster2]                         "f:versions": {}
2021-07-16T18:15:02.8663603Z [cluster2]                     },
2021-07-16T18:15:02.8664073Z [cluster2]                     "f:status": {
2021-07-16T18:15:02.8664667Z [cluster2]                         "f:storedVersions": {}
2021-07-16T18:15:02.8665227Z [cluster2]                     }
2021-07-16T18:15:02.8665660Z [cluster2]                 },
2021-07-16T18:15:02.8666170Z [cluster2]                 "manager": "subctl",
2021-07-16T18:15:02.8666862Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:02.8667641Z [cluster2]                 "time": "2021-07-16T18:13:45Z"
2021-07-16T18:15:02.8668117Z [cluster2]             }
2021-07-16T18:15:02.8668549Z [cluster2]         ],
2021-07-16T18:15:02.8669215Z [cluster2]         "name": "submariners.submariner.io",
2021-07-16T18:15:02.8669996Z [cluster2]         "resourceVersion": "1677",
2021-07-16T18:15:02.8671236Z [cluster2]         "selfLink": "/apis/apiextensions.k8s.io/v1/customresourcedefinitions/submariners.submariner.io",
2021-07-16T18:15:02.8673107Z [cluster2]         "uid": "6ed8ca1c-f643-4425-8330-00c15cdbff1d"
2021-07-16T18:15:02.8673917Z [cluster2]     },
2021-07-16T18:15:02.8674487Z [cluster2]     "spec": {
2021-07-16T18:15:02.8675014Z [cluster2]         "conversion": {
2021-07-16T18:15:02.8675710Z [cluster2]             "strategy": "None"
2021-07-16T18:15:02.8676349Z [cluster2]         },
2021-07-16T18:15:02.8677012Z [cluster2]         "group": "submariner.io",
2021-07-16T18:15:02.8677721Z [cluster2]         "names": {
2021-07-16T18:15:02.8678386Z [cluster2]             "kind": "Submariner",
2021-07-16T18:15:02.8679184Z [cluster2]             "listKind": "SubmarinerList",
2021-07-16T18:15:02.8679990Z [cluster2]             "plural": "submariners",
2021-07-16T18:15:02.8680763Z [cluster2]             "singular": "submariner"
2021-07-16T18:15:02.8681293Z [cluster2]         },
2021-07-16T18:15:02.8681938Z [cluster2]         "scope": "Namespaced",
2021-07-16T18:15:02.8682872Z [cluster2]         "versions": [
2021-07-16T18:15:02.8683436Z [cluster2]             {
2021-07-16T18:15:02.8684180Z [cluster2]                 "name": "v1alpha1",
2021-07-16T18:15:02.8684692Z [cluster2]                 "schema": {
2021-07-16T18:15:02.8685291Z [cluster2]                     "openAPIV3Schema": {
2021-07-16T18:15:02.8686138Z [cluster2]                         "description": "Submariner is the Schema for the submariners API",
2021-07-16T18:15:02.8686935Z [cluster2]                         "properties": {
2021-07-16T18:15:02.8687536Z [cluster2]                             "apiVersion": {
2021-07-16T18:15:02.8690108Z [cluster2]                                 "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources",
2021-07-16T18:15:02.8692078Z [cluster2]                                 "type": "string"
2021-07-16T18:15:02.8692604Z [cluster2]                             },
2021-07-16T18:15:02.8693087Z [cluster2]                             "kind": {
2021-07-16T18:15:02.8695494Z [cluster2]                                 "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds",
2021-07-16T18:15:02.8697428Z [cluster2]                                 "type": "string"
2021-07-16T18:15:02.8697943Z [cluster2]                             },
2021-07-16T18:15:02.8698453Z [cluster2]                             "metadata": {
2021-07-16T18:15:02.8699011Z [cluster2]                                 "type": "object"
2021-07-16T18:15:02.8699554Z [cluster2]                             },
2021-07-16T18:15:02.8700029Z [cluster2]                             "spec": {
2021-07-16T18:15:02.8700879Z [cluster2]                                 "description": "SubmarinerSpec defines the desired state of Submariner",
2021-07-16T18:15:02.8701953Z [cluster2]                                 "properties": {
2021-07-16T18:15:02.8702697Z [cluster2]                                     "broker": {
2021-07-16T18:15:02.8703408Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8703948Z [cluster2]                                     },
2021-07-16T18:15:02.8704695Z [cluster2]                                     "brokerK8sApiServer": {
2021-07-16T18:15:02.8705369Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8705892Z [cluster2]                                     },
2021-07-16T18:15:02.8706575Z [cluster2]                                     "brokerK8sApiServerToken": {
2021-07-16T18:15:02.8707332Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8707858Z [cluster2]                                     },
2021-07-16T18:15:02.8708393Z [cluster2]                                     "brokerK8sCA": {
2021-07-16T18:15:02.8708990Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8709505Z [cluster2]                                     },
2021-07-16T18:15:02.8710223Z [cluster2]                                     "brokerK8sRemoteNamespace": {
2021-07-16T18:15:02.8710977Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8711503Z [cluster2]                                     },
2021-07-16T18:15:02.8712040Z [cluster2]                                     "cableDriver": {
2021-07-16T18:15:02.8712649Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8713171Z [cluster2]                                     },
2021-07-16T18:15:02.8713729Z [cluster2]                                     "ceIPSecDebug": {
2021-07-16T18:15:02.8714359Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.8714957Z [cluster2]                                     },
2021-07-16T18:15:02.8715635Z [cluster2]                                     "ceIPSecForceUDPEncaps": {
2021-07-16T18:15:02.8716883Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.8717583Z [cluster2]                                     },
2021-07-16T18:15:02.8718166Z [cluster2]                                     "ceIPSecIKEPort": {
2021-07-16T18:15:02.8718812Z [cluster2]                                         "type": "integer"
2021-07-16T18:15:02.8719348Z [cluster2]                                     },
2021-07-16T18:15:02.8719931Z [cluster2]                                     "ceIPSecNATTPort": {
2021-07-16T18:15:02.8720594Z [cluster2]                                         "type": "integer"
2021-07-16T18:15:02.8721112Z [cluster2]                                     },
2021-07-16T18:15:02.8721658Z [cluster2]                                     "ceIPSecPSK": {
2021-07-16T18:15:02.8722248Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8722771Z [cluster2]                                     },
2021-07-16T18:15:02.8723471Z [cluster2]                                     "ceIPSecPreferredServer": {
2021-07-16T18:15:02.8724617Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.8725154Z [cluster2]                                     },
2021-07-16T18:15:02.8725695Z [cluster2]                                     "clusterCIDR": {
2021-07-16T18:15:02.8726296Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8726809Z [cluster2]                                     },
2021-07-16T18:15:02.8727336Z [cluster2]                                     "clusterID": {
2021-07-16T18:15:02.8727907Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8728429Z [cluster2]                                     },
2021-07-16T18:15:02.8729734Z [cluster2]                                     "colorCodes": {
2021-07-16T18:15:02.8730342Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8730868Z [cluster2]                                     },
2021-07-16T18:15:02.8731529Z [cluster2]                                     "connectionHealthCheck": {
2021-07-16T18:15:02.8732278Z [cluster2]                                         "properties": {
2021-07-16T18:15:02.8732872Z [cluster2]                                             "enabled": {
2021-07-16T18:15:02.8734207Z [cluster2]                                                 "type": "boolean"
2021-07-16T18:15:02.8734763Z [cluster2]                                             },
2021-07-16T18:15:02.8735373Z [cluster2]                                             "intervalSeconds": {
2021-07-16T18:15:02.8736251Z [cluster2]                                                 "description": "The interval at which health check pings are sent.",
2021-07-16T18:15:02.8737038Z [cluster2]                                                 "format": "int64",
2021-07-16T18:15:02.8737651Z [cluster2]                                                 "type": "integer"
2021-07-16T18:15:02.8738190Z [cluster2]                                             },
2021-07-16T18:15:02.8738835Z [cluster2]                                             "maxPacketLossCount": {
2021-07-16T18:15:02.8739917Z [cluster2]                                                 "description": "The maximum number of packets lost at which the health checker will mark the connection as down.",
2021-07-16T18:15:02.8740858Z [cluster2]                                                 "format": "int64",
2021-07-16T18:15:02.8741466Z [cluster2]                                                 "type": "integer"
2021-07-16T18:15:02.8742000Z [cluster2]                                             }
2021-07-16T18:15:02.8742487Z [cluster2]                                         },
2021-07-16T18:15:02.8743025Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.8743611Z [cluster2]                                     },
2021-07-16T18:15:02.8744251Z [cluster2]                                     "coreDNSCustomConfig": {
2021-07-16T18:15:02.8744952Z [cluster2]                                         "properties": {
2021-07-16T18:15:02.8745606Z [cluster2]                                             "configMapName": {
2021-07-16T18:15:02.8746230Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.8747344Z [cluster2]                                             },
2021-07-16T18:15:02.8748059Z [cluster2]                                             "namespace": {
2021-07-16T18:15:02.8748679Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.8749219Z [cluster2]                                             }
2021-07-16T18:15:02.8749695Z [cluster2]                                         },
2021-07-16T18:15:02.8750230Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.8751943Z [cluster2]                                     },
2021-07-16T18:15:02.8752523Z [cluster2]                                     "customDomains": {
2021-07-16T18:15:02.8753105Z [cluster2]                                         "items": {
2021-07-16T18:15:02.8753672Z [cluster2]                                             "type": "string"
2021-07-16T18:15:02.8754192Z [cluster2]                                         },
2021-07-16T18:15:02.8754727Z [cluster2]                                         "type": "array",
2021-07-16T18:15:02.8755848Z [cluster2]                                         "x-kubernetes-list-type": "set"
2021-07-16T18:15:02.8756519Z [cluster2]                                     },
2021-07-16T18:15:02.8757018Z [cluster2]                                     "debug": {
2021-07-16T18:15:02.8757565Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.8758726Z [cluster2]                                     },
2021-07-16T18:15:02.8759270Z [cluster2]                                     "globalCIDR": {
2021-07-16T18:15:02.8759866Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8760397Z [cluster2]                                     },
2021-07-16T18:15:02.8760963Z [cluster2]                                     "imageOverrides": {
2021-07-16T18:15:02.8761732Z [cluster2]                                         "additionalProperties": {
2021-07-16T18:15:02.8762560Z [cluster2]                                             "type": "string"
2021-07-16T18:15:02.8763106Z [cluster2]                                         },
2021-07-16T18:15:02.8763735Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.8764260Z [cluster2]                                     },
2021-07-16T18:15:02.8764887Z [cluster2]                                     "loadBalancerEnabled": {
2021-07-16T18:15:02.8765598Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.8766138Z [cluster2]                                     },
2021-07-16T18:15:02.8766658Z [cluster2]                                     "namespace": {
2021-07-16T18:15:02.8767249Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8767754Z [cluster2]                                     },
2021-07-16T18:15:02.8768297Z [cluster2]                                     "natEnabled": {
2021-07-16T18:15:02.8768888Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.8769418Z [cluster2]                                     },
2021-07-16T18:15:02.8769942Z [cluster2]                                     "repository": {
2021-07-16T18:15:02.8770540Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8771066Z [cluster2]                                     },
2021-07-16T18:15:02.8771613Z [cluster2]                                     "serviceCIDR": {
2021-07-16T18:15:02.8772217Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8772813Z [cluster2]                                     },
2021-07-16T18:15:02.8773518Z [cluster2]                                     "serviceDiscoveryEnabled": {
2021-07-16T18:15:02.8774556Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.8775095Z [cluster2]                                     },
2021-07-16T18:15:02.8775597Z [cluster2]                                     "version": {
2021-07-16T18:15:02.8776170Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8776698Z [cluster2]                                     }
2021-07-16T18:15:02.8777160Z [cluster2]                                 },
2021-07-16T18:15:02.8777681Z [cluster2]                                 "required": [
2021-07-16T18:15:02.8778220Z [cluster2]                                     "broker",
2021-07-16T18:15:02.8778870Z [cluster2]                                     "brokerK8sApiServer",
2021-07-16T18:15:02.8779688Z [cluster2]                                     "brokerK8sApiServerToken",
2021-07-16T18:15:02.8780449Z [cluster2]                                     "brokerK8sCA",
2021-07-16T18:15:02.8781218Z [cluster2]                                     "brokerK8sRemoteNamespace",
2021-07-16T18:15:02.8781993Z [cluster2]                                     "ceIPSecDebug",
2021-07-16T18:15:02.8782626Z [cluster2]                                     "ceIPSecPSK",
2021-07-16T18:15:02.8783225Z [cluster2]                                     "clusterCIDR",
2021-07-16T18:15:02.8783812Z [cluster2]                                     "clusterID",
2021-07-16T18:15:02.8784346Z [cluster2]                                     "debug",
2021-07-16T18:15:02.8784913Z [cluster2]                                     "namespace",
2021-07-16T18:15:02.8785486Z [cluster2]                                     "natEnabled",
2021-07-16T18:15:02.8786087Z [cluster2]                                     "serviceCIDR"
2021-07-16T18:15:02.8786620Z [cluster2]                                 ],
2021-07-16T18:15:02.8787132Z [cluster2]                                 "type": "object"
2021-07-16T18:15:02.8787642Z [cluster2]                             },
2021-07-16T18:15:02.8788559Z [cluster2]                             "status": {
2021-07-16T18:15:02.8789439Z [cluster2]                                 "description": "SubmarinerStatus defines the observed state of Submariner",
2021-07-16T18:15:02.8790489Z [cluster2]                                 "properties": {
2021-07-16T18:15:02.8791118Z [cluster2]                                     "clusterCIDR": {
2021-07-16T18:15:02.8791753Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8792282Z [cluster2]                                     },
2021-07-16T18:15:02.8792818Z [cluster2]                                     "clusterID": {
2021-07-16T18:15:02.8793387Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8793919Z [cluster2]                                     },
2021-07-16T18:15:02.8794446Z [cluster2]                                     "colorCodes": {
2021-07-16T18:15:02.8795042Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.8795550Z [cluster2]                                     },
2021-07-16T18:15:02.8796129Z [cluster2]                                     "deploymentInfo": {
2021-07-16T18:15:02.8796771Z [cluster2]                                         "properties": {
2021-07-16T18:15:02.8797420Z [cluster2]                                             "cloudProvider": {
2021-07-16T18:15:02.8798059Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.8798590Z [cluster2]                                             },
2021-07-16T18:15:02.8799185Z [cluster2]                                             "kubernetesType": {
2021-07-16T18:15:02.8799875Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.8800488Z [cluster2]                                             },
2021-07-16T18:15:02.8801161Z [cluster2]                                             "kubernetesTypeVersion": {
2021-07-16T18:15:02.8801901Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.8802439Z [cluster2]                                             },
2021-07-16T18:15:02.8803053Z [cluster2]                                             "kubernetesVersion": {
2021-07-16T18:15:02.8803819Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.8804439Z [cluster2]                                             }
2021-07-16T18:15:02.8805117Z [cluster2]                                         },
2021-07-16T18:15:02.8805645Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.8806177Z [cluster2]                                     },
2021-07-16T18:15:02.8806846Z [cluster2]                                     "gatewayDaemonSetStatus": {
2021-07-16T18:15:02.8807612Z [cluster2]                                         "properties": {
2021-07-16T18:15:02.8808336Z [cluster2]                                             "lastResourceVersion": {
2021-07-16T18:15:02.8809032Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.8809573Z [cluster2]                                             },
2021-07-16T18:15:02.8810299Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-16T18:15:02.8811113Z [cluster2]                                                 "type": "boolean"
2021-07-16T18:15:02.8811648Z [cluster2]                                             },
2021-07-16T18:15:02.8812363Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-16T18:15:02.8813107Z [cluster2]                                                 "items": {
2021-07-16T18:15:02.8814471Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-16T18:15:02.8815887Z [cluster2]                                                     "properties": {
2021-07-16T18:15:02.8816502Z [cluster2]                                                         "running": {
2021-07-16T18:15:02.8817351Z [cluster2]                                                             "description": "Details about a running container",
2021-07-16T18:15:02.8818147Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.8818788Z [cluster2]                                                                 "startedAt": {
2021-07-16T18:15:02.8820051Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-16T18:15:02.8821115Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.8821760Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8823063Z [cluster2]                                                                 }
2021-07-16T18:15:02.8823748Z [cluster2]                                                             },
2021-07-16T18:15:02.8824333Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.8824898Z [cluster2]                                                         },
2021-07-16T18:15:02.8825462Z [cluster2]                                                         "terminated": {
2021-07-16T18:15:02.8826520Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-16T18:15:02.8827320Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.8828096Z [cluster2]                                                                 "containerID": {
2021-07-16T18:15:02.8829353Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-16T18:15:02.8830209Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8830788Z [cluster2]                                                                 },
2021-07-16T18:15:02.8831360Z [cluster2]                                                                 "exitCode": {
2021-07-16T18:15:02.8832228Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-16T18:15:02.8833065Z [cluster2]                                                                     "format": "int32",
2021-07-16T18:15:02.8833685Z [cluster2]                                                                     "type": "integer"
2021-07-16T18:15:02.8834270Z [cluster2]                                                                 },
2021-07-16T18:15:02.8834839Z [cluster2]                                                                 "finishedAt": {
2021-07-16T18:15:02.8835678Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-16T18:15:02.8836750Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.8837392Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8837964Z [cluster2]                                                                 },
2021-07-16T18:15:02.8838512Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.8839397Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-16T18:15:02.8840269Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8840824Z [cluster2]                                                                 },
2021-07-16T18:15:02.8841380Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.8842292Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-16T18:15:02.8843145Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8843828Z [cluster2]                                                                 },
2021-07-16T18:15:02.8844369Z [cluster2]                                                                 "signal": {
2021-07-16T18:15:02.8845199Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-16T18:15:02.8846023Z [cluster2]                                                                     "format": "int32",
2021-07-16T18:15:02.8846658Z [cluster2]                                                                     "type": "integer"
2021-07-16T18:15:02.8847223Z [cluster2]                                                                 },
2021-07-16T18:15:02.8847808Z [cluster2]                                                                 "startedAt": {
2021-07-16T18:15:02.8848696Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-16T18:15:02.8849823Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.8850473Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8851116Z [cluster2]                                                                 }
2021-07-16T18:15:02.8851636Z [cluster2]                                                             },
2021-07-16T18:15:02.8852200Z [cluster2]                                                             "required": [
2021-07-16T18:15:02.8852808Z [cluster2]                                                                 "exitCode"
2021-07-16T18:15:02.8853379Z [cluster2]                                                             ],
2021-07-16T18:15:02.8853933Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.8854491Z [cluster2]                                                         },
2021-07-16T18:15:02.8855030Z [cluster2]                                                         "waiting": {
2021-07-16T18:15:02.8855791Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-16T18:15:02.8857502Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.8858995Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.8859864Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-16T18:15:02.8860705Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8861276Z [cluster2]                                                                 },
2021-07-16T18:15:02.8861839Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.8862629Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-16T18:15:02.8863441Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8864005Z [cluster2]                                                                 }
2021-07-16T18:15:02.8865886Z [cluster2]                                                             },
2021-07-16T18:15:02.8866766Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.8867337Z [cluster2]                                                         }
2021-07-16T18:15:02.8867987Z [cluster2]                                                     },
2021-07-16T18:15:02.8868548Z [cluster2]                                                     "type": "object"
2021-07-16T18:15:02.8869102Z [cluster2]                                                 },
2021-07-16T18:15:02.8869626Z [cluster2]                                                 "type": "array"
2021-07-16T18:15:02.8870165Z [cluster2]                                             },
2021-07-16T18:15:02.8870677Z [cluster2]                                             "status": {
2021-07-16T18:15:02.8871582Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-16T18:15:02.8872880Z [cluster2]                                                 "properties": {
2021-07-16T18:15:02.8873539Z [cluster2]                                                     "collisionCount": {
2021-07-16T18:15:02.8875040Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-16T18:15:02.8876476Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8877602Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8878335Z [cluster2]                                                     },
2021-07-16T18:15:02.8878996Z [cluster2]                                                     "conditions": {
2021-07-16T18:15:02.8880433Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-16T18:15:02.8881373Z [cluster2]                                                         "items": {
2021-07-16T18:15:02.8882335Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-16T18:15:02.8883331Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.8884147Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-16T18:15:02.8885918Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-16T18:15:02.8887136Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.8887780Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8888582Z [cluster2]                                                                 },
2021-07-16T18:15:02.8889292Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.8890241Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-16T18:15:02.8891160Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8891720Z [cluster2]                                                                 },
2021-07-16T18:15:02.8892285Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.8893399Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-16T18:15:02.8894224Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8894800Z [cluster2]                                                                 },
2021-07-16T18:15:02.8895456Z [cluster2]                                                                 "status": {
2021-07-16T18:15:02.8910963Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-16T18:15:02.8911812Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8912390Z [cluster2]                                                                 },
2021-07-16T18:15:02.8912933Z [cluster2]                                                                 "type": {
2021-07-16T18:15:02.8913703Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-16T18:15:02.8914476Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.8915036Z [cluster2]                                                                 }
2021-07-16T18:15:02.8915567Z [cluster2]                                                             },
2021-07-16T18:15:02.8916124Z [cluster2]                                                             "required": [
2021-07-16T18:15:02.8916726Z [cluster2]                                                                 "status",
2021-07-16T18:15:02.8917285Z [cluster2]                                                                 "type"
2021-07-16T18:15:02.8917823Z [cluster2]                                                             ],
2021-07-16T18:15:02.8918398Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.8920157Z [cluster2]                                                         },
2021-07-16T18:15:02.8920889Z [cluster2]                                                         "type": "array"
2021-07-16T18:15:02.8921425Z [cluster2]                                                     },
2021-07-16T18:15:02.8922998Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-16T18:15:02.8924920Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8926296Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8926934Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8927492Z [cluster2]                                                     },
2021-07-16T18:15:02.8928204Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-16T18:15:02.8929858Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8931310Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8931935Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8932503Z [cluster2]                                                     },
2021-07-16T18:15:02.8933109Z [cluster2]                                                     "numberAvailable": {
2021-07-16T18:15:02.8934444Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.8936157Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8936761Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8937452Z [cluster2]                                                     },
2021-07-16T18:15:02.8938096Z [cluster2]                                                     "numberMisscheduled": {
2021-07-16T18:15:02.8939584Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.8940954Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8941565Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8942128Z [cluster2]                                                     },
2021-07-16T18:15:02.8942692Z [cluster2]                                                     "numberReady": {
2021-07-16T18:15:02.8943746Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-16T18:15:02.8944734Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8945344Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8945910Z [cluster2]                                                     },
2021-07-16T18:15:02.8946532Z [cluster2]                                                     "numberUnavailable": {
2021-07-16T18:15:02.8947940Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.8949171Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8949901Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8950466Z [cluster2]                                                     },
2021-07-16T18:15:02.8952123Z [cluster2]                                                     "observedGeneration": {
2021-07-16T18:15:02.8953145Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-16T18:15:02.8954022Z [cluster2]                                                         "format": "int64",
2021-07-16T18:15:02.8954645Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8955210Z [cluster2]                                                     },
2021-07-16T18:15:02.8955909Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-16T18:15:02.8956931Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-16T18:15:02.8957759Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.8958373Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.8958919Z [cluster2]                                                     }
2021-07-16T18:15:02.8959421Z [cluster2]                                                 },
2021-07-16T18:15:02.8959965Z [cluster2]                                                 "required": [
2021-07-16T18:15:02.8978680Z [cluster2]                                                     "currentNumberScheduled",
2021-07-16T18:15:02.8979661Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-16T18:15:02.8980495Z [cluster2]                                                     "numberMisscheduled",
2021-07-16T18:15:02.8981214Z [cluster2]                                                     "numberReady"
2021-07-16T18:15:02.8981938Z [cluster2]                                                 ],
2021-07-16T18:15:02.8982497Z [cluster2]                                                 "type": "object"
2021-07-16T18:15:02.8983048Z [cluster2]                                             }
2021-07-16T18:15:02.8983522Z [cluster2]                                         },
2021-07-16T18:15:02.8984058Z [cluster2]                                         "required": [
2021-07-16T18:15:02.8984824Z [cluster2]                                             "mismatchedContainerImages"
2021-07-16T18:15:02.8985561Z [cluster2]                                         ],
2021-07-16T18:15:02.8986085Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.8986618Z [cluster2]                                     },
2021-07-16T18:15:02.8987146Z [cluster2]                                     "gateways": {
2021-07-16T18:15:02.8987688Z [cluster2]                                         "items": {
2021-07-16T18:15:02.8988275Z [cluster2]                                             "properties": {
2021-07-16T18:15:02.8988901Z [cluster2]                                                 "connections": {
2021-07-16T18:15:02.8989498Z [cluster2]                                                     "items": {
2021-07-16T18:15:02.8990086Z [cluster2]                                                         "properties": {
2021-07-16T18:15:02.8990722Z [cluster2]                                                             "endpoint": {
2021-07-16T18:15:02.8991347Z [cluster2]                                                                 "properties": {
2021-07-16T18:15:02.8992059Z [cluster2]                                                                     "backend": {
2021-07-16T18:15:02.8992675Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.8993240Z [cluster2]                                                                     },
2021-07-16T18:15:02.8993838Z [cluster2]                                                                     "backend_config": {
2021-07-16T18:15:02.8994595Z [cluster2]                                                                         "additionalProperties": {
2021-07-16T18:15:02.8995360Z [cluster2]                                                                             "type": "string"
2021-07-16T18:15:02.8995945Z [cluster2]                                                                         },
2021-07-16T18:15:02.8996520Z [cluster2]                                                                         "type": "object"
2021-07-16T18:15:02.8997101Z [cluster2]                                                                     },
2021-07-16T18:15:02.8997650Z [cluster2]                                                                     "cable_name": {
2021-07-16T18:15:02.8998277Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.8998839Z [cluster2]                                                                     },
2021-07-16T18:15:02.8999466Z [cluster2]                                                                     "cluster_id": {
2021-07-16T18:15:02.9000235Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9000800Z [cluster2]                                                                     },
2021-07-16T18:15:02.9001424Z [cluster2]                                                                     "healthCheckIP": {
2021-07-16T18:15:02.9002094Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9002668Z [cluster2]                                                                     },
2021-07-16T18:15:02.9003241Z [cluster2]                                                                     "hostname": {
2021-07-16T18:15:02.9004538Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9005224Z [cluster2]                                                                     },
2021-07-16T18:15:02.9005804Z [cluster2]                                                                     "nat_enabled": {
2021-07-16T18:15:02.9006450Z [cluster2]                                                                         "type": "boolean"
2021-07-16T18:15:02.9007019Z [cluster2]                                                                     },
2021-07-16T18:15:02.9007603Z [cluster2]                                                                     "private_ip": {
2021-07-16T18:15:02.9008842Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9009417Z [cluster2]                                                                     },
2021-07-16T18:15:02.9009984Z [cluster2]                                                                     "public_ip": {
2021-07-16T18:15:02.9010601Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9011178Z [cluster2]                                                                     },
2021-07-16T18:15:02.9011745Z [cluster2]                                                                     "subnets": {
2021-07-16T18:15:02.9012330Z [cluster2]                                                                         "items": {
2021-07-16T18:15:02.9013200Z [cluster2]                                                                             "type": "string"
2021-07-16T18:15:02.9013877Z [cluster2]                                                                         },
2021-07-16T18:15:02.9079624Z [cluster2]                                                                         "type": "array"
2021-07-16T18:15:02.9080382Z [cluster2]                                                                     }
2021-07-16T18:15:02.9080920Z [cluster2]                                                                 },
2021-07-16T18:15:02.9081513Z [cluster2]                                                                 "required": [
2021-07-16T18:15:02.9082113Z [cluster2]                                                                     "backend",
2021-07-16T18:15:02.9082719Z [cluster2]                                                                     "cable_name",
2021-07-16T18:15:02.9083319Z [cluster2]                                                                     "cluster_id",
2021-07-16T18:15:02.9084737Z [cluster2]                                                                     "hostname",
2021-07-16T18:15:02.9085431Z [cluster2]                                                                     "nat_enabled",
2021-07-16T18:15:02.9086057Z [cluster2]                                                                     "private_ip",
2021-07-16T18:15:02.9086673Z [cluster2]                                                                     "public_ip",
2021-07-16T18:15:02.9087262Z [cluster2]                                                                     "subnets"
2021-07-16T18:15:02.9087829Z [cluster2]                                                                 ],
2021-07-16T18:15:02.9088398Z [cluster2]                                                                 "type": "object"
2021-07-16T18:15:02.9088967Z [cluster2]                                                             },
2021-07-16T18:15:02.9089551Z [cluster2]                                                             "latencyRTT": {
2021-07-16T18:15:02.9090659Z [cluster2]                                                                 "description": "LatencySpec describes the round trip time information for a packet between the gateway pods of two clusters.",
2021-07-16T18:15:02.9091848Z [cluster2]                                                                 "properties": {
2021-07-16T18:15:02.9092465Z [cluster2]                                                                     "average": {
2021-07-16T18:15:02.9093386Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9093982Z [cluster2]                                                                     },
2021-07-16T18:15:02.9094520Z [cluster2]                                                                     "last": {
2021-07-16T18:15:02.9095124Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9095689Z [cluster2]                                                                     },
2021-07-16T18:15:02.9096238Z [cluster2]                                                                     "max": {
2021-07-16T18:15:02.9096815Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9097390Z [cluster2]                                                                     },
2021-07-16T18:15:02.9097936Z [cluster2]                                                                     "min": {
2021-07-16T18:15:02.9098525Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9099096Z [cluster2]                                                                     },
2021-07-16T18:15:02.9099678Z [cluster2]                                                                     "stdDev": {
2021-07-16T18:15:02.9100297Z [cluster2]                                                                         "type": "string"
2021-07-16T18:15:02.9100857Z [cluster2]                                                                     }
2021-07-16T18:15:02.9101478Z [cluster2]                                                                 },
2021-07-16T18:15:02.9102051Z [cluster2]                                                                 "type": "object"
2021-07-16T18:15:02.9102599Z [cluster2]                                                             },
2021-07-16T18:15:02.9103142Z [cluster2]                                                             "status": {
2021-07-16T18:15:02.9103725Z [cluster2]                                                                 "type": "string"
2021-07-16T18:15:02.9104284Z [cluster2]                                                             },
2021-07-16T18:15:02.9104870Z [cluster2]                                                             "statusMessage": {
2021-07-16T18:15:02.9105533Z [cluster2]                                                                 "type": "string"
2021-07-16T18:15:02.9106098Z [cluster2]                                                             },
2021-07-16T18:15:02.9106651Z [cluster2]                                                             "usingIP": {
2021-07-16T18:15:02.9107262Z [cluster2]                                                                 "type": "string"
2021-07-16T18:15:02.9107815Z [cluster2]                                                             },
2021-07-16T18:15:02.9108380Z [cluster2]                                                             "usingNAT": {
2021-07-16T18:15:02.9109011Z [cluster2]                                                                 "type": "boolean"
2021-07-16T18:15:02.9109573Z [cluster2]                                                             }
2021-07-16T18:15:02.9110093Z [cluster2]                                                         },
2021-07-16T18:15:02.9110638Z [cluster2]                                                         "required": [
2021-07-16T18:15:02.9111249Z [cluster2]                                                             "endpoint",
2021-07-16T18:15:02.9111826Z [cluster2]                                                             "status",
2021-07-16T18:15:02.9112449Z [cluster2]                                                             "statusMessage"
2021-07-16T18:15:02.9113044Z [cluster2]                                                         ],
2021-07-16T18:15:02.9113594Z [cluster2]                                                         "type": "object"
2021-07-16T18:15:02.9114204Z [cluster2]                                                     },
2021-07-16T18:15:02.9114744Z [cluster2]                                                     "type": "array"
2021-07-16T18:15:02.9115290Z [cluster2]                                                 },
2021-07-16T18:15:02.9115821Z [cluster2]                                                 "haStatus": {
2021-07-16T18:15:02.9116420Z [cluster2]                                                     "type": "string"
2021-07-16T18:15:02.9116954Z [cluster2]                                                 },
2021-07-16T18:15:02.9117545Z [cluster2]                                                 "localEndpoint": {
2021-07-16T18:15:02.9118204Z [cluster2]                                                     "properties": {
2021-07-16T18:15:02.9118811Z [cluster2]                                                         "backend": {
2021-07-16T18:15:02.9119414Z [cluster2]                                                             "type": "string"
2021-07-16T18:15:02.9119964Z [cluster2]                                                         },
2021-07-16T18:15:02.9120543Z [cluster2]                                                         "backend_config": {
2021-07-16T18:15:02.9121288Z [cluster2]                                                             "additionalProperties": {
2021-07-16T18:15:02.9122029Z [cluster2]                                                                 "type": "string"
2021-07-16T18:15:02.9122719Z [cluster2]                                                             },
2021-07-16T18:15:02.9123346Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9124233Z [cluster2]                                                         },
2021-07-16T18:15:02.9124776Z [cluster2]                                                         "cable_name": {
2021-07-16T18:15:02.9125380Z [cluster2]                                                             "type": "string"
2021-07-16T18:15:02.9125947Z [cluster2]                                                         },
2021-07-16T18:15:02.9126487Z [cluster2]                                                         "cluster_id": {
2021-07-16T18:15:02.9127092Z [cluster2]                                                             "type": "string"
2021-07-16T18:15:02.9127636Z [cluster2]                                                         },
2021-07-16T18:15:02.9128242Z [cluster2]                                                         "healthCheckIP": {
2021-07-16T18:15:02.9128892Z [cluster2]                                                             "type": "string"
2021-07-16T18:15:02.9129459Z [cluster2]                                                         },
2021-07-16T18:15:02.9130012Z [cluster2]                                                         "hostname": {
2021-07-16T18:15:02.9130604Z [cluster2]                                                             "type": "string"
2021-07-16T18:15:02.9131165Z [cluster2]                                                         },
2021-07-16T18:15:02.9131718Z [cluster2]                                                         "nat_enabled": {
2021-07-16T18:15:02.9132339Z [cluster2]                                                             "type": "boolean"
2021-07-16T18:15:02.9132889Z [cluster2]                                                         },
2021-07-16T18:15:02.9133449Z [cluster2]                                                         "private_ip": {
2021-07-16T18:15:02.9134064Z [cluster2]                                                             "type": "string"
2021-07-16T18:15:02.9134609Z [cluster2]                                                         },
2021-07-16T18:15:02.9135153Z [cluster2]                                                         "public_ip": {
2021-07-16T18:15:02.9135741Z [cluster2]                                                             "type": "string"
2021-07-16T18:15:02.9136291Z [cluster2]                                                         },
2021-07-16T18:15:02.9136909Z [cluster2]                                                         "subnets": {
2021-07-16T18:15:02.9137492Z [cluster2]                                                             "items": {
2021-07-16T18:15:02.9138090Z [cluster2]                                                                 "type": "string"
2021-07-16T18:15:02.9138639Z [cluster2]                                                             },
2021-07-16T18:15:02.9139200Z [cluster2]                                                             "type": "array"
2021-07-16T18:15:02.9139740Z [cluster2]                                                         }
2021-07-16T18:15:02.9140250Z [cluster2]                                                     },
2021-07-16T18:15:02.9140788Z [cluster2]                                                     "required": [
2021-07-16T18:15:02.9141377Z [cluster2]                                                         "backend",
2021-07-16T18:15:02.9141956Z [cluster2]                                                         "cable_name",
2021-07-16T18:15:02.9142635Z [cluster2]                                                         "cluster_id",
2021-07-16T18:15:02.9143220Z [cluster2]                                                         "hostname",
2021-07-16T18:15:02.9143806Z [cluster2]                                                         "nat_enabled",
2021-07-16T18:15:02.9144410Z [cluster2]                                                         "private_ip",
2021-07-16T18:15:02.9144980Z [cluster2]                                                         "public_ip",
2021-07-16T18:15:02.9145633Z [cluster2]                                                         "subnets"
2021-07-16T18:15:02.9146176Z [cluster2]                                                     ],
2021-07-16T18:15:02.9146716Z [cluster2]                                                     "type": "object"
2021-07-16T18:15:02.9147265Z [cluster2]                                                 },
2021-07-16T18:15:02.9147840Z [cluster2]                                                 "statusFailure": {
2021-07-16T18:15:02.9148482Z [cluster2]                                                     "type": "string"
2021-07-16T18:15:02.9149015Z [cluster2]                                                 },
2021-07-16T18:15:02.9149553Z [cluster2]                                                 "version": {
2021-07-16T18:15:02.9150131Z [cluster2]                                                     "type": "string"
2021-07-16T18:15:02.9150680Z [cluster2]                                                 }
2021-07-16T18:15:02.9151176Z [cluster2]                                             },
2021-07-16T18:15:02.9151698Z [cluster2]                                             "required": [
2021-07-16T18:15:02.9152308Z [cluster2]                                                 "connections",
2021-07-16T18:15:02.9152897Z [cluster2]                                                 "haStatus",
2021-07-16T18:15:02.9153513Z [cluster2]                                                 "localEndpoint",
2021-07-16T18:15:02.9154166Z [cluster2]                                                 "statusFailure",
2021-07-16T18:15:02.9154784Z [cluster2]                                                 "version"
2021-07-16T18:15:02.9155307Z [cluster2]                                             ],
2021-07-16T18:15:02.9155835Z [cluster2]                                             "type": "object"
2021-07-16T18:15:02.9156368Z [cluster2]                                         },
2021-07-16T18:15:02.9156882Z [cluster2]                                         "type": "array"
2021-07-16T18:15:02.9157401Z [cluster2]                                     },
2021-07-16T18:15:02.9157931Z [cluster2]                                     "globalCIDR": {
2021-07-16T18:15:02.9158523Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.9159040Z [cluster2]                                     },
2021-07-16T18:15:02.9159763Z [cluster2]                                     "globalnetDaemonSetStatus": {
2021-07-16T18:15:02.9160618Z [cluster2]                                         "properties": {
2021-07-16T18:15:02.9161334Z [cluster2]                                             "lastResourceVersion": {
2021-07-16T18:15:02.9162043Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.9162572Z [cluster2]                                             },
2021-07-16T18:15:02.9163427Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-16T18:15:02.9164404Z [cluster2]                                                 "type": "boolean"
2021-07-16T18:15:02.9164951Z [cluster2]                                             },
2021-07-16T18:15:02.9165665Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-16T18:15:02.9166400Z [cluster2]                                                 "items": {
2021-07-16T18:15:02.9167783Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-16T18:15:02.9169170Z [cluster2]                                                     "properties": {
2021-07-16T18:15:02.9169782Z [cluster2]                                                         "running": {
2021-07-16T18:15:02.9170548Z [cluster2]                                                             "description": "Details about a running container",
2021-07-16T18:15:02.9171402Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9172044Z [cluster2]                                                                 "startedAt": {
2021-07-16T18:15:02.9173527Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-16T18:15:02.9174596Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9175250Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9175809Z [cluster2]                                                                 }
2021-07-16T18:15:02.9176330Z [cluster2]                                                             },
2021-07-16T18:15:02.9176885Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9177444Z [cluster2]                                                         },
2021-07-16T18:15:02.9178007Z [cluster2]                                                         "terminated": {
2021-07-16T18:15:02.9178815Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-16T18:15:02.9179617Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9180270Z [cluster2]                                                                 "containerID": {
2021-07-16T18:15:02.9181461Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-16T18:15:02.9182310Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9182875Z [cluster2]                                                                 },
2021-07-16T18:15:02.9183459Z [cluster2]                                                                 "exitCode": {
2021-07-16T18:15:02.9184304Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-16T18:15:02.9185146Z [cluster2]                                                                     "format": "int32",
2021-07-16T18:15:02.9185859Z [cluster2]                                                                     "type": "integer"
2021-07-16T18:15:02.9186450Z [cluster2]                                                                 },
2021-07-16T18:15:02.9187023Z [cluster2]                                                                 "finishedAt": {
2021-07-16T18:15:02.9187866Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-16T18:15:02.9188940Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9189581Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9190155Z [cluster2]                                                                 },
2021-07-16T18:15:02.9190711Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.9191595Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-16T18:15:02.9192468Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9193025Z [cluster2]                                                                 },
2021-07-16T18:15:02.9193580Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.9194418Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-16T18:15:02.9195365Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9195933Z [cluster2]                                                                 },
2021-07-16T18:15:02.9196468Z [cluster2]                                                                 "signal": {
2021-07-16T18:15:02.9197293Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-16T18:15:02.9198103Z [cluster2]                                                                     "format": "int32",
2021-07-16T18:15:02.9198735Z [cluster2]                                                                     "type": "integer"
2021-07-16T18:15:02.9199303Z [cluster2]                                                                 },
2021-07-16T18:15:02.9199924Z [cluster2]                                                                 "startedAt": {
2021-07-16T18:15:02.9200817Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-16T18:15:02.9201942Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9202590Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9203151Z [cluster2]                                                                 }
2021-07-16T18:15:02.9203957Z [cluster2]                                                             },
2021-07-16T18:15:02.9204535Z [cluster2]                                                             "required": [
2021-07-16T18:15:02.9205264Z [cluster2]                                                                 "exitCode"
2021-07-16T18:15:02.9205827Z [cluster2]                                                             ],
2021-07-16T18:15:02.9206388Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9206949Z [cluster2]                                                         },
2021-07-16T18:15:02.9207484Z [cluster2]                                                         "waiting": {
2021-07-16T18:15:02.9208246Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-16T18:15:02.9209163Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9209782Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.9210642Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-16T18:15:02.9211474Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9212044Z [cluster2]                                                                 },
2021-07-16T18:15:02.9212600Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.9213389Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-16T18:15:02.9214199Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9214754Z [cluster2]                                                                 }
2021-07-16T18:15:02.9215272Z [cluster2]                                                             },
2021-07-16T18:15:02.9215824Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9216390Z [cluster2]                                                         }
2021-07-16T18:15:02.9216902Z [cluster2]                                                     },
2021-07-16T18:15:02.9217527Z [cluster2]                                                     "type": "object"
2021-07-16T18:15:02.9218073Z [cluster2]                                                 },
2021-07-16T18:15:02.9218598Z [cluster2]                                                 "type": "array"
2021-07-16T18:15:02.9219135Z [cluster2]                                             },
2021-07-16T18:15:02.9219637Z [cluster2]                                             "status": {
2021-07-16T18:15:02.9220542Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-16T18:15:02.9221563Z [cluster2]                                                 "properties": {
2021-07-16T18:15:02.9222218Z [cluster2]                                                     "collisionCount": {
2021-07-16T18:15:02.9223715Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-16T18:15:02.9225119Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9225733Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9226297Z [cluster2]                                                     },
2021-07-16T18:15:02.9226854Z [cluster2]                                                     "conditions": {
2021-07-16T18:15:02.9228198Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-16T18:15:02.9229124Z [cluster2]                                                         "items": {
2021-07-16T18:15:02.9230096Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-16T18:15:02.9231103Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9231822Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-16T18:15:02.9232824Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-16T18:15:02.9234039Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9234704Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9235277Z [cluster2]                                                                 },
2021-07-16T18:15:02.9235829Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.9236759Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-16T18:15:02.9237656Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9238231Z [cluster2]                                                                 },
2021-07-16T18:15:02.9238769Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.9239863Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-16T18:15:02.9240683Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9241236Z [cluster2]                                                                 },
2021-07-16T18:15:02.9241788Z [cluster2]                                                                 "status": {
2021-07-16T18:15:02.9242684Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-16T18:15:02.9243525Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9248517Z [cluster2]                                                                 },
2021-07-16T18:15:02.9249125Z [cluster2]                                                                 "type": {
2021-07-16T18:15:02.9250007Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-16T18:15:02.9250803Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9271812Z [cluster2]                                                                 }
2021-07-16T18:15:02.9272348Z [cluster2]                                                             },
2021-07-16T18:15:02.9272940Z [cluster2]                                                             "required": [
2021-07-16T18:15:02.9273550Z [cluster2]                                                                 "status",
2021-07-16T18:15:02.9274113Z [cluster2]                                                                 "type"
2021-07-16T18:15:02.9274660Z [cluster2]                                                             ],
2021-07-16T18:15:02.9275225Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9275792Z [cluster2]                                                         },
2021-07-16T18:15:02.9276330Z [cluster2]                                                         "type": "array"
2021-07-16T18:15:02.9276879Z [cluster2]                                                     },
2021-07-16T18:15:02.9277598Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-16T18:15:02.9279163Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.9280557Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9282186Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9282981Z [cluster2]                                                     },
2021-07-16T18:15:02.9283864Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-16T18:15:02.9285508Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.9286984Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9287597Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9288166Z [cluster2]                                                     },
2021-07-16T18:15:02.9288789Z [cluster2]                                                     "numberAvailable": {
2021-07-16T18:15:02.9290120Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.9292022Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9292632Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9293200Z [cluster2]                                                     },
2021-07-16T18:15:02.9293963Z [cluster2]                                                     "numberMisscheduled": {
2021-07-16T18:15:02.9295444Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.9296820Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9297430Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9298646Z [cluster2]                                                     },
2021-07-16T18:15:02.9299390Z [cluster2]                                                     "numberReady": {
2021-07-16T18:15:02.9300482Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-16T18:15:02.9301500Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9302100Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9302668Z [cluster2]                                                     },
2021-07-16T18:15:02.9303295Z [cluster2]                                                     "numberUnavailable": {
2021-07-16T18:15:02.9304896Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.9306149Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9306750Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9307313Z [cluster2]                                                     },
2021-07-16T18:15:02.9307963Z [cluster2]                                                     "observedGeneration": {
2021-07-16T18:15:02.9309527Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-16T18:15:02.9310420Z [cluster2]                                                         "format": "int64",
2021-07-16T18:15:02.9311126Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9311698Z [cluster2]                                                     },
2021-07-16T18:15:02.9312395Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-16T18:15:02.9313418Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-16T18:15:02.9314253Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9314866Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9315438Z [cluster2]                                                     }
2021-07-16T18:15:02.9315932Z [cluster2]                                                 },
2021-07-16T18:15:02.9316484Z [cluster2]                                                 "required": [
2021-07-16T18:15:02.9317223Z [cluster2]                                                     "currentNumberScheduled",
2021-07-16T18:15:02.9318125Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-16T18:15:02.9318967Z [cluster2]                                                     "numberMisscheduled",
2021-07-16T18:15:02.9319661Z [cluster2]                                                     "numberReady"
2021-07-16T18:15:02.9320231Z [cluster2]                                                 ],
2021-07-16T18:15:02.9320835Z [cluster2]                                                 "type": "object"
2021-07-16T18:15:02.9321389Z [cluster2]                                             }
2021-07-16T18:15:02.9321864Z [cluster2]                                         },
2021-07-16T18:15:02.9322396Z [cluster2]                                         "required": [
2021-07-16T18:15:02.9323181Z [cluster2]                                             "mismatchedContainerImages"
2021-07-16T18:15:02.9324099Z [cluster2]                                         ],
2021-07-16T18:15:02.9324645Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.9325163Z [cluster2]                                     },
2021-07-16T18:15:02.9325788Z [cluster2]                                     "loadBalancerStatus": {
2021-07-16T18:15:02.9326471Z [cluster2]                                         "properties": {
2021-07-16T18:15:02.9327058Z [cluster2]                                             "status": {
2021-07-16T18:15:02.9328599Z [cluster2]                                                 "description": "LoadBalancerStatus represents the status of a load-balancer.",
2021-07-16T18:15:02.9329573Z [cluster2]                                                 "properties": {
2021-07-16T18:15:02.9330444Z [cluster2]                                                     "ingress": {
2021-07-16T18:15:02.9333823Z [cluster2]                                                         "description": "Ingress is a list containing ingress points for the load-balancer. Traffic intended for the service should be sent to these ingress points.",
2021-07-16T18:15:02.9335359Z [cluster2]                                                         "items": {
2021-07-16T18:15:02.9337026Z [cluster2]                                                             "description": "LoadBalancerIngress represents the status of a load-balancer ingress point: traffic intended for the service should be sent to an ingress point.",
2021-07-16T18:15:02.9338330Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9338976Z [cluster2]                                                                 "hostname": {
2021-07-16T18:15:02.9340697Z [cluster2]                                                                     "description": "Hostname is set for load-balancer ingress points that are DNS based (typically AWS load-balancers)",
2021-07-16T18:15:02.9341925Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9342512Z [cluster2]                                                                 },
2021-07-16T18:15:02.9343043Z [cluster2]                                                                 "ip": {
2021-07-16T18:15:02.9344460Z [cluster2]                                                                     "description": "IP is set for load-balancer ingress points that are IP based (typically GCE or OpenStack load-balancers)",
2021-07-16T18:15:02.9345546Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9346114Z [cluster2]                                                                 }
2021-07-16T18:15:02.9346646Z [cluster2]                                                             },
2021-07-16T18:15:02.9347224Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9347779Z [cluster2]                                                         },
2021-07-16T18:15:02.9348342Z [cluster2]                                                         "type": "array"
2021-07-16T18:15:02.9348879Z [cluster2]                                                     }
2021-07-16T18:15:02.9349385Z [cluster2]                                                 },
2021-07-16T18:15:02.9349923Z [cluster2]                                                 "type": "object"
2021-07-16T18:15:02.9350470Z [cluster2]                                             }
2021-07-16T18:15:02.9351045Z [cluster2]                                         },
2021-07-16T18:15:02.9351574Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.9352105Z [cluster2]                                     },
2021-07-16T18:15:02.9352631Z [cluster2]                                     "natEnabled": {
2021-07-16T18:15:02.9353229Z [cluster2]                                         "type": "boolean"
2021-07-16T18:15:02.9353750Z [cluster2]                                     },
2021-07-16T18:15:02.9354329Z [cluster2]                                     "networkPlugin": {
2021-07-16T18:15:02.9354940Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.9355467Z [cluster2]                                     },
2021-07-16T18:15:02.9356205Z [cluster2]                                     "routeAgentDaemonSetStatus": {
2021-07-16T18:15:02.9356999Z [cluster2]                                         "properties": {
2021-07-16T18:15:02.9357729Z [cluster2]                                             "lastResourceVersion": {
2021-07-16T18:15:02.9358431Z [cluster2]                                                 "type": "string"
2021-07-16T18:15:02.9358969Z [cluster2]                                             },
2021-07-16T18:15:02.9359693Z [cluster2]                                             "mismatchedContainerImages": {
2021-07-16T18:15:02.9360506Z [cluster2]                                                 "type": "boolean"
2021-07-16T18:15:02.9361058Z [cluster2]                                             },
2021-07-16T18:15:02.9361754Z [cluster2]                                             "nonReadyContainerStates": {
2021-07-16T18:15:02.9362501Z [cluster2]                                                 "items": {
2021-07-16T18:15:02.9364094Z [cluster2]                                                     "description": "ContainerState holds a possible state of container. Only one of its members may be specified. If none of them is specified, the default one is ContainerStateWaiting.",
2021-07-16T18:15:02.9365513Z [cluster2]                                                     "properties": {
2021-07-16T18:15:02.9366123Z [cluster2]                                                         "running": {
2021-07-16T18:15:02.9366870Z [cluster2]                                                             "description": "Details about a running container",
2021-07-16T18:15:02.9367657Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9368374Z [cluster2]                                                                 "startedAt": {
2021-07-16T18:15:02.9369534Z [cluster2]                                                                     "description": "Time at which the container was last (re-)started",
2021-07-16T18:15:02.9370594Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9371234Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9371811Z [cluster2]                                                                 }
2021-07-16T18:15:02.9372323Z [cluster2]                                                             },
2021-07-16T18:15:02.9372900Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9373454Z [cluster2]                                                         },
2021-07-16T18:15:02.9374030Z [cluster2]                                                         "terminated": {
2021-07-16T18:15:02.9374841Z [cluster2]                                                             "description": "Details about a terminated container",
2021-07-16T18:15:02.9375630Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9376298Z [cluster2]                                                                 "containerID": {
2021-07-16T18:15:02.9377485Z [cluster2]                                                                     "description": "Container's ID in the format 'docker://\u003ccontainer_id\u003e'",
2021-07-16T18:15:02.9378439Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9379015Z [cluster2]                                                                 },
2021-07-16T18:15:02.9379579Z [cluster2]                                                                 "exitCode": {
2021-07-16T18:15:02.9380443Z [cluster2]                                                                     "description": "Exit status from the last termination of the container",
2021-07-16T18:15:02.9381272Z [cluster2]                                                                     "format": "int32",
2021-07-16T18:15:02.9381904Z [cluster2]                                                                     "type": "integer"
2021-07-16T18:15:02.9382466Z [cluster2]                                                                 },
2021-07-16T18:15:02.9383061Z [cluster2]                                                                 "finishedAt": {
2021-07-16T18:15:02.9383900Z [cluster2]                                                                     "description": "Time at which the container last terminated",
2021-07-16T18:15:02.9384952Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9385615Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9386173Z [cluster2]                                                                 },
2021-07-16T18:15:02.9386741Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.9387624Z [cluster2]                                                                     "description": "Message regarding the last termination of the container",
2021-07-16T18:15:02.9388481Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9389057Z [cluster2]                                                                 },
2021-07-16T18:15:02.9389601Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.9390458Z [cluster2]                                                                     "description": "(brief) reason from the last termination of the container",
2021-07-16T18:15:02.9391412Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9391983Z [cluster2]                                                                 },
2021-07-16T18:15:02.9392529Z [cluster2]                                                                 "signal": {
2021-07-16T18:15:02.9393340Z [cluster2]                                                                     "description": "Signal from the last termination of the container",
2021-07-16T18:15:02.9394168Z [cluster2]                                                                     "format": "int32",
2021-07-16T18:15:02.9394785Z [cluster2]                                                                     "type": "integer"
2021-07-16T18:15:02.9395370Z [cluster2]                                                                 },
2021-07-16T18:15:02.9395941Z [cluster2]                                                                 "startedAt": {
2021-07-16T18:15:02.9396825Z [cluster2]                                                                     "description": "Time at which previous execution of the container started",
2021-07-16T18:15:02.9397963Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9398604Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9399175Z [cluster2]                                                                 }
2021-07-16T18:15:02.9400133Z [cluster2]                                                             },
2021-07-16T18:15:02.9400691Z [cluster2]                                                             "required": [
2021-07-16T18:15:02.9401305Z [cluster2]                                                                 "exitCode"
2021-07-16T18:15:02.9401856Z [cluster2]                                                             ],
2021-07-16T18:15:02.9402710Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9403467Z [cluster2]                                                         },
2021-07-16T18:15:02.9404171Z [cluster2]                                                         "waiting": {
2021-07-16T18:15:02.9404937Z [cluster2]                                                             "description": "Details about a waiting container",
2021-07-16T18:15:02.9405706Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9406337Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.9407179Z [cluster2]                                                                     "description": "Message regarding why the container is not yet running.",
2021-07-16T18:15:02.9408040Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9408613Z [cluster2]                                                                 },
2021-07-16T18:15:02.9409164Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.9409972Z [cluster2]                                                                     "description": "(brief) reason the container is not yet running.",
2021-07-16T18:15:02.9410965Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9411684Z [cluster2]                                                                 }
2021-07-16T18:15:02.9412220Z [cluster2]                                                             },
2021-07-16T18:15:02.9412790Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9413346Z [cluster2]                                                         }
2021-07-16T18:15:02.9413844Z [cluster2]                                                     },
2021-07-16T18:15:02.9414509Z [cluster2]                                                     "type": "object"
2021-07-16T18:15:02.9415053Z [cluster2]                                                 },
2021-07-16T18:15:02.9415599Z [cluster2]                                                 "type": "array"
2021-07-16T18:15:02.9416278Z [cluster2]                                             },
2021-07-16T18:15:02.9416814Z [cluster2]                                             "status": {
2021-07-16T18:15:02.9418335Z [cluster2]                                                 "description": "DaemonSetStatus represents the current status of a daemon set.",
2021-07-16T18:15:02.9419749Z [cluster2]                                                 "properties": {
2021-07-16T18:15:02.9420563Z [cluster2]                                                     "collisionCount": {
2021-07-16T18:15:02.9422493Z [cluster2]                                                         "description": "Count of hash collisions for the DaemonSet. The DaemonSet controller uses this field as a collision avoidance mechanism when it needs to create the name for the newest ControllerRevision.",
2021-07-16T18:15:02.9425548Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9426335Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9427353Z [cluster2]                                                     },
2021-07-16T18:15:02.9427923Z [cluster2]                                                     "conditions": {
2021-07-16T18:15:02.9429493Z [cluster2]                                                         "description": "Represents the latest available observations of a DaemonSet's current state.",
2021-07-16T18:15:02.9430431Z [cluster2]                                                         "items": {
2021-07-16T18:15:02.9431404Z [cluster2]                                                             "description": "DaemonSetCondition describes the state of a DaemonSet at a certain point.",
2021-07-16T18:15:02.9432405Z [cluster2]                                                             "properties": {
2021-07-16T18:15:02.9433131Z [cluster2]                                                                 "lastTransitionTime": {
2021-07-16T18:15:02.9434119Z [cluster2]                                                                     "description": "Last time the condition transitioned from one status to another.",
2021-07-16T18:15:02.9435262Z [cluster2]                                                                     "format": "date-time",
2021-07-16T18:15:02.9436974Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9437554Z [cluster2]                                                                 },
2021-07-16T18:15:02.9438120Z [cluster2]                                                                 "message": {
2021-07-16T18:15:02.9439030Z [cluster2]                                                                     "description": "A human readable message indicating details about the transition.",
2021-07-16T18:15:02.9439947Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9440515Z [cluster2]                                                                 },
2021-07-16T18:15:02.9441053Z [cluster2]                                                                 "reason": {
2021-07-16T18:15:02.9442552Z [cluster2]                                                                     "description": "The reason for the condition's last transition.",
2021-07-16T18:15:02.9443364Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9444064Z [cluster2]                                                                 },
2021-07-16T18:15:02.9444605Z [cluster2]                                                                 "status": {
2021-07-16T18:15:02.9445810Z [cluster2]                                                                     "description": "Status of the condition, one of True, False, Unknown.",
2021-07-16T18:15:02.9446655Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9447209Z [cluster2]                                                                 },
2021-07-16T18:15:02.9447755Z [cluster2]                                                                 "type": {
2021-07-16T18:15:02.9448496Z [cluster2]                                                                     "description": "Type of DaemonSet condition.",
2021-07-16T18:15:02.9449635Z [cluster2]                                                                     "type": "string"
2021-07-16T18:15:02.9450223Z [cluster2]                                                                 }
2021-07-16T18:15:02.9450893Z [cluster2]                                                             },
2021-07-16T18:15:02.9451465Z [cluster2]                                                             "required": [
2021-07-16T18:15:02.9452052Z [cluster2]                                                                 "status",
2021-07-16T18:15:02.9452623Z [cluster2]                                                                 "type"
2021-07-16T18:15:02.9453144Z [cluster2]                                                             ],
2021-07-16T18:15:02.9453713Z [cluster2]                                                             "type": "object"
2021-07-16T18:15:02.9454276Z [cluster2]                                                         },
2021-07-16T18:15:02.9454916Z [cluster2]                                                         "type": "array"
2021-07-16T18:15:02.9455462Z [cluster2]                                                     },
2021-07-16T18:15:02.9456153Z [cluster2]                                                     "currentNumberScheduled": {
2021-07-16T18:15:02.9457733Z [cluster2]                                                         "description": "The number of nodes that are running at least 1 daemon pod and are supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.9459119Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9459725Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9460290Z [cluster2]                                                     },
2021-07-16T18:15:02.9460997Z [cluster2]                                                     "desiredNumberScheduled": {
2021-07-16T18:15:02.9462637Z [cluster2]                                                         "description": "The total number of nodes that should be running the daemon pod (including nodes correctly running the daemon pod). More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.9464093Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9464710Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9465275Z [cluster2]                                                     },
2021-07-16T18:15:02.9465872Z [cluster2]                                                     "numberAvailable": {
2021-07-16T18:15:02.9467210Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.9468472Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9469070Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9469640Z [cluster2]                                                     },
2021-07-16T18:15:02.9470283Z [cluster2]                                                     "numberMisscheduled": {
2021-07-16T18:15:02.9471843Z [cluster2]                                                         "description": "The number of nodes that are running the daemon pod, but are not supposed to run the daemon pod. More info: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/",
2021-07-16T18:15:02.9473221Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9473830Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9474404Z [cluster2]                                                     },
2021-07-16T18:15:02.9474972Z [cluster2]                                                     "numberReady": {
2021-07-16T18:15:02.9476023Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have one or more of the daemon pod running and ready.",
2021-07-16T18:15:02.9477015Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9477634Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9478198Z [cluster2]                                                     },
2021-07-16T18:15:02.9478816Z [cluster2]                                                     "numberUnavailable": {
2021-07-16T18:15:02.9480157Z [cluster2]                                                         "description": "The number of nodes that should be running the daemon pod and have none of the daemon pod running and available (ready for at least spec.minReadySeconds)",
2021-07-16T18:15:02.9481452Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9482067Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9482634Z [cluster2]                                                     },
2021-07-16T18:15:02.9483279Z [cluster2]                                                     "observedGeneration": {
2021-07-16T18:15:02.9484485Z [cluster2]                                                         "description": "The most recent generation observed by the daemon set controller.",
2021-07-16T18:15:02.9485365Z [cluster2]                                                         "format": "int64",
2021-07-16T18:15:02.9485978Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9486544Z [cluster2]                                                     },
2021-07-16T18:15:02.9487251Z [cluster2]                                                     "updatedNumberScheduled": {
2021-07-16T18:15:02.9488264Z [cluster2]                                                         "description": "The total number of nodes that are running updated daemon pod",
2021-07-16T18:15:02.9489779Z [cluster2]                                                         "format": "int32",
2021-07-16T18:15:02.9490412Z [cluster2]                                                         "type": "integer"
2021-07-16T18:15:02.9490964Z [cluster2]                                                     }
2021-07-16T18:15:02.9491471Z [cluster2]                                                 },
2021-07-16T18:15:02.9492018Z [cluster2]                                                 "required": [
2021-07-16T18:15:02.9492748Z [cluster2]                                                     "currentNumberScheduled",
2021-07-16T18:15:02.9493643Z [cluster2]                                                     "desiredNumberScheduled",
2021-07-16T18:15:02.9494476Z [cluster2]                                                     "numberMisscheduled",
2021-07-16T18:15:02.9495189Z [cluster2]                                                     "numberReady"
2021-07-16T18:15:02.9495745Z [cluster2]                                                 ],
2021-07-16T18:15:02.9496302Z [cluster2]                                                 "type": "object"
2021-07-16T18:15:02.9496938Z [cluster2]                                             }
2021-07-16T18:15:02.9497419Z [cluster2]                                         },
2021-07-16T18:15:02.9497956Z [cluster2]                                         "required": [
2021-07-16T18:15:02.9498720Z [cluster2]                                             "mismatchedContainerImages"
2021-07-16T18:15:02.9499449Z [cluster2]                                         ],
2021-07-16T18:15:02.9500009Z [cluster2]                                         "type": "object"
2021-07-16T18:15:02.9500547Z [cluster2]                                     },
2021-07-16T18:15:02.9501088Z [cluster2]                                     "serviceCIDR": {
2021-07-16T18:15:02.9501703Z [cluster2]                                         "type": "string"
2021-07-16T18:15:02.9502233Z [cluster2]                                     }
2021-07-16T18:15:02.9502692Z [cluster2]                                 },
2021-07-16T18:15:02.9503214Z [cluster2]                                 "required": [
2021-07-16T18:15:02.9503780Z [cluster2]                                     "clusterID",
2021-07-16T18:15:02.9504364Z [cluster2]                                     "natEnabled"
2021-07-16T18:15:02.9504873Z [cluster2]                                 ],
2021-07-16T18:15:04.3558698Z [cluster2]                                 "type": "object"
2021-07-16T18:15:04.3559276Z [cluster2]                             }
2021-07-16T18:15:04.3559728Z [cluster2]                         },
2021-07-16T18:15:04.3560507Z [cluster2]                         "type": "object"
2021-07-16T18:15:04.3560996Z [cluster2]                     }
2021-07-16T18:15:04.3561448Z [cluster2]                 },
2021-07-16T18:15:04.3561935Z [cluster2]                 "served": true,
2021-07-16T18:15:04.3562495Z [cluster2]                 "storage": true,
2021-07-16T18:15:04.3563069Z [cluster2]                 "subresources": {
2021-07-16T18:15:04.3564338Z [cluster2]                     "status": {}
2021-07-16T18:15:04.3564826Z [cluster2]                 }
2021-07-16T18:15:04.3565261Z [cluster2]             }
2021-07-16T18:15:04.3565695Z [cluster2]         ]
2021-07-16T18:15:04.3566106Z [cluster2]     },
2021-07-16T18:15:04.3566560Z [cluster2]     "status": {
2021-07-16T18:15:04.3567098Z [cluster2]         "acceptedNames": {
2021-07-16T18:15:04.3567795Z [cluster2]             "kind": "Submariner",
2021-07-16T18:15:04.3568432Z [cluster2]             "listKind": "SubmarinerList",
2021-07-16T18:15:04.3569109Z [cluster2]             "plural": "submariners",
2021-07-16T18:15:04.3569827Z [cluster2]             "singular": "submariner"
2021-07-16T18:15:04.3570334Z [cluster2]         },
2021-07-16T18:15:04.3570815Z [cluster2]         "conditions": [
2021-07-16T18:15:04.3571291Z [cluster2]             {
2021-07-16T18:15:04.3572664Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:45Z",
2021-07-16T18:15:04.3573405Z [cluster2]                 "message": "no conflicts found",
2021-07-16T18:15:04.3574059Z [cluster2]                 "reason": "NoConflicts",
2021-07-16T18:15:04.3574633Z [cluster2]                 "status": "True",
2021-07-16T18:15:04.3575247Z [cluster2]                 "type": "NamesAccepted"
2021-07-16T18:15:04.3575815Z [cluster2]             },
2021-07-16T18:15:04.3576247Z [cluster2]             {
2021-07-16T18:15:04.3577067Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:45Z",
2021-07-16T18:15:04.3577891Z [cluster2]                 "message": "the initial names have been accepted",
2021-07-16T18:15:04.3578835Z [cluster2]                 "reason": "InitialNamesAccepted",
2021-07-16T18:15:04.3579514Z [cluster2]                 "status": "True",
2021-07-16T18:15:04.3580094Z [cluster2]                 "type": "Established"
2021-07-16T18:15:04.3580602Z [cluster2]             }
2021-07-16T18:15:04.3581028Z [cluster2]         ],
2021-07-16T18:15:04.3581549Z [cluster2]         "storedVersions": [
2021-07-16T18:15:04.3582129Z [cluster2]             "v1alpha1"
2021-07-16T18:15:04.3582579Z [cluster2]         ]
2021-07-16T18:15:04.3583162Z [cluster2]     }
2021-07-16T18:15:04.3583575Z [cluster2] }
2021-07-16T18:15:04.3584813Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariners.submariner.io
2021-07-16T18:15:04.3586573Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariners.submariner.io
2021-07-16T18:15:04.3588541Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/submariners.submariner.io.cluster2.json
2021-07-16T18:15:04.3590292Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-16T18:15:04.3591611Z [submariner-operator]$ [cluster2] validate_equals .spec.scope Namespaced
2021-07-16T18:15:04.3593366Z [submariner-operator]$ [cluster2] jq -r -M .spec.scope /tmp/submariners.submariner.io.cluster2.json
2021-07-16T18:15:04.3595229Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-16T18:15:04.3596597Z [submariner-operator]$ [cluster2] validate_equals .spec.group submariner.io
2021-07-16T18:15:04.3598375Z [submariner-operator]$ [cluster2] jq -r -M .spec.group /tmp/submariners.submariner.io.cluster2.json
2021-07-16T18:15:04.3600121Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Submariner
2021-07-16T18:15:04.3601534Z [submariner-operator]$ [cluster2] validate_equals .spec.names.kind Submariner
2021-07-16T18:15:04.3603350Z [submariner-operator]$ [cluster2] jq -r -M .spec.names.kind /tmp/submariners.submariner.io.cluster2.json
2021-07-16T18:15:04.3605334Z [submariner-operator]$ [cluster2] verify_subm_operator
2021-07-16T18:15:04.3606443Z [submariner-operator]$ [cluster2] verify_subm_operator
2021-07-16T18:15:04.3607670Z [submariner-operator]$ [cluster2] kubectl get ns submariner-operator
2021-07-16T18:15:04.3609028Z [submariner-operator]$ [cluster2] kubectl get ns submariner-operator
2021-07-16T18:15:04.3610521Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get ns submariner-operator
2021-07-16T18:15:04.3611523Z [cluster2] NAME                  STATUS   AGE
2021-07-16T18:15:04.3612349Z [cluster2] submariner-operator   Active   78s
2021-07-16T18:15:04.3613629Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-16T18:15:04.3615167Z [submariner-operator]$ [cluster2] kubectl get crds submariners.submariner.io
2021-07-16T18:15:04.3616838Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get crds submariners.submariner.io
2021-07-16T18:15:04.3618029Z [cluster2] NAME                        CREATED AT
2021-07-16T18:15:04.3618966Z [cluster2] submariners.submariner.io   2021-07-16T18:13:45Z
2021-07-16T18:15:04.3620113Z [submariner-operator]$ [cluster2] kubectl api-resources
2021-07-16T18:15:04.3621227Z [submariner-operator]$ [cluster2] grep submariners
2021-07-16T18:15:04.3622361Z [submariner-operator]$ [cluster2] kubectl api-resources
2021-07-16T18:15:04.3623665Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 api-resources
2021-07-16T18:15:04.3624806Z [cluster2] submariners                                    submariner.io/v1alpha1            true         Submariner
2021-07-16T18:15:04.6830695Z [submariner-operator]$ [cluscluster1]             "listKind": "SubmarinerList",
2021-07-16T18:15:04.6831722Z [cluster1]             "plural": "submariners",
2021-07-16T18:15:04.6832371Z [cluster1]             "singular": "submariner"
2021-07-16T18:15:04.6832933Z [cluster1]         },
2021-07-16T18:15:04.6833410Z [cluster1]         "conditions": [
2021-07-16T18:15:04.6833907Z [cluster1]             {
2021-07-16T18:15:04.6834742Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:33Z",
2021-07-16T18:15:04.6835498Z [cluster1]                 "message": "no conflicts found",
2021-07-16T18:15:04.6836140Z [cluster1]                 "reason": "NoConflicts",
2021-07-16T18:15:04.6836935Z [cluster1]                 "status": "True",
2021-07-16T18:15:04.6837538Z [cluster1]                 "type": "NamesAccepted"
2021-07-16T18:15:04.6838090Z [cluster1]             },
2021-07-16T18:15:04.6838512Z [cluster1]             {
2021-07-16T18:15:04.6839346Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:34Z",
2021-07-16T18:15:04.6840173Z [cluster1]                 "message": "the initial names have been accepted",
2021-07-16T18:15:04.6841003Z [cluster1]                 "reason": "InitialNamesAccepted",
2021-07-16T18:15:04.6841710Z [cluster1]                 "status": "True",
2021-07-16T18:15:04.6842271Z [cluster1]                 "type": "Established"
2021-07-16T18:15:04.6842791Z [cluster1]             }
2021-07-16T18:15:04.6843204Z [cluster1]         ],
2021-07-16T18:15:04.6844474Z [cluster1]         "storedVersions": [
2021-07-16T18:15:04.6845044Z [cluster1]             "v1alpha1"
2021-07-16T18:15:04.6845509Z [cluster1]         ]
2021-07-16T18:15:04.6845915Z [cluster1]     }
2021-07-16T18:15:04.6846343Z [cluster1] }
2021-07-16T18:15:04.6847574Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariners.submariner.io
2021-07-16T18:15:04.6849308Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariners.submariner.io
2021-07-16T18:15:04.6851285Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariners.submariner.io.cluster1.json
2021-07-16T18:15:04.6853024Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-16T18:15:04.6854546Z [submariner-operator]$ [cluster1] validate_equals .spec.scope Namespaced
2021-07-16T18:15:04.6856281Z [submariner-operator]$ [cluster1] jq -r -M .spec.scope /tmp/submariners.submariner.io.cluster1.json
2021-07-16T18:15:04.6858033Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-16T18:15:04.6859408Z [submariner-operator]$ [cluster1] validate_equals .spec.group submariner.io
2021-07-16T18:15:04.6861169Z [submariner-operator]$ [cluster1] jq -r -M .spec.group /tmp/submariners.submariner.io.cluster1.json
2021-07-16T18:15:04.6862926Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Submariner
2021-07-16T18:15:04.6864317Z [submariner-operator]$ [cluster1] validate_equals .spec.names.kind Submariner
2021-07-16T18:15:04.6866151Z [submariner-operator]$ [cluster1] jq -r -M .spec.names.kind /tmp/submariners.submariner.io.cluster1.json
2021-07-16T18:15:04.6867796Z [submariner-operator]$ [cluster1] verify_subm_operator
2021-07-16T18:15:04.6868882Z [submariner-operator]$ [cluster1] verify_subm_operator
2021-07-16T18:15:04.6870116Z [submariner-operator]$ [cluster1] kubectl get ns submariner-operator
2021-07-16T18:15:04.6871456Z [submariner-operator]$ [cluster1] kubectl get ns submariner-operator
2021-07-16T18:15:04.6872959Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get ns submariner-operator
2021-07-16T18:15:04.6873946Z [cluster1] NAME                  STATUS   AGE
2021-07-16T18:15:04.6874787Z [cluster1] submariner-operator   Active   89s
2021-07-16T18:15:04.6876044Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-16T18:15:04.6877588Z [submariner-operator]$ [cluster1] kubectl get crds submariners.submariner.io
2021-07-16T18:15:04.6879267Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get crds submariners.submariner.io
2021-07-16T18:15:04.6880341Z [cluster1] NAME                        CREATED AT
2021-07-16T18:15:04.6881281Z [cluster1] submariners.submariner.io   2021-07-16T18:13:33Z
2021-07-16T18:15:04.6882407Z [submariner-operator]$ [cluster1] kubectl api-resources
2021-07-16T18:15:04.6963806Z [submariner-operator]$ [cluster1] grep submariners
2021-07-16T18:15:04.6965643Z [submariner-operator]$ [cluster1] kubectl api-resources
2021-07-16T18:15:04.6967252Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 api-resources
2021-07-16T18:15:04.6968411Z [cluster1] submariners                                    submariner.io/v1alpha1            true         Submariner
2021-07-16T18:15:04.6969928Z [submariner-operator]$ [cluster1] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-16T18:15:04.6971838Z [submariner-operator]$ [cluster1] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-16T18:15:04.6973817Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get sa --namespace=submariner-operator submariner-operator
2021-07-16T18:15:04.6975038Z [cluster1] NAME                  SECRETS   AGE
2021-07-16T18:15:04.6975876Z [cluster1] submariner-operator   1         90s
2021-07-16T18:15:04.7014850Z [submariner-operator]$ [cluster1] kubectl get roles --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8361359Z [submariner-operator]$ [cluster1] kubectl get roles --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8364114Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get roles --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8365343Z [cluster1] NAME                  CREATED AT
2021-07-16T18:15:06.8366235Z [cluster1] submariner-operator   2021-07-16T18:13:34Z
2021-07-16T18:15:06.8367723Z [submariner-operator]$ [cluster1] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8369921Z [submariner-operator]$ [cluster1] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8380738Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get rolebindings --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8382070Z [cluster1] NAME                  ROLE                       AGE
2021-07-16T18:15:06.8383313Z [cluster1] submariner-operator   Role/submariner-operator   91s
2021-07-16T18:15:06.8384945Z [submariner-operator]$ [cluster1] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8387033Z [submariner-operator]$ [cluster1] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8389148Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get deployments --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.8390894Z [cluster1] NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
2021-07-16T18:15:06.8391876Z [cluster1] submariner-operator   1/1     1            1           85s
2021-07-16T18:15:06.8392884Z [submariner-operator]$ [cluster1] verify_subm_op_pod
2021-07-16T18:15:06.8393950Z [submariner-operator]$ [cluster1] verify_subm_op_pod
2021-07-16T18:15:06.8395758Z [submariner-operator]$ [cluster1] subm_operator_pod_name=$(kubectl get pods --namespace=$subm_ns -l name=$operator_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-16T18:15:06.8398036Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:06.8400378Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:06.8402835Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:06.8405425Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator -o json
2021-07-16T18:15:06.8407917Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator -o json
2021-07-16T18:15:06.8410632Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator -o json
2021-07-16T18:15:06.8412037Z [cluster1] {
2021-07-16T18:15:06.8412540Z [cluster1]     "apiVersion": "v1",
2021-07-16T18:15:06.8413063Z [cluster1]     "kind": "Pod",
2021-07-16T18:15:06.8413567Z [cluster1]     "metadata": {
2021-07-16T18:15:06.8414405Z [cluster1]         "creationTimestamp": "2021-07-16T18:13:40Z",
2021-07-16T18:15:06.8415597Z [cluster1]         "generateName": "submariner-operator-546678bc8f-",
2021-07-16T18:15:06.8416443Z [cluster1]         "labels": {
2021-07-16T18:15:06.8417246Z [cluster1]             "name": "submariner-operator",
2021-07-16T18:15:06.8418180Z [cluster1]             "pod-template-hash": "546678bc8f"
2021-07-16T18:15:06.8418746Z [cluster1]         },
2021-07-16T18:15:06.8419258Z [cluster1]         "managedFields": [
2021-07-16T18:15:06.8419767Z [cluster1]             {
2021-07-16T18:15:06.8420284Z [cluster1]                 "apiVersion": "v1",
2021-07-16T18:15:06.8420891Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:06.8421487Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:06.8422005Z [cluster1]                     "f:metadata": {
2021-07-16T18:15:06.8422719Z [cluster1]                         "f:generateName": {},
2021-07-16T18:15:06.8423297Z [cluster1]                         "f:labels": {
2021-07-16T18:15:06.8423804Z [cluster1]                             ".": {},
2021-07-16T18:15:06.8424418Z [cluster1]                             "f:name": {},
2021-07-16T18:15:06.8425243Z [cluster1]                             "f:pod-template-hash": {}
2021-07-16T18:15:06.8425844Z [cluster1]                         },
2021-07-16T18:15:06.8426418Z [cluster1]                         "f:ownerReferences": {
2021-07-16T18:15:06.8427013Z [cluster1]                             ".": {},
2021-07-16T18:15:06.8427911Z [cluster1]                             "k:{\"uid\":\"6e95e778-4152-4d14-a159-0eaa4da2893c\"}": {
2021-07-16T18:15:06.8428596Z [cluster1]                                 ".": {},
2021-07-16T18:15:06.8429128Z [cluster1]                                 "f:apiVersion": {},
2021-07-16T18:15:06.8429825Z [cluster1]                                 "f:blockOwnerDeletion": {},
2021-07-16T18:15:06.8430525Z [cluster1]                                 "f:controller": {},
2021-07-16T18:15:06.8431074Z [cluster1]                                 "f:kind": {},
2021-07-16T18:15:06.8431720Z [cluster1]                                 "f:name": {},
2021-07-16T18:15:06.8482832Z [cluster1]                                 "f:uid": {}
2021-07-16T18:15:06.8485523Z [cluster1]                             }
2021-07-16T18:15:06.8486013Z [cluster1]                         }
2021-07-16T18:15:06.8486459Z [cluster1]                     },
2021-07-16T18:15:06.8486948Z [cluster1]                     "f:spec": {
2021-07-16T18:15:06.8488312Z [cluster1]                         "f:containers": {
2021-07-16T18:15:06.8492284Z [cluster1]                             "k:{\"name\":\"submariner-operator\"}": {
2021-07-16T18:15:06.8492993Z [cluster1]                                 ".": {},
2021-07-16T18:15:06.8493550Z [cluster1]                                 "f:command": {},
2021-07-16T18:15:06.8494235Z [cluster1]                                 "f:env": {
2021-07-16T18:15:06.8494787Z [cluster1]                                     ".": {},
2021-07-16T18:15:06.8495430Z [cluster1]                                     "k:{\"name\":\"OPERATOR_NAME\"}": {
2021-07-16T18:15:06.8496792Z [cluster1]                                         ".": {},
2021-07-16T18:15:06.8497380Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:06.8497909Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:06.8498431Z [cluster1]                                     },
2021-07-16T18:15:06.8498996Z [cluster1]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-16T18:15:06.8499746Z [cluster1]                                         ".": {},
2021-07-16T18:15:06.8500655Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:06.8501386Z [cluster1]                                         "f:valueFrom": {
2021-07-16T18:15:06.8502196Z [cluster1]                                             ".": {},
2021-07-16T18:15:06.8502743Z [cluster1]                                             "f:fieldRef": {
2021-07-16T18:15:06.8503306Z [cluster1]                                                 ".": {},
2021-07-16T18:15:06.8504265Z [cluster1]                                                 "f:apiVersion": {},
2021-07-16T18:15:06.8504910Z [cluster1]                                                 "f:fieldPath": {}
2021-07-16T18:15:06.8505473Z [cluster1]                                             }
2021-07-16T18:15:06.8505946Z [cluster1]                                         }
2021-07-16T18:15:06.8506429Z [cluster1]                                     },
2021-07-16T18:15:06.8507031Z [cluster1]                                     "k:{\"name\":\"WATCH_NAMESPACE\"}": {
2021-07-16T18:15:06.8507642Z [cluster1]                                         ".": {},
2021-07-16T18:15:06.8508149Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:06.8508721Z [cluster1]                                         "f:valueFrom": {
2021-07-16T18:15:06.8509277Z [cluster1]                                             ".": {},
2021-07-16T18:15:06.8509940Z [cluster1]                                             "f:fieldRef": {
2021-07-16T18:15:06.8510638Z [cluster1]                                                 ".": {},
2021-07-16T18:15:06.8511206Z [cluster1]                                                 "f:apiVersion": {},
2021-07-16T18:15:06.8511843Z [cluster1]                                                 "f:fieldPath": {}
2021-07-16T18:15:06.8512396Z [cluster1]                                             }
2021-07-16T18:15:06.8512887Z [cluster1]                                         }
2021-07-16T18:15:06.8513365Z [cluster1]                                     }
2021-07-16T18:15:06.8513847Z [cluster1]                                 },
2021-07-16T18:15:06.8514354Z [cluster1]                                 "f:image": {},
2021-07-16T18:15:06.8514967Z [cluster1]                                 "f:imagePullPolicy": {},
2021-07-16T18:15:06.8515588Z [cluster1]                                 "f:name": {},
2021-07-16T18:15:06.8516134Z [cluster1]                                 "f:resources": {},
2021-07-16T18:15:06.8516886Z [cluster1]                                 "f:terminationMessagePath": {},
2021-07-16T18:15:06.8517797Z [cluster1]                                 "f:terminationMessagePolicy": {}
2021-07-16T18:15:06.8518511Z [cluster1]                             }
2021-07-16T18:15:06.8518956Z [cluster1]                         },
2021-07-16T18:15:06.8519486Z [cluster1]                         "f:dnsPolicy": {},
2021-07-16T18:15:06.8520170Z [cluster1]                         "f:enableServiceLinks": {},
2021-07-16T18:15:06.8520876Z [cluster1]                         "f:restartPolicy": {},
2021-07-16T18:15:06.8521525Z [cluster1]                         "f:schedulerName": {},
2021-07-16T18:15:06.8522190Z [cluster1]                         "f:securityContext": {},
2021-07-16T18:15:06.8522878Z [cluster1]                         "f:serviceAccount": {},
2021-07-16T18:15:06.8523697Z [cluster1]                         "f:serviceAccountName": {},
2021-07-16T18:15:06.8524654Z [cluster1]                         "f:terminationGracePeriodSeconds": {}
2021-07-16T18:15:06.8525440Z [cluster1]                     }
2021-07-16T18:15:06.8525891Z [cluster1]                 },
2021-07-16T18:15:06.8526957Z [cluster1]                 "manager": "kube-controller-manager",
2021-07-16T18:15:06.8527713Z [cluster1]                 "operation": "Update",
2021-07-16T18:15:06.8528482Z [cluster1]                 "time": "2021-07-16T18:13:40Z"
2021-07-16T18:15:06.8528963Z [cluster1]             },
2021-07-16T18:15:06.8529402Z [cluster1]             {
2021-07-16T18:15:06.8530002Z [cluster1]                 "apiVersion": "v1",
2021-07-16T18:15:06.8530634Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:06.8531215Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:06.8531855Z [cluster1]                     "f:status": {
2021-07-16T18:15:06.8532400Z [cluster1]                         "f:conditions": {
2021-07-16T18:15:06.8533204Z [cluster1]                             "k:{\"type\":\"ContainersReady\"}": {
2021-07-16T18:15:06.8533843Z [cluster1]                                 ".": {},
2021-07-16T18:15:06.8534417Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-16T18:15:06.8535275Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:06.8535928Z [cluster1]                                 "f:status": {},
2021-07-16T18:15:06.8536470Z [cluster1]                                 "f:type": {}
2021-07-16T18:15:06.8536951Z [cluster1]                             },
2021-07-16T18:15:06.8537530Z [cluster1]                             "k:{\"type\":\"Initialized\"}": {
2021-07-16T18:15:06.8538116Z [cluster1]                                 ".": {},
2021-07-16T18:15:06.8538756Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-16T18:15:06.8539483Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:06.8540129Z [cluster1]                                 "f:status": {},
2021-07-16T18:15:06.8540663Z [cluster1]                                 "f:type": {}
2021-07-16T18:15:06.8541231Z [cluster1]                             },
2021-07-16T18:15:06.8541758Z [cluster1]                             "k:{\"type\":\"Ready\"}": {
2021-07-16T18:15:06.8542284Z [cluster1]                                 ".": {},
2021-07-16T18:15:06.8542864Z [cluster1]                                 "f:lastProbeTime": {},
2021-07-16T18:15:06.8543586Z [cluster1]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:06.8544243Z [cluster1]                                 "f:status": {},
2021-07-16T18:15:06.8545626Z [cluster1]                                 "f:type": {}
2021-07-16T18:15:06.8564798Z [cluster1]                             }
2021-07-16T18:15:06.8565291Z [cluster1]                         },
2021-07-16T18:15:06.8565890Z [cluster1]                         "f:containerStatuses": {},
2021-07-16T18:15:06.8566544Z [cluster1]                         "f:hostIP": {},
2021-07-16T18:15:06.8567074Z [cluster1]                         "f:phase": {},
2021-07-16T18:15:06.8567623Z [cluster1]                         "f:podIP": {},
2021-07-16T18:15:06.8568148Z [cluster1]                         "f:podIPs": {
2021-07-16T18:15:06.8568658Z [cluster1]                             ".": {},
2021-07-16T18:15:06.8569170Z [cluster1]                             "k:{\"ip\":\"10.1.0.3\"}": {
2021-07-16T18:15:06.8569758Z [cluster1]                                 ".": {},
2021-07-16T18:15:06.8570253Z [cluster1]                                 "f:ip": {}
2021-07-16T18:15:06.8570732Z [cluster1]                             }
2021-07-16T18:15:06.8571196Z [cluster1]                         },
2021-07-16T18:15:06.8571705Z [cluster1]                         "f:startTime": {}
2021-07-16T18:15:06.8572215Z [cluster1]                     }
2021-07-16T18:15:06.8572723Z [cluster1]                 },
2021-07-16T18:15:06.8573241Z [cluster1]                 "manager": "kubelet",
2021-07-16T18:15:06.8573847Z [cluster1]                 "operation": "Update",
2021-07-16T18:15:06.8574865Z [cluster1]                 "time": "2021-07-16T18:13:48Z"
2021-07-16T18:15:06.8576139Z [cluster1]             }
2021-07-16T18:15:06.8576705Z [cluster1]         ],
2021-07-16T18:15:06.8577791Z [cluster1]         "name": "submariner-operator-546678bc8f-k4qbm",
2021-07-16T18:15:06.8579003Z [cluster1]         "namespace": "submariner-operator",
2021-07-16T18:15:06.8579746Z [cluster1]         "ownerReferences": [
2021-07-16T18:15:06.8580300Z [cluster1]             {
2021-07-16T18:15:06.8581128Z [cluster1]                 "apiVersion": "apps/v1",
2021-07-16T18:15:06.8581851Z [cluster1]                 "blockOwnerDeletion": true,
2021-07-16T18:15:06.8582537Z [cluster1]                 "controller": true,
2021-07-16T18:15:06.8583124Z [cluster1]                 "kind": "ReplicaSet",
2021-07-16T18:15:06.8584181Z [cluster1]                 "name": "submariner-operator-546678bc8f",
2021-07-16T18:15:06.8585305Z [cluster1]                 "uid": "6e95e778-4152-4d14-a159-0eaa4da2893c"
2021-07-16T18:15:06.8585927Z [cluster1]             }
2021-07-16T18:15:06.8586346Z [cluster1]         ],
2021-07-16T18:15:06.8586898Z [cluster1]         "resourceVersion": "1823",
2021-07-16T18:15:06.8588363Z [cluster1]         "selfLink": "/api/v1/namespaces/submariner-operator/pods/submariner-operator-546678bc8f-k4qbm",
2021-07-16T18:15:06.8589897Z [cluster1]         "uid": "5457dbfa-f81b-472e-8383-6e2c60dcae02"
2021-07-16T18:15:06.8590544Z [cluster1]     },
2021-07-16T18:15:06.8590978Z [cluster1]     "spec": {
2021-07-16T18:15:06.8591492Z [cluster1]         "containers": [
2021-07-16T18:15:06.8591973Z [cluster1]             {
2021-07-16T18:15:06.8592444Z [cluster1]                 "command": [
2021-07-16T18:15:06.8593241Z [cluster1]                     "submariner-operator",
2021-07-16T18:15:06.8594106Z [cluster1]                     "-alsologtostderr",
2021-07-16T18:15:06.8594805Z [cluster1]                     "-v=1"
2021-07-16T18:15:06.8595256Z [cluster1]                 ],
2021-07-16T18:15:06.8595704Z [cluster1]                 "env": [
2021-07-16T18:15:06.8596270Z [cluster1]                     {
2021-07-16T18:15:06.8596816Z [cluster1]                         "name": "WATCH_NAMESPACE",
2021-07-16T18:15:06.8597427Z [cluster1]                         "valueFrom": {
2021-07-16T18:15:06.8598001Z [cluster1]                             "fieldRef": {
2021-07-16T18:15:06.8598584Z [cluster1]                                 "apiVersion": "v1",
2021-07-16T18:15:06.8599327Z [cluster1]                                 "fieldPath": "metadata.namespace"
2021-07-16T18:15:06.8599974Z [cluster1]                             }
2021-07-16T18:15:06.8600436Z [cluster1]                         }
2021-07-16T18:15:06.8600873Z [cluster1]                     },
2021-07-16T18:15:06.8601323Z [cluster1]                     {
2021-07-16T18:15:06.8601820Z [cluster1]                         "name": "POD_NAME",
2021-07-16T18:15:06.8602382Z [cluster1]                         "valueFrom": {
2021-07-16T18:15:06.8602944Z [cluster1]                             "fieldRef": {
2021-07-16T18:15:06.8641418Z [cluster1]                                 "apiVersion": "v1",
2021-07-16T18:15:06.8642262Z [cluster1]                                 "fieldPath": "metadata.name"
2021-07-16T18:15:06.8642857Z [cluster1]                             }
2021-07-16T18:15:06.8643325Z [cluster1]                         }
2021-07-16T18:15:06.8643934Z [cluster1]                     },
2021-07-16T18:15:06.8644387Z [cluster1]                     {
2021-07-16T18:15:06.8645058Z [cluster1]                         "name": "OPERATOR_NAME",
2021-07-16T18:15:06.8646188Z [cluster1]                         "value": "submariner-operator"
2021-07-16T18:15:06.8646898Z [cluster1]                     }
2021-07-16T18:15:06.8647347Z [cluster1]                 ],
2021-07-16T18:15:06.8648279Z [cluster1]                 "image": "localhost:5000/submariner-operator:local",
2021-07-16T18:15:06.8649250Z [cluster1]                 "imagePullPolicy": "IfNotPresent",
2021-07-16T18:15:06.8650239Z [cluster1]                 "name": "submariner-operator",
2021-07-16T18:15:06.8650899Z [cluster1]                 "resources": {},
2021-07-16T18:15:06.8651914Z [cluster1]                 "terminationMessagePath": "/dev/termination-log",
2021-07-16T18:15:06.8652925Z [cluster1]                 "terminationMessagePolicy": "File",
2021-07-16T18:15:06.8653697Z [cluster1]                 "volumeMounts": [
2021-07-16T18:15:06.8654202Z [cluster1]                     {
2021-07-16T18:15:06.8655134Z [cluster1]                         "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
2021-07-16T18:15:06.8656465Z [cluster1]                         "name": "submariner-operator-token-46pk2",
2021-07-16T18:15:06.8657332Z [cluster1]                         "readOnly": true
2021-07-16T18:15:06.8657849Z [cluster1]                     }
2021-07-16T18:15:06.8658274Z [cluster1]                 ]
2021-07-16T18:15:06.8658712Z [cluster1]             }
2021-07-16T18:15:06.8659130Z [cluster1]         ],
2021-07-16T18:15:06.8659693Z [cluster1]         "dnsPolicy": "ClusterFirst",
2021-07-16T18:15:06.8660414Z [cluster1]         "enableServiceLinks": true,
2021-07-16T18:15:06.8661334Z [cluster1]         "nodeName": "cluster1-worker",
2021-07-16T18:15:06.8661946Z [cluster1]         "priority": 0,
2021-07-16T18:15:06.8662542Z [cluster1]         "restartPolicy": "Always",
2021-07-16T18:15:06.8663474Z [cluster1]         "schedulerName": "default-scheduler",
2021-07-16T18:15:06.8664221Z [cluster1]         "securityContext": {},
2021-07-16T18:15:06.8665197Z [cluster1]         "serviceAccount": "submariner-operator",
2021-07-16T18:15:06.8666333Z [cluster1]         "serviceAccountName": "submariner-operator",
2021-07-16T18:15:06.8667384Z [cluster1]         "terminationGracePeriodSeconds": 30,
2021-07-16T18:15:06.8668206Z [cluster1]         "tolerations": [
2021-07-16T18:15:06.8668712Z [cluster1]             {
2021-07-16T18:15:06.8669227Z [cluster1]                 "effect": "NoExecute",
2021-07-16T18:15:06.8670137Z [cluster1]                 "key": "node.kubernetes.io/not-ready",
2021-07-16T18:15:06.8670961Z [cluster1]                 "operator": "Exists",
2021-07-16T18:15:06.8671600Z [cluster1]                 "tolerationSeconds": 300
2021-07-16T18:15:06.8672169Z [cluster1]             },
2021-07-16T18:15:06.8672590Z [cluster1]             {
2021-07-16T18:15:06.8673121Z [cluster1]                 "effect": "NoExecute",
2021-07-16T18:15:06.8673854Z [cluster1]                 "key": "node.kubernetes.io/unreachable",
2021-07-16T18:15:06.8674593Z [cluster1]                 "operator": "Exists",
2021-07-16T18:15:06.8675228Z [cluster1]                 "tolerationSeconds": 300
2021-07-16T18:15:06.8675804Z [cluster1]             }
2021-07-16T18:15:06.8676221Z [cluster1]         ],
2021-07-16T18:15:06.8678326Z [cluster1]         "volumes": [
2021-07-16T18:15:06.8679340Z [cluster1]             {
2021-07-16T18:15:06.8680502Z [cluster1]                 "name": "submariner-operator-token-46pk2",
2021-07-16T18:15:06.8681763Z [cluster1]                 "secret": {
2021-07-16T18:15:06.8682328Z [cluster1]                     "defaultMode": 420,
2021-07-16T18:15:06.8683512Z [cluster1]                     "secretName": "submariner-operator-token-46pk2"
2021-07-16T18:15:06.8684727Z [cluster1]                 }
2021-07-16T18:15:06.8685168Z [cluster1]             }
2021-07-16T18:15:06.8685585Z [cluster1]         ]
2021-07-16T18:15:06.8686011Z [cluster1]     },
2021-07-16T18:15:06.8686449Z [cluster1]     "status": {
2021-07-16T18:15:06.8686966Z [cluster1]         "conditions": [
2021-07-16T18:15:06.8687449Z [cluster1]             {
2021-07-16T18:15:06.8688006Z [cluster1]                 "lastProbeTime": null,
2021-07-16T18:15:06.8690472Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:40Z",
2021-07-16T18:15:06.8691139Z [cluster1]                 "status": "True",
2021-07-16T18:15:06.8691722Z [cluster1]                 "type": "Initialized"
2021-07-16T18:15:06.8692229Z [cluster1]             },
2021-07-16T18:15:06.8692656Z [cluster1]             {
2021-07-16T18:15:06.8693187Z [cluster1]                 "lastProbeTime": null,
2021-07-16T18:15:06.8694117Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:48Z",
2021-07-16T18:15:06.8694776Z [cluster1]                 "status": "True",
2021-07-16T18:15:06.8695308Z [cluster1]                 "type": "Ready"
2021-07-16T18:15:06.8695772Z [cluster1]             },
2021-07-16T18:15:06.8696206Z [cluster1]             {
2021-07-16T18:15:06.8696757Z [cluster1]                 "lastProbeTime": null,
2021-07-16T18:15:06.8697815Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:48Z",
2021-07-16T18:15:06.8698503Z [cluster1]                 "status": "True",
2021-07-16T18:15:06.8699113Z [cluster1]                 "type": "ContainersReady"
2021-07-16T18:15:06.8699684Z [cluster1]             },
2021-07-16T18:15:06.8700110Z [cluster1]             {
2021-07-16T18:15:06.8700661Z [cluster1]                 "lastProbeTime": null,
2021-07-16T18:15:06.8701577Z [cluster1]                 "lastTransitionTime": "2021-07-16T18:13:40Z",
2021-07-16T18:15:06.8702256Z [cluster1]                 "status": "True",
2021-07-16T18:15:06.8702845Z [cluster1]                 "type": "PodScheduled"
2021-07-16T18:15:06.8703388Z [cluster1]             }
2021-07-16T18:15:06.8703811Z [cluster1]         ],
2021-07-16T18:15:06.8704364Z [cluster1]         "containerStatuses": [
2021-07-16T18:15:06.8704935Z [cluster1]             {
2021-07-16T18:15:06.8706755Z [cluster1]                 "containerID": "containerd://66af6d06450c6e0a9e2ec585119d36aea814b907ba9f24334174ee85685dd712",
2021-07-16T18:15:06.8709019Z [cluster1]                 "image": "localhost:5000/submariner-operator:local",
2021-07-16T18:15:06.9081246Z [cluster1]                 "imageID": "localhost:5000/submariner-operator@sha256:3e2788fae3e11a10b9728e6797f3ba25c5d4c6aa583e9c0095a16ter2] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9084210Z [submariner-operator]$ [cluster2] kubectl get sa --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9086710Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get sa --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9088096Z [cluster2] NAME                  SECRETS   AGE
2021-07-16T18:15:06.9089065Z [cluster2] submariner-operator   1         79s
2021-07-16T18:15:06.9090624Z [submariner-operator]$ [cluster2] kubectl get roles --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9092595Z [submariner-operator]$ [cluster2] kubectl get roles --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9094756Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get roles --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9096090Z [cluster2] NAME                  CREATED AT
2021-07-16T18:15:06.9097051Z [cluster2] submariner-operator   2021-07-16T18:13:46Z
2021-07-16T18:15:06.9098683Z [submariner-operator]$ [cluster2] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9100757Z [submariner-operator]$ [cluster2] kubectl get rolebindings --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9102994Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get rolebindings --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9126307Z [cluster2] NAME                  ROLE                       AGE
2021-07-16T18:15:06.9127696Z [cluster2] submariner-operator   Role/submariner-operator   79s
2021-07-16T18:15:06.9129340Z [submariner-operator]$ [cluster2] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9131300Z [submariner-operator]$ [cluster2] kubectl get deployments --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9133383Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get deployments --namespace=submariner-operator submariner-operator
2021-07-16T18:15:06.9134937Z [cluster2] NAME                  READY   UP-TO-DATE   AVAILABLE   AGE
2021-07-16T18:15:06.9135892Z [cluster2] submariner-operator   1/1     1            1           73s
2021-07-16T18:15:06.9136911Z [submariner-operator]$ [cluster2] verify_subm_op_pod
2021-07-16T18:15:06.9137969Z [submariner-operator]$ [cluster2] verify_subm_op_pod
2021-07-16T18:15:06.9139873Z [submariner-operator]$ [cluster2] subm_operator_pod_name=$(kubectl get pods --namespace=$subm_ns -l name=$operator_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-16T18:15:06.9142279Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:06.9144608Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:06.9147188Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods --namespace=submariner-operator -l name=submariner-operator -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:06.9149683Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator -o json
2021-07-16T18:15:06.9152080Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator -o json
2021-07-16T18:15:06.9154605Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pod submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator -o json
2021-07-16T18:15:06.9155990Z [cluster2] {
2021-07-16T18:15:06.9156471Z [cluster2]     "apiVersion": "v1",
2021-07-16T18:15:06.9157002Z [cluster2]     "kind": "Pod",
2021-07-16T18:15:06.9157486Z [cluster2]     "metadata": {
2021-07-16T18:15:06.9158323Z [cluster2]         "creationTimestamp": "2021-07-16T18:13:52Z",
2021-07-16T18:15:06.9159687Z [cluster2]         "generateName": "submariner-operator-546678bc8f-",
2021-07-16T18:15:06.9160544Z [cluster2]         "labels": {
2021-07-16T18:15:06.9161335Z [cluster2]             "name": "submariner-operator",
2021-07-16T18:15:06.9162271Z [cluster2]             "pod-template-hash": "546678bc8f"
2021-07-16T18:15:06.9162839Z [cluster2]         },
2021-07-16T18:15:06.9163356Z [cluster2]         "managedFields": [
2021-07-16T18:15:06.9164780Z [cluster2]             {
2021-07-16T18:15:06.9165297Z [cluster2]                 "apiVersion": "v1",
2021-07-16T18:15:06.9165927Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:06.9166504Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:06.9167046Z [cluster2]                     "f:metadata": {
2021-07-16T18:15:06.9167625Z [cluster2]                         "f:generateName": {},
2021-07-16T18:15:06.9168211Z [cluster2]                         "f:labels": {
2021-07-16T18:15:06.9168701Z [cluster2]                             ".": {},
2021-07-16T18:15:06.9169206Z [cluster2]                             "f:name": {},
2021-07-16T18:15:06.9170061Z [cluster2]                             "f:pod-template-hash": {}
2021-07-16T18:15:06.9170659Z [cluster2]                         },
2021-07-16T18:15:06.9171244Z [cluster2]                         "f:ownerReferences": {
2021-07-16T18:15:06.9171816Z [cluster2]                             ".": {},
2021-07-16T18:15:06.9172815Z [cluster2]                             "k:{\"uid\":\"e1b7a0fa-f7d3-441a-9259-8df6ea70a1e8\"}": {
2021-07-16T18:15:06.9173571Z [cluster2]                                 ".": {},
2021-07-16T18:15:06.9174133Z [cluster2]                                 "f:apiVersion": {},
2021-07-16T18:15:06.9174824Z [cluster2]                                 "f:blockOwnerDeletion": {},
2021-07-16T18:15:06.9175524Z [cluster2]                                 "f:controller": {},
2021-07-16T18:15:06.9176069Z [cluster2]                                 "f:kind": {},
2021-07-16T18:15:06.9176593Z [cluster2]                                 "f:name": {},
2021-07-16T18:15:06.9177117Z [cluster2]                                 "f:uid": {}
2021-07-16T18:15:06.9177584Z [cluster2]                             }
2021-07-16T18:15:06.9178040Z [cluster2]                         }
2021-07-16T18:15:06.9178483Z [cluster2]                     },
2021-07-16T18:15:06.9179026Z [cluster2]                     "f:spec": {
2021-07-16T18:15:06.9179562Z [cluster2]                         "f:containers": {
2021-07-16T18:15:06.9180621Z [cluster2]                             "k:{\"name\":\"submariner-operator\"}": {
2021-07-16T18:15:06.9181290Z [cluster2]                                 ".": {},
2021-07-16T18:15:06.9181818Z [cluster2]                                 "f:command": {},
2021-07-16T18:15:06.9182357Z [cluster2]                                 "f:env": {
2021-07-16T18:15:06.9182848Z [cluster2]                                     ".": {},
2021-07-16T18:15:06.9183446Z [cluster2]                                     "k:{\"name\":\"OPERATOR_NAME\"}": {
2021-07-16T18:15:06.9184035Z [cluster2]                                         ".": {},
2021-07-16T18:15:06.9185215Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:06.9185775Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:06.9186288Z [cluster2]                                     },
2021-07-16T18:15:06.9186830Z [cluster2]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-16T18:15:06.9187408Z [cluster2]                                         ".": {},
2021-07-16T18:15:06.9187931Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:06.9188491Z [cluster2]                                         "f:valueFrom": {
2021-07-16T18:15:06.9189048Z [cluster2]                                             ".": {},
2021-07-16T18:15:06.9189587Z [cluster2]                                             "f:fieldRef": {
2021-07-16T18:15:06.9190145Z [cluster2]                                                 ".": {},
2021-07-16T18:15:06.9190848Z [cluster2]                                                 "f:apiVersion": {},
2021-07-16T18:15:06.9191482Z [cluster2]                                                 "f:fieldPath": {}
2021-07-16T18:15:06.9192048Z [cluster2]                                             }
2021-07-16T18:15:06.9192525Z [cluster2]                                         }
2021-07-16T18:15:06.9193010Z [cluster2]                                     },
2021-07-16T18:15:06.9193600Z [cluster2]                                     "k:{\"name\":\"WATCH_NAMESPACE\"}": {
2021-07-16T18:15:06.9194213Z [cluster2]                                         ".": {},
2021-07-16T18:15:06.9194720Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:06.9195293Z [cluster2]                                         "f:valueFrom": {
2021-07-16T18:15:06.9195835Z [cluster2]                                             ".": {},
2021-07-16T18:15:06.9196386Z [cluster2]                                             "f:fieldRef": {
2021-07-16T18:15:06.9198052Z [cluster2]                                                 ".": {},
2021-07-16T18:15:06.9198805Z [cluster2]                                                 "f:apiVersion": {},
2021-07-16T18:15:06.9199448Z [cluster2]                                                 "f:fieldPath": {}
2021-07-16T18:15:06.9199998Z [cluster2]                                             }
2021-07-16T18:15:06.9200494Z [cluster2]                                         }
2021-07-16T18:15:06.9200971Z [cluster2]                                     }
2021-07-16T18:15:06.9201451Z [cluster2]                                 },
2021-07-16T18:15:06.9201944Z [cluster2]                                 "f:image": {},
2021-07-16T18:15:06.9202575Z [cluster2]                                 "f:imagePullPolicy": {},
2021-07-16T18:15:06.9203198Z [cluster2]                                 "f:name": {},
2021-07-16T18:15:06.9204100Z [cluster2]                                 "f:resources": {},
2021-07-16T18:15:06.9204862Z [cluster2]                                 "f:terminationMessagePath": {},
2021-07-16T18:15:06.9205786Z [cluster2]                                 "f:terminationMessagePolicy": {}
2021-07-16T18:15:06.9206506Z [cluster2]                             }
2021-07-16T18:15:06.9206954Z [cluster2]                         },
2021-07-16T18:15:06.9207476Z [cluster2]                         "f:dnsPolicy": {},
2021-07-16T18:15:06.9208312Z [cluster2]                         "f:enableServiceLinks": {},
2021-07-16T18:15:06.9209025Z [cluster2]                         "f:restartPolicy": {},
2021-07-16T18:15:06.9209684Z [cluster2]                         "f:schedulerName": {},
2021-07-16T18:15:06.9210345Z [cluster2]                         "f:securityContext": {},
2021-07-16T18:15:06.9211031Z [cluster2]                         "f:serviceAccount": {},
2021-07-16T18:15:06.9211737Z [cluster2]                         "f:serviceAccountName": {},
2021-07-16T18:15:06.9212689Z [cluster2]                         "f:terminationGracePeriodSeconds": {}
2021-07-16T18:15:06.9228869Z [cluster2]                     }
2021-07-16T18:15:06.9229354Z [cluster2]                 },
2021-07-16T18:15:06.9230476Z [cluster2]                 "manager": "kube-controller-manager",
2021-07-16T18:15:06.9231251Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:06.9232003Z [cluster2]                 "time": "2021-07-16T18:13:52Z"
2021-07-16T18:15:06.9232494Z [cluster2]             },
2021-07-16T18:15:06.9232936Z [cluster2]             {
2021-07-16T18:15:06.9233453Z [cluster2]                 "apiVersion": "v1",
2021-07-16T18:15:06.9234060Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:06.9234651Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:06.9235293Z [cluster2]                     "f:status": {
2021-07-16T18:15:06.9235843Z [cluster2]                         "f:conditions": {
2021-07-16T18:15:06.9236516Z [cluster2]                             "k:{\"type\":\"ContainersReady\"}": {
2021-07-16T18:15:06.9237289Z [cluster2]                                 ".": {},
2021-07-16T18:15:06.9237882Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-16T18:15:06.9238599Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:06.9239263Z [cluster2]                                 "f:status": {},
2021-07-16T18:15:06.9239790Z [cluster2]                                 "f:type": {}
2021-07-16T18:15:06.9240290Z [cluster2]                             },
2021-07-16T18:15:06.9240850Z [cluster2]                             "k:{\"type\":\"Initialized\"}": {
2021-07-16T18:15:06.9241434Z [cluster2]                                 ".": {},
2021-07-16T18:15:06.9242016Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-16T18:15:06.9242728Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:06.9243391Z [cluster2]                                 "f:status": {},
2021-07-16T18:15:06.9244525Z [cluster2]                                 "f:type": {}
2021-07-16T18:15:06.9245027Z [cluster2]                             },
2021-07-16T18:15:06.9245543Z [cluster2]                             "k:{\"type\":\"Ready\"}": {
2021-07-16T18:15:06.9246080Z [cluster2]                                 ".": {},
2021-07-16T18:15:06.9246649Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-16T18:15:06.9247380Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:06.9248055Z [cluster2]                                 "f:status": {},
2021-07-16T18:15:06.9257643Z [cluster2]                                 "f:type": {}
2021-07-16T18:15:06.9258162Z [cluster2]                             }
2021-07-16T18:15:06.9258619Z [cluster2]                         },
2021-07-16T18:15:06.9259230Z [cluster2]                         "f:containerStatuses": {},
2021-07-16T18:15:06.9259992Z [cluster2]                         "f:hostIP": {},
2021-07-16T18:15:06.9260540Z [cluster2]                         "f:phase": {},
2021-07-16T18:15:06.9261080Z [cluster2]                         "f:podIP": {},
2021-07-16T18:15:06.9261621Z [cluster2]                         "f:podIPs": {
2021-07-16T18:15:06.9262232Z [cluster2]                             ".": {},
2021-07-16T18:15:06.9262742Z [cluster2]                             "k:{\"ip\":\"10.2.128.2\"}": {
2021-07-16T18:15:06.9263262Z [cluster2]                                 ".": {},
2021-07-16T18:15:06.9263902Z [cluster2]                                 "f:ip": {}
2021-07-16T18:15:06.9264401Z [cluster2]                             }
2021-07-16T18:15:06.9264852Z [cluster2]                         },
2021-07-16T18:15:06.9265377Z [cluster2]                         "f:startTime": {}
2021-07-16T18:15:06.9265878Z [cluster2]                     }
2021-07-16T18:15:06.9266327Z [cluster2]                 },
2021-07-16T18:15:06.9266837Z [cluster2]                 "manager": "kubelet",
2021-07-16T18:15:06.9267442Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:06.9268411Z [cluster2]                 "time": "2021-07-16T18:14:07Z"
2021-07-16T18:15:06.9268891Z [cluster2]             }
2021-07-16T18:15:06.9269324Z [cluster2]         ],
2021-07-16T18:15:06.9270342Z [cluster2]         "name": "submariner-operator-546678bc8f-c2k7c",
2021-07-16T18:15:06.9271554Z [cluster2]         "namespace": "submariner-operator",
2021-07-16T18:15:06.9272291Z [cluster2]         "ownerReferences": [
2021-07-16T18:15:06.9272835Z [cluster2]             {
2021-07-16T18:15:06.9273361Z [cluster2]                 "apiVersion": "apps/v1",
2021-07-16T18:15:06.9274077Z [cluster2]                 "blockOwnerDeletion": true,
2021-07-16T18:15:06.9274749Z [cluster2]                 "controller": true,
2021-07-16T18:15:06.9275348Z [cluster2]                 "kind": "ReplicaSet",
2021-07-16T18:15:06.9276373Z [cluster2]                 "name": "submariner-operator-546678bc8f",
2021-07-16T18:15:06.9278983Z [cluster2]                 "uid": "e1b7a0fa-f7d3-441a-9259-8df6ea70a1e8"
2021-07-16T18:15:06.9279824Z [cluster2]             }
2021-07-16T18:15:06.9280283Z [cluster2]         ],
2021-07-16T18:15:06.9280842Z [cluster2]         "resourceVersion": "1850",
2021-07-16T18:15:06.9282321Z [cluster2]         "selfLink": "/api/v1/namespaces/submariner-operator/pods/submariner-operator-546678bc8f-c2k7c",
2021-07-16T18:15:06.9284336Z [cluster2]         "uid": "08bc41fc-2e13-4ec9-9f03-9c305d4c80e7"
2021-07-16T18:15:06.9284983Z [cluster2]     },
2021-07-16T18:15:06.9285653Z [cluster2]     "spec": {
2021-07-16T18:15:06.9287994Z [cluster2]         "containers": [
2021-07-16T18:15:06.9297721Z [cluster2]             {
2021-07-16T18:15:06.9298270Z [cluster2]                 "command": [
2021-07-16T18:15:06.9299245Z [cluster2]                     "submariner-operator",
2021-07-16T18:15:06.9300129Z [cluster2]                     "-alsologtostderr",
2021-07-16T18:15:06.9301092Z [cluster2]                     "-v=1"
2021-07-16T18:15:06.9301722Z [cluster2]                 ],
2021-07-16T18:15:06.9302192Z [cluster2]                 "env": [
2021-07-16T18:15:06.9302661Z [cluster2]                     {
2021-07-16T18:15:06.9303205Z [cluster2]                         "name": "WATCH_NAMESPACE",
2021-07-16T18:15:06.9303812Z [cluster2]                         "valueFrom": {
2021-07-16T18:15:06.9304472Z [cluster2]                             "fieldRef": {
2021-07-16T18:15:06.9305971Z [cluster2]                                 "apiVersion": "v1",
2021-07-16T18:15:06.9306999Z [cluster2]                                 "fieldPath": "metadata.namespace"
2021-07-16T18:15:06.9307911Z [cluster2]                             }
2021-07-16T18:15:06.9308373Z [cluster2]                         }
2021-07-16T18:15:06.9308814Z [cluster2]                     },
2021-07-16T18:15:06.9309266Z [cluster2]                     {
2021-07-16T18:15:06.9309772Z [cluster2]                         "name": "POD_NAME",
2021-07-16T18:15:06.9310335Z [cluster2]                         "valueFrom": {
2021-07-16T18:15:06.9310879Z [cluster2]                             "fieldRef": {
2021-07-16T18:15:06.9311475Z [cluster2]                                 "apiVersion": "v1",
2021-07-16T18:15:06.9312151Z [cluster2]                                 "fieldPath": "metadata.name"
2021-07-16T18:15:06.9312736Z [cluster2]                             }
2021-07-16T18:15:06.9313197Z [cluster2]                         }
2021-07-16T18:15:06.9314118Z [cluster2]                     },
2021-07-16T18:15:06.9314591Z [cluster2]                     {
2021-07-16T18:15:06.9315505Z [cluster2]                         "name": "OPERATOR_NAME",
2021-07-16T18:15:06.9316836Z [cluster2]                         "value": "submariner-operator"
2021-07-16T18:15:06.9317619Z [cluster2]                     }
2021-07-16T18:15:06.9318221Z [cluster2]                 ],
2021-07-16T18:15:06.9319318Z [cluster2]                 "image": "localhost:5000/submariner-operator:local",
2021-07-16T18:15:06.9320521Z [cluster2]                 "imagePullPolicy": "IfNotPresent",
2021-07-16T18:15:06.9321708Z [cluster2]                 "name": "submariner-operator",
2021-07-16T18:15:06.9322535Z [cluster2]                 "resources": {},
2021-07-16T18:15:06.9324014Z [cluster2]                 "terminationMessagePath": "/dev/termination-log",
2021-07-16T18:15:06.9325679Z [cluster2]                 "terminationMessagePolicy": "File",
2021-07-16T18:15:06.9326457Z [cluster2]                 "volumeMounts": [
2021-07-16T18:15:06.9326968Z [cluster2]                     {
2021-07-16T18:15:06.9327736Z [cluster2]                         "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
2021-07-16T18:15:06.9329082Z [cluster2]                         "name": "submariner-operator-token-cdfj2",
2021-07-16T18:15:06.9337261Z [cluster2]                         "readOnly": true
2021-07-16T18:15:06.9337800Z [cluster2]                     }
2021-07-16T18:15:06.9338238Z [cluster2]                 ]
2021-07-16T18:15:06.9338682Z [cluster2]             }
2021-07-16T18:15:06.9339098Z [cluster2]         ],
2021-07-16T18:15:06.9339832Z [cluster2]         "dnsPolicy": "ClusterFirst",
2021-07-16T18:15:06.9340554Z [cluster2]         "enableServiceLinks": true,
2021-07-16T18:15:06.9341640Z [cluster2]         "nodeName": "cluster2-worker",
2021-07-16T18:15:06.9342251Z [cluster2]         "priority": 0,
2021-07-16T18:15:06.9342853Z [cluster2]         "restartPolicy": "Always",
2021-07-16T18:15:06.9343784Z [cluster2]         "schedulerName": "default-scheduler",
2021-07-16T18:15:06.9344544Z [cluster2]         "securityContext": {},
2021-07-16T18:15:06.9345523Z [cluster2]         "serviceAccount": "submariner-operator",
2021-07-16T18:15:06.9346661Z [cluster2]         "serviceAccountName": "submariner-operator",
2021-07-16T18:15:06.9347721Z [cluster2]         "terminationGracePeriodSeconds": 30,
2021-07-16T18:15:06.9348537Z [cluster2]         "tolerations": [
2021-07-16T18:15:06.9349035Z [cluster2]             {
2021-07-16T18:15:06.9350044Z [cluster2]                 "effect": "NoExecute",
2021-07-16T18:15:06.9351749Z [cluster2]                 "key": "node.kubernetes.io/not-ready",
2021-07-16T18:15:06.9352508Z [cluster2]                 "operator": "Exists",
2021-07-16T18:15:06.9353165Z [cluster2]                 "tolerationSeconds": 300
2021-07-16T18:15:06.9355743Z [cluster2]             },
2021-07-16T18:15:06.9356173Z [cluster2]             {
2021-07-16T18:15:06.9356709Z [cluster2]                 "effect": "NoExecute",
2021-07-16T18:15:06.9357447Z [cluster2]                 "key": "node.kubernetes.io/unreachable",
2021-07-16T18:15:06.9358195Z [cluster2]                 "operator": "Exists",
2021-07-16T18:15:06.9358837Z [cluster2]                 "tolerationSeconds": 300
2021-07-16T18:15:06.9359778Z [cluster2]             }
2021-07-16T18:15:06.9360199Z [cluster2]         ],
2021-07-16T18:15:06.9360669Z [cluster2]         "volumes": [
2021-07-16T18:15:06.9361124Z [cluster2]             {
2021-07-16T18:15:06.9362693Z [cluster2]                 "name": "submariner-operator-token-cdfj2",
2021-07-16T18:15:06.9364154Z [cluster2]                 "secret": {
2021-07-16T18:15:06.9364710Z [cluster2]                     "defaultMode": 420,
2021-07-16T18:15:06.9365893Z [cluster2]                     "secretName": "submariner-operator-token-cdfj2"
2021-07-16T18:15:06.9366975Z [cluster2]                 }
2021-07-16T18:15:06.9422634Z [cluster2]             }
2021-07-16T18:15:06.9423532Z [cluster2]         ]
2021-07-16T18:15:06.9423995Z [cluster2]     },
2021-07-16T18:15:06.9424440Z [cluster2]     "status": {
2021-07-16T18:15:06.9424960Z [cluster2]         "conditions": [
2021-07-16T18:15:06.9426005Z [cluster2]             {
2021-07-16T18:15:06.9426614Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:06.9427746Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:52Z",
2021-07-16T18:15:06.9429007Z [cluster2]                 "status": "True",
2021-07-16T18:15:06.9429987Z [cluster2]                 "type": "Initialized"
2021-07-16T18:15:06.9430755Z [cluster2]             },
2021-07-16T18:15:06.9431575Z [cluster2]             {
2021-07-16T18:15:06.9432139Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:06.9433158Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:14:07Z",
2021-07-16T18:15:06.9433816Z [cluster2]                 "status": "True",
2021-07-16T18:15:06.9434634Z [cluster2]                 "type": "Ready"
2021-07-16T18:15:06.9435103Z [cluster2]             },
2021-07-16T18:15:06.9435540Z [cluster2]             {
2021-07-16T18:15:06.9436195Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:06.9437561Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:14:07Z",
2021-07-16T18:15:06.9438246Z [cluster2]                 "status": "True",
2021-07-16T18:15:06.9438855Z [cluster2]                 "type": "ContainersReady"
2021-07-16T18:15:06.9439419Z [cluster2]             },
2021-07-16T18:15:06.9439832Z [cluster2]             {
2021-07-16T18:15:06.9440381Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:07.8390218Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:13:52Z",
2021-07-16T18:15:07.8391233Z [cluster2]                 "status": "True",
2021-07-16T18:15:07.8391832Z [cluster2]                 "type": "PodScheduled"
2021-07-16T18:15:07.8392379Z [cluster2]             }
2021-07-16T18:15:07.8392795Z [cluster2]         ],
2021-07-16T18:15:07.8393366Z [cluster2]         "containerStatuses": [
2021-07-16T18:15:07.8393919Z [cluster2]             {
2021-07-16T18:15:07.8395725Z [cluster2]                 "containerID": "containerd://12ae38651c99a933faf251e77538c5416fcf5d548cc16b945de38370102f7e22",
2021-07-16T18:15:07.8397992Z [cluster2]                 "image": "localhost:5000/submariner-operator:local",
2021-07-16T18:15:07.8399834Z [cluster2]                 "imageID": "localhost:5000/submariner-operator@sha256:3e2788fae3e11a10b9728e6797f3ba25c5d4c6aa583e9c0095a169b2bc33572a",
2021-07-16T18:15:07.8401177Z [cluster2]                 "lastState": {
2021-07-16T18:15:07.8401722Z [cluster2]                     "terminated": {
2021-07-16T18:15:07.8404326Z [cluster2]                         "containerID": "containerd://ed1946e2e453ea121f172b044a56b65f2230bc46428e1c866ff8dbe6a8a40368",
2021-07-16T18:15:07.8406194Z [cluster2]                         "exitCode": 1,
2021-07-16T18:15:07.8407019Z [cluster2]                         "finishedAt": "2021-07-16T18:14:03Z",
2021-07-16T18:15:07.8407639Z [cluster2]                         "reason": "Error",
2021-07-16T18:15:07.8408409Z [cluster2]                         "startedAt": "2021-07-16T18:13:53Z"
2021-07-16T18:15:07.8408957Z [cluster2]                     }
2021-07-16T18:15:07.8409392Z [cluster2]                 },
2021-07-16T18:15:07.8410187Z [cluster2]                 "name": "submariner-operator",
2021-07-16T18:15:07.8410835Z [cluster2]                 "ready": true,
2021-07-16T18:15:07.8411398Z [cluster2]                 "restartCount": 1,
2021-07-16T18:15:07.8411972Z [cluster2]                 "started": true,
2021-07-16T18:15:07.8412472Z [cluster2]                 "state": {
2021-07-16T18:15:07.8412983Z [cluster2]                     "running": {
2021-07-16T18:15:07.8413735Z [cluster2]                         "startedAt": "2021-07-16T18:14:07Z"
2021-07-16T18:15:07.8414277Z [cluster2]                     }
2021-07-16T18:15:07.8414709Z [cluster2]                 }
2021-07-16T18:15:07.8415146Z [cluster2]             }
2021-07-16T18:15:07.8415560Z [cluster2]         ],
2021-07-16T18:15:07.8416038Z [cluster2]         "hostIP": "172.18.0.5",
2021-07-16T18:15:07.8416567Z [cluster2]         "phase": "Running",
2021-07-16T18:15:07.8417266Z [cluster2]         "podIP": "10.2.128.2",
2021-07-16T18:15:07.8417780Z [cluster2]         "podIPs": [
2021-07-16T18:15:07.8418237Z [cluster2]             {
2021-07-16T18:15:07.8418705Z [cluster2]                 "ip": "10.2.128.2"
2021-07-16T18:15:07.8419150Z [cluster2]             }
2021-07-16T18:15:07.8419582Z [cluster2]         ],
2021-07-16T18:15:07.8420106Z [cluster2]         "qosClass": "BestEffort",
2021-07-16T18:15:07.8420927Z [cluster2]         "startTime": "2021-07-16T18:13:52Z"
2021-07-16T18:15:07.8421426Z [cluster2]     }
2021-07-16T18:15:07.8421844Z [cluster2] }
2021-07-16T18:15:07.8423503Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-16T18:15:07.8425290Z [submariner-operator]$ [cluster2] grep Running
2021-07-16T18:15:07.8427173Z [submariner-operator]$ [cluster2] kubectl get pod submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-16T18:15:07.8429911Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pod submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-16T18:15:07.8431429Z [cluster2] Running
2021-07-16T18:15:07.8432951Z [submariner-operator]$ [cluster2] kubectl logs submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator
2021-07-16T18:15:07.8435222Z [submariner-operator]$ [cluster2] kubectl logs submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator
2021-07-16T18:15:07.8437780Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 logs submariner-operator-546678bc8f-c2k7c --namespace=submariner-operator
2021-07-16T18:15:07.8439314Z [cluster2] I0716 18:14:07.772201       1 main.go:77] cmd "msg"="Go Version: go1.16.5"  
2021-07-16T18:15:07.8440065Z [cluster2] I0716 18:14:07.772255       1 main.go:78] cmd "msg"="Go OS/Arch: linux/amd64"  
2021-07-16T18:15:07.8441115Z [cluster2] I0716 18:14:07.772263       1 main.go:79] cmd "msg"="Version of operator-sdk: v0.19.0"  
2021-07-16T18:15:07.8442290Z [cluster2] I0716 18:14:07.772270       1 main.go:80] cmd "msg"="Submariner operator version: devel-0aaed33"  
2021-07-16T18:15:07.8443266Z [cluster2] I0716 18:14:07.772894       1 leader.go:89] leader "msg"="Trying to become the leader."  
2021-07-16T18:15:07.8444559Z [cluster2] I0716 18:14:08.178992       1 leader.go:122] leader "msg"="Found existing lock with my name. I was likely restarted."  
2021-07-16T18:15:07.8445565Z [cluster2] I0716 18:14:08.179022       1 leader.go:123] leader "msg"="Continuing as the leader."  
2021-07-16T18:15:07.8447000Z [cluster2] I0716 18:14:08.602966       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"="0.0.0.0:8383"
2021-07-16T18:15:07.8448093Z [cluster2] I0716 18:14:08.605055       1 main.go:129] cmd "msg"="Registering Components."  
2021-07-16T18:15:07.8448984Z [cluster2] I0716 18:14:08.605360       1 main.go:137] cmd "msg"="Creating the Lighthouse CRDs"  
2021-07-16T18:15:07.8507964Z [cluster2] I0716 18:14:11.747354       1 main.go:182] cmd "msg"="Could not create ServiceMonitor object"  "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8510563Z [cluster2] I0716 18:14:11.747400       1 main.go:186] cmd "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects"  "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8511906Z [cluster2] I0716 18:14:11.747444       1 main.go:203] cmd "msg"="Starting the Cmd."  
2021-07-16T18:15:07.8513213Z [cluster2] I0716 18:14:11.747644       1 internal.go:406] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
2021-07-16T18:15:07.8519617Z [cluster2] I0716 18:14:11.748171       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-16T18:15:07.8529439Z [cluster2] I0716 18:14:11.748434       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-16T18:15:07.8536636Z [cluster2] I0716 18:14:11.748616       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-16T18:15:07.8541880Z [cluster2] I0716 18:14:11.748739       1 controller.go:158] controller-runtime/manager/controller/servicediscovery "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-16T18:15:07.8545849Z [cluster2] I0716 18:14:11.748851       1 controller.go:158] controller-runtime/manager/controller/broker "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Broker" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{}}}
2021-07-16T18:15:07.8549913Z [cluster2] I0716 18:14:11.850067       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"updateStrategy":{}},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}}
2021-07-16T18:15:07.8553423Z [cluster2] I0716 18:14:11.854997       1 controller.go:165] controller-runtime/manager/controller/broker "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:07.8555850Z [cluster2] I0716 18:14:11.855083       1 controller.go:165] controller-runtime/manager/controller/servicediscovery "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-16T18:15:07.8559112Z [cluster2] I0716 18:14:11.855385       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-16T18:15:07.8563119Z [cluster2] I0716 18:14:11.855592       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-16T18:15:07.8591958Z [cluster2] I0716 18:14:11.951309       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"status":{"version":"","haStatus":"","localEndpoint":{"cluster_id":"","cable_name":"","hostname":"","subnets":null,"private_ip":"","public_ip":"","nat_enabled":false,"backend":""},"statusFailure":"","connections":null}}}
2021-07-16T18:15:07.8595042Z [cluster2] I0716 18:14:11.955614       1 controller.go:165] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting Controller"  
2021-07-16T18:15:07.8597250Z [cluster2] I0716 18:14:11.955909       1 controller.go:192] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting workers"  "worker count"=1
2021-07-16T18:15:07.8599780Z [cluster2] I0716 18:14:11.955784       1 controller.go:192] controller-runtime/manager/controller/servicediscovery "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "worker count"=1
2021-07-16T18:15:07.8602359Z [cluster2] I0716 18:14:11.955818       1 controller.go:192] controller-runtime/manager/controller/broker "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Broker" "worker count"=1
2021-07-16T18:15:07.8605163Z [cluster2] I0716 18:14:11.955872       1 controller.go:165] controller-runtime/manager/controller/submariner "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Submariner" 
2021-07-16T18:15:07.8607626Z [cluster2] I0716 18:14:11.956253       1 controller.go:192] controller-runtime/manager/controller/submariner "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "worker count"=1
2021-07-16T18:15:07.8610037Z [cluster2] I0716 18:14:11.956416       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8611858Z [cluster2] I0716 18:14:12.019578       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"  
2021-07-16T18:15:07.8613763Z [cluster2] I0716 18:14:12.019615       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.2.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.2.0.0/16"]
2021-07-16T18:15:07.8615452Z [cluster2] I0716 18:14:12.019626       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8616996Z [cluster2] I0716 18:14:12.019636       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8631826Z [cluster2] I0716 18:14:12.052815       1 controller.go:165] controller-runtime/manager/controller/submariner-controller "msg"="Starting Controller"  
2021-07-16T18:15:07.8634163Z [cluster2] I0716 18:14:12.052884       1 controller.go:192] controller-runtime/manager/controller/submariner-controller "msg"="Starting workers"  "worker count"=1
2021-07-16T18:15:07.8636393Z [cluster2] I0716 18:14:12.053131       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8638188Z [cluster2] I0716 18:14:12.053317       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8639653Z [cluster2] I0716 18:14:12.053346       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8642479Z [cluster2] I0716 18:14:12.132320       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8646089Z [cluster2] I0716 18:14:12.141335       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8649511Z [cluster2] I0716 18:14:12.372059       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8653545Z [cluster2] E0716 18:14:12.372303       1 controller.go:267] controller-runtime/manager/controller/submariner-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-gateway-metrics: Service \"submariner-gateway-metrics\" not found" "name"="submariner" "namespace"="submariner-operator" 
2021-07-16T18:15:07.8656888Z [cluster2] I0716 18:14:12.372507       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8658697Z [cluster2] I0716 18:14:12.372675       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8660262Z [cluster2] I0716 18:14:12.372803       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8662917Z [cluster2] I0716 18:14:12.436534       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8666365Z [cluster2] I0716 18:14:12.693652       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8670683Z [cluster2] E0716 18:14:12.857126       1 controller.go:267] controller-runtime/manager/controller/submariner "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-gateway-metrics: services \"submariner-gateway-metrics\" already exists" "name"="submariner" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Submariner" 
2021-07-16T18:15:07.8674239Z [cluster2] I0716 18:14:12.862860       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8675999Z [cluster2] I0716 18:14:12.863029       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8677479Z [cluster2] I0716 18:14:12.863112       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8680120Z [cluster2] I0716 18:14:12.914929       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8685678Z [cluster2] I0716 18:14:12.939388       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8689297Z [cluster2] I0716 18:14:13.097703       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8692712Z [cluster2] I0716 18:14:13.130235       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8696054Z [cluster2] I0716 18:14:13.166414       1 servicediscovery_resources.go:66] controller_submariner "msg"="Created Service Discovery CR" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Name"="service-discovery" "Namespace"="submariner-operator"
2021-07-16T18:15:07.8699150Z [cluster2] I0716 18:14:13.177852       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8702060Z [cluster2] I0716 18:14:13.178314       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8705542Z [cluster2] I0716 18:14:13.253372       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.8709322Z [cluster2] I0716 18:14:13.473227       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8712163Z [cluster2] I0716 18:14:13.569062       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8713915Z [cluster2] I0716 18:14:13.569109       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8715380Z [cluster2] I0716 18:14:13.569120       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8718968Z [cluster2] E0716 18:14:13.589829       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: deployments.apps \"submariner-lighthouse-agent\" already exists" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-16T18:15:07.8722948Z [cluster2] I0716 18:14:13.589884       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8726561Z [cluster2] I0716 18:14:13.931140       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8730021Z [cluster2] I0716 18:14:13.931942       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8733786Z [cluster2] I0716 18:14:14.127973       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8738145Z [cluster2] E0716 18:14:14.555038       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8741385Z [cluster2] I0716 18:14:14.555085       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8743123Z [cluster2] I0716 18:14:14.555149       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8744599Z [cluster2] I0716 18:14:14.555160       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8748479Z [cluster2] I0716 18:14:14.608779       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8753829Z [cluster2] I0716 18:14:14.619591       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.8757884Z [cluster2] I0716 18:14:14.664124       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8761353Z [cluster2] I0716 18:14:14.770121       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8765238Z [cluster2] I0716 18:14:14.869672       1 helpers.go:175] controller_servicediscovery "msg"="Created a new ConfigMap" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "ConfigMap.Name"="submariner-lighthouse-coredns" "ConfigMap.Namespace"="submariner-operator"
2021-07-16T18:15:07.8768933Z [cluster2] I0716 18:14:15.093251       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8772662Z [cluster2] I0716 18:14:15.325184       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8777477Z [cluster2] I0716 18:14:15.325517       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.8782404Z [cluster2] I0716 18:14:15.326116       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8788316Z [cluster2] I0716 18:14:15.339969       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8791793Z [cluster2] I0716 18:14:15.770432       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8795249Z [cluster2] I0716 18:14:15.770995       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8799240Z [cluster2] E0716 18:14:16.110188       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8802453Z [cluster2] I0716 18:14:16.110257       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8804367Z [cluster2] I0716 18:14:16.110308       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8805934Z [cluster2] I0716 18:14:16.110319       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8827966Z [cluster2] I0716 18:14:16.338106       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.8831988Z [cluster2] I0716 18:14:16.350172       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8835433Z [cluster2] I0716 18:14:16.400240       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8839037Z [cluster2] I0716 18:14:16.963260       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8842115Z [cluster2] I0716 18:14:17.041853       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8844141Z [cluster2] I0716 18:14:17.041977       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8845608Z [cluster2] I0716 18:14:17.041998       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8848239Z [cluster2] I0716 18:14:17.560897       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8851903Z [cluster2] I0716 18:14:17.826712       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8855758Z [cluster2] I0716 18:14:17.842384       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8859565Z [cluster2] I0716 18:14:18.256858       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8863105Z [cluster2] I0716 18:14:18.281499       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8866653Z [cluster2] I0716 18:14:18.586216       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8869993Z [cluster2] I0716 18:14:18.587349       1 servicediscovery_controller.go:487] controller_servicediscovery "msg"="coredns configmap does not have lighthouse configuration hence creating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8875430Z [cluster2] I0716 18:14:18.591412       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.229.54\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n"  
2021-07-16T18:15:07.8879791Z [cluster2] I0716 18:14:18.597934       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8884560Z [cluster2] I0716 18:14:19.076179       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8889016Z [cluster2] E0716 18:14:19.290642       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8892524Z [cluster2] I0716 18:14:19.290686       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8894277Z [cluster2] I0716 18:14:19.290725       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.8895758Z [cluster2] I0716 18:14:19.290734       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.8899077Z [cluster2] E0716 18:14:19.299799       1 servicediscovery_controller.go:169] controller_servicediscovery "msg"="Error creating the lighthouseCoreDNS service" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-coredns: services \"submariner-lighthouse-coredns\" already exists"  
2021-07-16T18:15:07.8905170Z [cluster2] E0716 18:14:19.299838       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-coredns: services \"submariner-lighthouse-coredns\" already exists" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-16T18:15:07.8909215Z [cluster2] I0716 18:14:19.299865       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.8912731Z [cluster2] I0716 18:14:20.191370       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.8916452Z [cluster2] I0716 18:14:20.191884       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8920142Z [cluster2] I0716 18:14:20.207935       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8923955Z [cluster2] I0716 18:14:20.481130       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8940834Z [cluster2] I0716 18:14:20.506005       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8944879Z [cluster2] I0716 18:14:20.888658       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8948324Z [cluster2] I0716 18:14:20.902222       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.8951896Z [cluster2] I0716 18:14:20.909210       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.8955771Z [cluster2] I0716 18:14:20.925961       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.8960677Z [cluster2] I0716 18:14:20.932724       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.8965843Z [cluster2] I0716 18:14:21.335914       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9003892Z [cluster2] I0716 18:14:21.347560       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9008895Z [cluster2] I0716 18:14:21.347612       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.229.54\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n"  
2021-07-16T18:15:07.9012966Z [cluster2] I0716 18:14:21.361344       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9016621Z [cluster2] I0716 18:14:21.374024       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.9020934Z [cluster2] I0716 18:14:21.381932       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9024970Z [cluster2] I0716 18:14:21.785169       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9028829Z [cluster2] I0716 18:14:21.792618       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.9032963Z [cluster2] I0716 18:14:21.807533       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9037048Z [cluster2] I0716 18:14:22.209955       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9040452Z [cluster2] I0716 18:14:22.214084       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9076945Z [cluster2] I0716 18:14:22.214122       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.229.54\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n"  
2021-07-16T18:15:07.9081214Z [cluster2] I0716 18:14:30.752574       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9082991Z [cluster2] I0716 18:14:30.752629       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9084643Z [cluster2] I0716 18:14:30.752639       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9087318Z [cluster2] I0716 18:14:30.796516       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9090765Z [cluster2] I0716 18:14:30.831620       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9094203Z [cluster2] I0716 18:14:31.233489       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9097623Z [cluster2] I0716 18:14:31.240461       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9100638Z [cluster2] I0716 18:14:31.258366       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9102674Z [cluster2] I0716 18:14:31.258406       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9104122Z [cluster2] I0716 18:14:31.258418       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9106207Z [cluster2] I0716 18:14:31.284838       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9107952Z [cluster2] I0716 18:14:31.284879       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9109414Z [cluster2] I0716 18:14:31.284889       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9112046Z [cluster2] I0716 18:14:31.309813       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9115406Z [cluster2] I0716 18:14:31.310390       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9118841Z [cluster2] I0716 18:14:31.352014       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9160080Z [cluster2] I0716 18:14:31.352311       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9163762Z [cluster2] I0716 18:14:31.754051       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9167134Z [cluster2] I0716 18:14:31.754101       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9170560Z [cluster2] I0716 18:14:31.759000       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9174030Z [cluster2] I0716 18:14:31.759625       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9177922Z [cluster2] E0716 18:14:31.768007       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9181280Z [cluster2] I0716 18:14:31.768046       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9183044Z [cluster2] I0716 18:14:31.768081       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9184517Z [cluster2] I0716 18:14:31.768091       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9186572Z [cluster2] I0716 18:14:31.807282       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9188326Z [cluster2] I0716 18:14:31.807402       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9189778Z [cluster2] I0716 18:14:31.807418       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9192388Z [cluster2] I0716 18:14:31.828571       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9195765Z [cluster2] I0716 18:14:31.847369       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9199234Z [cluster2] I0716 18:14:31.853778       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9202802Z [cluster2] I0716 18:14:31.870958       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9206382Z [cluster2] I0716 18:14:32.256724       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9209783Z [cluster2] I0716 18:14:32.270280       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9213220Z [cluster2] I0716 18:14:32.284705       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9271346Z [cluster2] E0716 18:14:32.300298       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9274878Z [cluster2] I0716 18:14:32.300367       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9276897Z [cluster2] I0716 18:14:32.300496       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9278354Z [cluster2] I0716 18:14:32.300514       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9281055Z [cluster2] I0716 18:14:32.314263       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9284924Z [cluster2] I0716 18:14:32.317499       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9288522Z [cluster2] I0716 18:14:32.349582       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9291980Z [cluster2] I0716 18:14:32.751433       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9295377Z [cluster2] I0716 18:14:32.756195       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9298247Z [cluster2] I0716 18:14:32.931780       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9300015Z [cluster2] I0716 18:14:32.931832       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9301585Z [cluster2] I0716 18:14:32.931845       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9303678Z [cluster2] I0716 18:14:32.939246       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9305434Z [cluster2] I0716 18:14:32.939298       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9306869Z [cluster2] I0716 18:14:32.939316       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9309505Z [cluster2] I0716 18:14:32.975104       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9313013Z [cluster2] I0716 18:14:32.977493       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9316545Z [cluster2] I0716 18:14:33.005763       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9320153Z [cluster2] I0716 18:14:33.006488       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9323712Z [cluster2] I0716 18:14:33.414638       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9327080Z [cluster2] I0716 18:14:33.416473       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9330465Z [cluster2] I0716 18:14:33.445521       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9333339Z [cluster2] I0716 18:14:33.474998       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9335208Z [cluster2] I0716 18:14:33.475621       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9336652Z [cluster2] I0716 18:14:33.475638       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9339330Z [cluster2] I0716 18:14:33.506316       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9377937Z [cluster2] I0716 18:14:33.514999       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9381667Z [cluster2] I0716 18:14:33.525823       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9385759Z [cluster2] E0716 18:14:33.543398       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9389063Z [cluster2] I0716 18:14:33.543444       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9390838Z [cluster2] I0716 18:14:33.543493       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9392303Z [cluster2] I0716 18:14:33.543510       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9394942Z [cluster2] I0716 18:14:33.610621       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9398388Z [cluster2] I0716 18:14:33.734586       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9401663Z [cluster2] I0716 18:14:33.776466       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9405357Z [cluster2] I0716 18:14:33.899474       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.9409420Z [cluster2] I0716 18:14:33.921292       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9413136Z [cluster2] I0716 18:14:33.953109       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9416554Z [cluster2] I0716 18:14:33.977616       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9419963Z [cluster2] I0716 18:14:34.196858       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9423362Z [cluster2] I0716 18:14:34.231527       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9427031Z [cluster2] I0716 18:14:34.323503       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9430908Z [cluster2] I0716 18:14:34.339222       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.9435055Z [cluster2] I0716 18:14:34.389602       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9499686Z [cluster2] I0716 18:14:34.792945       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9503155Z [cluster2] I0716 18:14:34.796071       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9507980Z [cluster2] I0716 18:14:34.796111       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.229.54\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n"  
2021-07-16T18:15:07.9512073Z [cluster2] I0716 18:14:35.015568       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9513835Z [cluster2] I0716 18:14:35.015616       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9515278Z [cluster2] I0716 18:14:35.015627       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9517925Z [cluster2] I0716 18:14:35.026887       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9521415Z [cluster2] I0716 18:14:35.031515       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9565438Z [cluster2] I0716 18:14:35.434836       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9569168Z [cluster2] I0716 18:14:35.440914       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9572091Z [cluster2] I0716 18:14:35.457957       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9574264Z [cluster2] I0716 18:14:35.458051       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9575742Z [cluster2] I0716 18:14:35.458077       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9577936Z [cluster2] I0716 18:14:35.466524       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9579702Z [cluster2] I0716 18:14:35.466566       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9581161Z [cluster2] I0716 18:14:35.466577       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9583790Z [cluster2] I0716 18:14:35.468218       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9587234Z [cluster2] I0716 18:14:35.478322       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9590800Z [cluster2] I0716 18:14:35.480190       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9594290Z [cluster2] I0716 18:14:35.489823       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9597713Z [cluster2] I0716 18:14:35.882829       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9621245Z [cluster2] I0716 18:14:35.888577       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9624732Z [cluster2] I0716 18:14:35.895444       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9627579Z [cluster2] I0716 18:14:35.906922       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9716193Z [cluster2] I0716 18:14:35.906968       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9717850Z [cluster2] I0716 18:14:35.907126       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9720878Z [cluster2] I0716 18:14:35.913973       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9725066Z [cluster2] I0716 18:14:35.914640       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9729011Z [cluster2] E0716 18:14:35.923368       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9732245Z [cluster2] I0716 18:14:35.923481       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9734000Z [cluster2] I0716 18:14:35.923570       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9735457Z [cluster2] I0716 18:14:35.923631       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9738316Z [cluster2] I0716 18:14:35.928718       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9741769Z [cluster2] I0716 18:14:35.931980       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9745467Z [cluster2] I0716 18:14:35.947480       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9748777Z [cluster2] I0716 18:14:36.008998       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9752291Z [cluster2] I0716 18:14:36.020088       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.9763348Z [cluster2] I0716 18:14:36.031285       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9767416Z [cluster2] I0716 18:14:36.330898       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9770843Z [cluster2] I0716 18:14:36.341080       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9774283Z [cluster2] I0716 18:14:36.349792       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9800317Z [cluster2] I0716 18:14:36.355843       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9804472Z [cluster2] I0716 18:14:36.433567       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9808387Z [cluster2] I0716 18:14:36.438026       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:07.9882792Z [cluster2] I0716 18:14:36.442281       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9887164Z [cluster2] I0716 18:14:36.843706       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9890569Z [cluster2] I0716 18:14:36.845917       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9895522Z [cluster2] I0716 18:14:36.845982       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.2.229.54\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster2.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n"  
2021-07-16T18:15:07.9899356Z [cluster2] I0716 18:14:40.799576       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9901210Z [cluster2] I0716 18:14:40.799627       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9902686Z [cluster2] I0716 18:14:40.799639       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9905307Z [cluster2] I0716 18:14:40.808026       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9908770Z [cluster2] I0716 18:14:40.811155       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:07.9912210Z [cluster2] I0716 18:14:41.213066       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:07.9915726Z [cluster2] I0716 18:14:41.218033       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9918644Z [cluster2] I0716 18:14:41.236449       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9920491Z [cluster2] I0716 18:14:41.236491       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9921946Z [cluster2] I0716 18:14:41.236501       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9926405Z [cluster2] I0716 18:14:41.237616       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:07.9928203Z [cluster2] I0716 18:14:41.237649       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:07.9929662Z [cluster2] I0716 18:14:41.237659       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:07.9932376Z [cluster2] I0716 18:14:41.249435       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9948998Z [cluster2] I0716 18:14:41.251308       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:07.9952510Z [cluster2] I0716 18:14:41.274352       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0022613Z [cluster2] I0716 18:14:41.276320       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0026364Z [cluster2] I0716 18:14:41.675887       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0029768Z [cluster2] I0716 18:14:41.679960       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0033201Z [cluster2] I0716 18:14:41.690194       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0036790Z [cluster2] I0716 18:14:41.692067       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0039694Z [cluster2] I0716 18:14:45.830835       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0041444Z [cluster2] I0716 18:14:45.830890       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0043279Z [cluster2] I0716 18:14:45.830901       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0046547Z [cluster2] I0716 18:14:45.883837       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0050001Z [cluster2] I0716 18:14:45.890754       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0053431Z [cluster2] I0716 18:14:46.292502       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0056836Z [cluster2] I0716 18:14:46.297593       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0059689Z [cluster2] I0716 18:14:46.306957       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0061576Z [cluster2] I0716 18:14:46.307079       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0063049Z [cluster2] I0716 18:14:46.307266       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0065133Z [cluster2] I0716 18:14:46.307714       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0066899Z [cluster2] I0716 18:14:46.307774       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0068359Z [cluster2] I0716 18:14:46.307784       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0070987Z [cluster2] I0716 18:14:46.325108       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0074375Z [cluster2] I0716 18:14:46.325574       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0095987Z [cluster2] I0716 18:14:46.331545       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0099565Z [cluster2] I0716 18:14:46.336661       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0103032Z [cluster2] I0716 18:14:46.733538       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0138421Z [cluster2] I0716 18:14:46.737793       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0142176Z [cluster2] I0716 18:14:46.743552       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0145693Z [cluster2] I0716 18:14:46.754513       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0148731Z [cluster2] I0716 18:14:50.913055       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0150491Z [cluster2] I0716 18:14:50.913114       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0151954Z [cluster2] I0716 18:14:50.913138       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0154770Z [cluster2] I0716 18:14:50.926682       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0158211Z [cluster2] I0716 18:14:50.935570       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0161660Z [cluster2] I0716 18:14:51.337299       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0222567Z [cluster2] I0716 18:14:51.342087       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0225970Z [cluster2] I0716 18:14:51.363060       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0227735Z [cluster2] I0716 18:14:51.363100       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0229204Z [cluster2] I0716 18:14:51.363111       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0231454Z [cluster2] I0716 18:14:51.370522       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0233321Z [cluster2] I0716 18:14:51.370581       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0234796Z [cluster2] I0716 18:14:51.370599       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0237692Z [cluster2] I0716 18:14:51.382311       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0241111Z [cluster2] I0716 18:14:51.382745       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0244892Z [cluster2] I0716 18:14:51.388865       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0248448Z [cluster2] I0716 18:14:51.390024       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0251992Z [cluster2] I0716 18:14:51.790412       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0255365Z [cluster2] I0716 18:14:51.797559       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0258909Z [cluster2] I0716 18:14:51.805720       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0262343Z [cluster2] I0716 18:14:51.815815       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0265226Z [cluster2] I0716 18:14:55.984226       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0266996Z [cluster2] I0716 18:14:55.984292       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0268465Z [cluster2] I0716 18:14:55.984304       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0271112Z [cluster2] I0716 18:14:56.014015       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0274539Z [cluster2] I0716 18:14:56.051351       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0298587Z [cluster2] I0716 18:14:56.453654       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0302072Z [cluster2] I0716 18:14:56.477854       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0305420Z [cluster2] I0716 18:14:56.502613       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0307265Z [cluster2] I0716 18:14:56.502867       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0308711Z [cluster2] I0716 18:14:56.503089       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0310833Z [cluster2] I0716 18:14:56.503381       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0312592Z [cluster2] I0716 18:14:56.503526       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0314040Z [cluster2] I0716 18:14:56.503631       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0316675Z [cluster2] I0716 18:14:56.558505       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0320071Z [cluster2] I0716 18:14:56.559583       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0343436Z [cluster2] I0716 18:14:56.585140       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0347305Z [cluster2] I0716 18:14:56.606889       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0350764Z [cluster2] I0716 18:14:56.987630       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0354173Z [cluster2] I0716 18:14:56.995943       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0367940Z [cluster2] I0716 18:14:57.023149       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0371383Z [cluster2] I0716 18:14:57.038038       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0374324Z [cluster2] I0716 18:15:01.063975       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0376076Z [cluster2] I0716 18:15:01.064041       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0377753Z [cluster2] I0716 18:15:01.064051       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0380394Z [cluster2] I0716 18:15:01.078933       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0383841Z [cluster2] I0716 18:15:01.085803       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0387318Z [cluster2] I0716 18:15:01.487619       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0413245Z [cluster2] I0716 18:15:01.492242       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0416166Z [cluster2] I0716 18:15:01.516839       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0418177Z [cluster2] I0716 18:15:01.517045       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0419743Z [cluster2] I0716 18:15:01.517128       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0421829Z [cluster2] I0716 18:15:01.518350       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0432631Z [cluster2] I0716 18:15:01.519213       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0434097Z [cluster2] I0716 18:15:01.519295       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0436921Z [cluster2] I0716 18:15:01.544940       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0440375Z [cluster2] I0716 18:15:01.550980       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0451367Z [cluster2] I0716 18:15:01.566020       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0455058Z [cluster2] I0716 18:15:01.586821       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0458544Z [cluster2] I0716 18:15:01.952607       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0488321Z [cluster2] I0716 18:15:01.958420       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0491815Z [cluster2] I0716 18:15:01.990654       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0495246Z [cluster2] I0716 18:15:01.997892       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0498126Z [cluster2] I0716 18:15:06.091566       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0499871Z [cluster2] I0716 18:15:06.091623       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0501328Z [cluster2] I0716 18:15:06.091634       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0503955Z [cluster2] I0716 18:15:06.103340       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0507649Z [cluster2] I0716 18:15:06.109020       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0511223Z [cluster2] I0716 18:15:06.510736       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0514647Z [cluster2] I0716 18:15:06.516317       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0517537Z [cluster2] I0716 18:15:06.529385       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0519282Z [cluster2] I0716 18:15:06.529458       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0520760Z [cluster2] I0716 18:15:06.529483       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0522833Z [cluster2] I0716 18:15:06.534140       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.0524692Z [cluster2] I0716 18:15:06.534182       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.2.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.0526156Z [cluster2] I0716 18:15:06.534195       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.2.0.0/16" "type"="Service"
2021-07-16T18:15:08.0528866Z [cluster2] I0716 18:15:06.546544       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0532268Z [cluster2] I0716 18:15:06.547132       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.0535722Z [cluster2] I0716 18:15:06.553703       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0539245Z [cluster2] I0716 18:15:06.553956       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.0542669Z [cluster2] I0716 18:15:06.955635       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0546054Z [cluster2] I0716 18:15:06.960643       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.0549567Z [cluster2] I0716 18:15:06.968625       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1245844Z [cluster2] I0716 18:15:069b2bc33572a",
2021-07-16T18:15:08.1246434Z [cluster1]                 "lastState": {},
2021-07-16T18:15:08.1247780Z [cluster1]                 "name": "submariner-operator",
2021-07-16T18:15:08.1248564Z [cluster1]                 "ready": true,
2021-07-16T18:15:08.1249130Z [cluster1]                 "restartCount": 1,
2021-07-16T18:15:08.1249693Z [cluster1]                 "started": true,
2021-07-16T18:15:08.1250215Z [cluster1]                 "state": {
2021-07-16T18:15:08.1250707Z [cluster1]                     "running": {
2021-07-16T18:15:08.1251504Z [cluster1]                         "startedAt": "2021-07-16T18:13:47Z"
2021-07-16T18:15:08.1252069Z [cluster1]                     }
2021-07-16T18:15:08.1252504Z [cluster1]                 }
2021-07-16T18:15:08.1252939Z [cluster1]             }
2021-07-16T18:15:08.1253358Z [cluster1]         ],
2021-07-16T18:15:08.1253837Z [cluster1]         "hostIP": "172.18.0.9",
2021-07-16T18:15:08.1254362Z [cluster1]         "phase": "Running",
2021-07-16T18:15:08.1254897Z [cluster1]         "podIP": "10.1.0.3",
2021-07-16T18:15:08.1255385Z [cluster1]         "podIPs": [
2021-07-16T18:15:08.1255858Z [cluster1]             {
2021-07-16T18:15:08.1256307Z [cluster1]                 "ip": "10.1.0.3"
2021-07-16T18:15:08.1256765Z [cluster1]             }
2021-07-16T18:15:08.1257178Z [cluster1]         ],
2021-07-16T18:15:08.1257715Z [cluster1]         "qosClass": "BestEffort",
2021-07-16T18:15:08.1258520Z [cluster1]         "startTime": "2021-07-16T18:13:40Z"
2021-07-16T18:15:08.1259022Z [cluster1]     }
2021-07-16T18:15:08.1259441Z [cluster1] }
2021-07-16T18:15:08.1261106Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-16T18:15:08.1263712Z [submariner-operator]$ [cluster1] kubectl get pod submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-16T18:15:08.1266657Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator -o jsonpath={.status.phase}
2021-07-16T18:15:08.1268564Z [submariner-operator]$ [cluster1] grep Running
2021-07-16T18:15:08.1269373Z [cluster1] Running
2021-07-16T18:15:08.1270888Z [submariner-operator]$ [cluster1] kubectl logs submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator
2021-07-16T18:15:08.1273181Z [submariner-operator]$ [cluster1] kubectl logs submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator
2021-07-16T18:15:08.1275608Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 logs submariner-operator-546678bc8f-k4qbm --namespace=submariner-operator
2021-07-16T18:15:08.1277135Z [cluster1] I0716 18:13:47.516649       1 main.go:77] cmd "msg"="Go Version: go1.16.5"  
2021-07-16T18:15:08.1277891Z [cluster1] I0716 18:13:47.516897       1 main.go:78] cmd "msg"="Go OS/Arch: linux/amd64"  
2021-07-16T18:15:08.1278919Z [cluster1] I0716 18:13:47.517051       1 main.go:79] cmd "msg"="Version of operator-sdk: v0.19.0"  
2021-07-16T18:15:08.1280108Z [cluster1] I0716 18:13:47.517124       1 main.go:80] cmd "msg"="Submariner operator version: devel-0aaed33"  
2021-07-16T18:15:08.1281066Z [cluster1] I0716 18:13:47.517371       1 leader.go:89] leader "msg"="Trying to become the leader."  
2021-07-16T18:15:08.1282090Z [cluster1] I0716 18:13:47.925520       1 leader.go:122] leader "msg"="Found existing lock with my name. I was likely restarted."  
2021-07-16T18:15:08.1283090Z [cluster1] I0716 18:13:47.925552       1 leader.go:123] leader "msg"="Continuing as the leader."  
2021-07-16T18:15:08.1284861Z [cluster1] I0716 18:13:48.331015       1 listener.go:44] controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"="0.0.0.0:8383"
2021-07-16T18:15:08.1286066Z [cluster1] I0716 18:13:48.331365       1 main.go:129] cmd "msg"="Registering Components."  
2021-07-16T18:15:08.1287035Z [cluster1] I0716 18:13:48.331513       1 main.go:137] cmd "msg"="Creating the Lighthouse CRDs"  
2021-07-16T18:15:08.1288211Z [cluster1] I0716 18:13:50.876771       1 main.go:182] cmd "msg"="Could not create ServiceMonitor object"  "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1290198Z [cluster1] I0716 18:13:50.876806       1 main.go:186] cmd "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects"  "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1291478Z [cluster1] I0716 18:13:50.876853       1 main.go:203] cmd "msg"="Starting the Cmd."  
2021-07-16T18:15:08.1292820Z [cluster1] I0716 18:13:50.877131       1 internal.go:406] controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
2021-07-16T18:15:08.1295250Z [cluster1] I0716 18:13:50.877616       1 controller.go:158] controller-runtime/manager/controller/broker "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Broker" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{}}}
2021-07-16T18:15:08.1337799Z [cluster1] I0716 18:13:50.877760       1 controller.go:158] controller-runtime/manager/controller/servicediscovery "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-16T18:15:08.1345858Z [cluster1] I0716 18:13:50.878236       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"status":{"deploymentInfo":{}},"metadata":{"creationTimestamp":null},"spec":{"brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","clusterID":"","namespace":"","debug":false}}}
2021-07-16T18:15:08.1353330Z [cluster1] I0716 18:13:50.878352       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-16T18:15:08.1363203Z [cluster1] I0716 18:13:50.880671       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"broker":"","brokerK8sApiServer":"","brokerK8sApiServerToken":"","brokerK8sCA":"","brokerK8sRemoteNamespace":"","ceIPSecPSK":"","clusterCIDR":"","clusterID":"","serviceCIDR":"","namespace":"","ceIPSecDebug":false,"debug":false,"natEnabled":false},"status":{"natEnabled":false,"clusterID":"","gatewayDaemonSetStatus":{"mismatchedContainerImages":false},"routeAgentDaemonSetStatus":{"mismatchedContainerImages":false},"globalnetDaemonSetStatus":{"mismatchedContainerImages":false},"loadBalancerStatus":{},"deploymentInfo":{}}}}
2021-07-16T18:15:08.1369136Z [cluster1] I0716 18:13:50.977940       1 controller.go:165] controller-runtime/manager/controller/broker "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1371513Z [cluster1] I0716 18:13:50.978180       1 controller.go:192] controller-runtime/manager/controller/broker "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Broker" "worker count"=1
2021-07-16T18:15:08.1373973Z [cluster1] I0716 18:13:50.978008       1 controller.go:165] controller-runtime/manager/controller/servicediscovery "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-16T18:15:08.1376620Z [cluster1] I0716 18:13:50.978309       1 controller.go:192] controller-runtime/manager/controller/servicediscovery "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" "worker count"=1
2021-07-16T18:15:08.1379899Z [cluster1] I0716 18:13:50.983545       1 controller.go:158] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-16T18:15:08.1384372Z [cluster1] I0716 18:13:50.983670       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"updateStrategy":{}},"status":{"currentNumberScheduled":0,"numberMisscheduled":0,"desiredNumberScheduled":0,"numberReady":0}}}
2021-07-16T18:15:08.1394607Z [cluster1] I0716 18:13:50.983733       1 controller.go:158] controller-runtime/manager/controller/submariner "msg"="Starting EventSource" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"selector":null,"template":{"metadata":{"creationTimestamp":null},"spec":{"containers":null}},"strategy":{}},"status":{}}}
2021-07-16T18:15:08.1398000Z [cluster1] I0716 18:13:51.084456       1 controller.go:165] controller-runtime/manager/controller/submariner "msg"="Starting Controller" "reconciler group"="submariner.io" "reconciler kind"="Submariner" 
2021-07-16T18:15:08.1400552Z [cluster1] I0716 18:13:51.084656       1 controller.go:192] controller-runtime/manager/controller/submariner "msg"="Starting workers" "reconciler group"="submariner.io" "reconciler kind"="Submariner" "worker count"=1
2021-07-16T18:15:08.1403397Z [cluster1] I0716 18:13:51.084850       1 controller.go:165] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting Controller"  
2021-07-16T18:15:08.1406017Z [cluster1] I0716 18:13:51.084946       1 controller.go:192] controller-runtime/manager/controller/servicediscovery-controller "msg"="Starting workers"  "worker count"=1
2021-07-16T18:15:08.1409339Z [cluster1] I0716 18:13:51.085089       1 controller.go:158] controller-runtime/manager/controller/submariner-controller "msg"="Starting EventSource"  "source"={"Type":{"metadata":{"creationTimestamp":null},"status":{"version":"","haStatus":"","localEndpoint":{"cluster_id":"","cable_name":"","hostname":"","subnets":null,"private_ip":"","public_ip":"","nat_enabled":false,"backend":""},"statusFailure":"","connections":null}}}
2021-07-16T18:15:08.1413194Z [cluster1] E0716 18:13:51.124126       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1416774Z [cluster1] E0716 18:13:51.173699       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1419590Z [cluster1] I0716 18:13:51.185306       1 controller.go:165] controller-runtime/manager/controller/submariner-controller "msg"="Starting Controller"  
2021-07-16T18:15:08.1431741Z [cluster1] I0716 18:13:51.185409       1 controller.go:192] controller-runtime/manager/controller/submariner-controller "msg"="Starting workers"  "worker count"=1
2021-07-16T18:15:08.1434640Z [cluster1] E0716 18:13:51.230588       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1438224Z [cluster1] E0716 18:13:51.295163       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1441784Z [cluster1] E0716 18:13:51.380145       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1445481Z [cluster1] E0716 18:13:51.690037       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1451604Z [cluster1] E0716 18:13:52.038193       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1454599Z [cluster1] I0716 18:13:52.316823       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1456983Z [cluster1] I0716 18:13:52.317186       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1458895Z [cluster1] I0716 18:13:52.371940       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"  
2021-07-16T18:15:08.1460795Z [cluster1] I0716 18:13:52.372115       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.1.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.1.0.0/16"]
2021-07-16T18:15:08.1462370Z [cluster1] I0716 18:13:52.372189       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1463827Z [cluster1] I0716 18:13:52.372264       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1465239Z [cluster1] I0716 18:13:52.376918       1 submariner_networkdiscovery.go:42] controller_submariner "msg"="Cluster network discovered"  
2021-07-16T18:15:08.1467085Z [cluster1] I0716 18:13:52.376947       1 network.go:56] controller_submariner "msg"="Discovered K8s network details"  "clusterCIDRs"=["10.1.0.0/16"] "plugin"="weave-net" "serviceCIDRs"=["100.1.0.0/16"]
2021-07-16T18:15:08.1468639Z [cluster1] I0716 18:13:52.376959       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1470271Z [cluster1] I0716 18:13:52.376968       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1472989Z [cluster1] I0716 18:13:52.394499       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1478293Z [cluster1] E0716 18:13:52.423362       1 controller.go:267] controller-runtime/manager/controller/submariner-controller "msg"="Reconciler error" "error"="error creating or updating DaemonSet submariner-operator/submariner-gateway: daemonsets.apps \"submariner-gateway\" already exists" "name"="submariner" "namespace"="submariner-operator" 
2021-07-16T18:15:08.1482090Z [cluster1] I0716 18:13:52.423572       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1484041Z [cluster1] I0716 18:13:52.423693       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1485532Z [cluster1] I0716 18:13:52.423770       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1488177Z [cluster1] I0716 18:13:52.486012       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1492967Z [cluster1] I0716 18:13:52.518387       1 helpers.go:231] controller_submariner "msg"="Created a new Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1497056Z [cluster1] E0716 18:13:52.518457       1 controller.go:267] controller-runtime/manager/controller/submariner-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-gateway-metrics: Service \"submariner-gateway-metrics\" not found" "name"="submariner" "namespace"="submariner-operator" 
2021-07-16T18:15:08.1500389Z [cluster1] I0716 18:13:52.518518       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1502651Z [cluster1] I0716 18:13:52.518549       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1504173Z [cluster1] I0716 18:13:52.518559       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1506854Z [cluster1] I0716 18:13:52.602655       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1511106Z [cluster1] E0716 18:13:52.737472       1 controller.go:267] controller-runtime/manager/controller/submariner "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-gateway-metrics: services \"submariner-gateway-metrics\" already exists" "name"="submariner" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Submariner" 
2021-07-16T18:15:08.1516277Z [cluster1] I0716 18:13:52.743419       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1518061Z [cluster1] I0716 18:13:52.743664       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1519537Z [cluster1] I0716 18:13:52.743893       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1522491Z [cluster1] I0716 18:13:52.752493       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1526110Z [cluster1] I0716 18:13:52.752833       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1530498Z [cluster1] E0716 18:13:52.771134       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1540965Z [cluster1] I0716 18:13:52.777851       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1544667Z [cluster1] I0716 18:13:53.154876       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1548093Z [cluster1] I0716 18:13:53.159570       1 helpers.go:83] controller_submariner "msg"="Created a new DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1551968Z [cluster1] I0716 18:13:53.166087       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1555279Z [cluster1] I0716 18:13:53.167338       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1558580Z [cluster1] I0716 18:13:53.169421       1 servicediscovery_resources.go:66] controller_submariner "msg"="Created Service Discovery CR" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Name"="service-discovery" "Namespace"="submariner-operator"
2021-07-16T18:15:08.1562231Z [cluster1] I0716 18:13:53.186838       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1566172Z [cluster1] I0716 18:13:53.189261       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1570802Z [cluster1] E0716 18:13:53.267696       1 controller.go:267] controller-runtime/manager/controller/servicediscovery "msg"="Reconciler error" "error"="error creating or updating Deployment submariner-operator/submariner-lighthouse-agent: deployments.apps \"submariner-lighthouse-agent\" already exists" "name"="service-discovery" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="ServiceDiscovery" 
2021-07-16T18:15:08.1587127Z [cluster1] I0716 18:13:53.273029       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1590572Z [cluster1] I0716 18:13:53.291759       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1593556Z [cluster1] I0716 18:13:53.422119       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1595341Z [cluster1] I0716 18:13:53.422160       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1596802Z [cluster1] I0716 18:13:53.422171       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1599916Z [cluster1] E0716 18:13:53.468704       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1603308Z [cluster1] I0716 18:13:53.470556       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1606288Z [cluster1] I0716 18:13:53.470618       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1608084Z [cluster1] I0716 18:13:53.470631       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1611131Z [cluster1] I0716 18:13:53.471934       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1615024Z [cluster1] I0716 18:13:53.512428       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1629257Z [cluster1] I0716 18:13:53.634570       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1633039Z [cluster1] I0716 18:13:53.696041       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1636994Z [cluster1] I0716 18:13:53.893452       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1640924Z [cluster1] I0716 18:13:53.894013       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1645075Z [cluster1] I0716 18:13:53.913326       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1649138Z [cluster1] I0716 18:13:54.030899       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1652890Z [cluster1] I0716 18:13:54.153886       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1656545Z [cluster1] I0716 18:13:54.228099       1 helpers.go:175] controller_servicediscovery "msg"="Created a new ConfigMap" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "ConfigMap.Name"="submariner-lighthouse-coredns" "ConfigMap.Namespace"="submariner-operator"
2021-07-16T18:15:08.1660686Z [cluster1] E0716 18:13:54.228148       1 servicediscovery_controller.go:151] controller_servicediscovery "msg"="Error creating the lighthouseCoreDNS configMap" "error"="error creating or updating ConfigMap submariner-operator/submariner-lighthouse-coredns: ConfigMap \"submariner-lighthouse-coredns\" not found"  
2021-07-16T18:15:08.1665465Z [cluster1] E0716 18:13:54.228184       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating ConfigMap submariner-operator/submariner-lighthouse-coredns: ConfigMap \"submariner-lighthouse-coredns\" not found" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-16T18:15:08.1669379Z [cluster1] I0716 18:13:54.228214       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1672599Z [cluster1] I0716 18:13:54.230921       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1676044Z [cluster1] I0716 18:13:54.295947       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1678975Z [cluster1] I0716 18:13:54.309887       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1680754Z [cluster1] I0716 18:13:54.309942       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1682214Z [cluster1] I0716 18:13:54.309952       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1685404Z [cluster1] I0716 18:13:54.381805       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1689518Z [cluster1] I0716 18:13:54.407401       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1701396Z [cluster1] I0716 18:13:54.407929       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1705196Z [cluster1] E0716 18:13:54.408266       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1709356Z [cluster1] I0716 18:13:54.408327       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1713283Z [cluster1] E0716 18:13:54.443584       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1716478Z [cluster1] I0716 18:13:54.443750       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1718251Z [cluster1] I0716 18:13:54.443836       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1719715Z [cluster1] I0716 18:13:54.443886       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1722492Z [cluster1] I0716 18:13:54.471101       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1726341Z [cluster1] I0716 18:13:54.476071       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1730307Z [cluster1] I0716 18:13:54.486923       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1733831Z [cluster1] I0716 18:13:54.730976       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1737590Z [cluster1] I0716 18:13:54.750369       1 helpers.go:129] controller_servicediscovery "msg"="Created a new Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1741421Z [cluster1] I0716 18:13:54.812299       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1744914Z [cluster1] I0716 18:13:54.907097       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1749250Z [cluster1] I0716 18:13:55.137874       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1753131Z [cluster1] I0716 18:13:55.882651       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1756924Z [cluster1] I0716 18:13:55.885249       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1770039Z [cluster1] I0716 18:13:55.899521       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1773734Z [cluster1] I0716 18:13:56.558305       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1778803Z [cluster1] E0716 18:13:56.647004       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1782053Z [cluster1] I0716 18:13:56.647049       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1783843Z [cluster1] I0716 18:13:56.647087       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1785314Z [cluster1] I0716 18:13:56.647097       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1788142Z [cluster1] I0716 18:13:56.772913       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1792062Z [cluster1] E0716 18:13:56.986786       1 servicediscovery_controller.go:169] controller_servicediscovery "msg"="Error creating the lighthouseCoreDNS service" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-coredns: services \"submariner-lighthouse-coredns\" already exists"  
2021-07-16T18:15:08.1796782Z [cluster1] E0716 18:13:56.986857       1 controller.go:267] controller-runtime/manager/controller/servicediscovery-controller "msg"="Reconciler error" "error"="error creating or updating Service submariner-operator/submariner-lighthouse-coredns: services \"submariner-lighthouse-coredns\" already exists" "name"="service-discovery" "namespace"="submariner-operator" 
2021-07-16T18:15:08.1800751Z [cluster1] I0716 18:13:56.986888       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1805371Z [cluster1] I0716 18:13:57.043818       1 helpers.go:231] controller_servicediscovery "msg"="Created a new Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1813155Z [cluster1] I0716 18:13:57.167796       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1817440Z [cluster1] I0716 18:13:57.182629       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1821329Z [cluster1] I0716 18:13:57.445813       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1824825Z [cluster1] I0716 18:13:57.569513       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1828128Z [cluster1] I0716 18:13:57.649941       1 servicediscovery_controller.go:487] controller_servicediscovery "msg"="coredns configmap does not have lighthouse configuration hence creating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1832687Z [cluster1] I0716 18:13:57.649971       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.97.160\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n"  
2021-07-16T18:15:08.1837001Z [cluster1] I0716 18:13:57.664673       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1842296Z [cluster1] E0716 18:13:57.675423       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1846866Z [cluster1] I0716 18:13:57.739379       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1850825Z [cluster1] I0716 18:13:58.142309       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1858206Z [cluster1] I0716 18:13:58.148634       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1862442Z [cluster1] I0716 18:13:58.158026       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1866366Z [cluster1] I0716 18:13:58.560645       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1869920Z [cluster1] I0716 18:13:58.572749       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1874461Z [cluster1] I0716 18:13:58.572787       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.97.160\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n"  
2021-07-16T18:15:08.1879208Z [cluster1] I0716 18:13:58.597351       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1882769Z [cluster1] I0716 18:13:58.608132       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1886986Z [cluster1] I0716 18:13:58.623325       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1891421Z [cluster1] I0716 18:13:59.032011       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1895441Z [cluster1] I0716 18:13:59.054019       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1899609Z [cluster1] I0716 18:13:59.130309       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1903488Z [cluster1] I0716 18:13:59.538890       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1907538Z [cluster1] I0716 18:13:59.555038       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1912348Z [cluster1] I0716 18:13:59.555081       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.97.160\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n"  
2021-07-16T18:15:08.1917273Z [cluster1] E0716 18:14:00.370196       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1920897Z [cluster1] E0716 18:14:05.918888       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.1924650Z [cluster1] I0716 18:14:10.494208       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1926431Z [cluster1] I0716 18:14:10.494260       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1927902Z [cluster1] I0716 18:14:10.494272       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1930587Z [cluster1] I0716 18:14:10.569328       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1934013Z [cluster1] I0716 18:14:10.637080       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1937465Z [cluster1] I0716 18:14:11.051175       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1940865Z [cluster1] I0716 18:14:11.115529       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1945188Z [cluster1] I0716 18:14:11.192656       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1947501Z [cluster1] I0716 18:14:11.192703       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1948985Z [cluster1] I0716 18:14:11.192715       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1951115Z [cluster1] I0716 18:14:11.198694       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1952879Z [cluster1] I0716 18:14:11.198750       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.1954348Z [cluster1] I0716 18:14:11.198761       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.1956979Z [cluster1] I0716 18:14:11.227741       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1961004Z [cluster1] I0716 18:14:11.244728       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1965002Z [cluster1] I0716 18:14:11.251809       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1968932Z [cluster1] I0716 18:14:11.251911       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1973029Z [cluster1] I0716 18:14:11.655465       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1976621Z [cluster1] I0716 18:14:11.656592       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.1980435Z [cluster1] I0716 18:14:11.664162       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1983932Z [cluster1] I0716 18:14:11.699506       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.1987113Z [cluster1] I0716 18:14:11.905201       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.1990745Z [cluster1] I0716 18:14:11.938688       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.1994865Z [cluster1] I0716 18:14:11.984305       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.1998740Z [cluster1] I0716 18:14:12.391548       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2002611Z [cluster1] I0716 18:14:12.400759       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.2006990Z [cluster1] I0716 18:14:12.434896       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2011653Z [cluster1] I0716 18:14:12.842178       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2015896Z [cluster1] I0716 18:14:12.857692       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2020450Z [cluster1] I0716 18:14:12.857742       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.97.160\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n"  
2021-07-16T18:15:08.2024497Z [cluster1] I0716 18:14:15.468779       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2027978Z [cluster1] I0716 18:14:15.637823       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.2031120Z [cluster1] I0716 18:14:15.663879       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2032892Z [cluster1] I0716 18:14:15.663928       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2034364Z [cluster1] I0716 18:14:15.663951       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2037990Z [cluster1] I0716 18:14:15.757098       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2041735Z [cluster1] I0716 18:14:15.798073       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2046317Z [cluster1] I0716 18:14:15.978459       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2049978Z [cluster1] I0716 18:14:16.158705       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2053815Z [cluster1] I0716 18:14:16.344883       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.2057538Z [cluster1] I0716 18:14:16.429164       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2061074Z [cluster1] I0716 18:14:16.459972       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2064927Z [cluster1] I0716 18:14:16.482329       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2068177Z [cluster1] I0716 18:14:16.543655       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2069923Z [cluster1] I0716 18:14:16.543701       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2071395Z [cluster1] I0716 18:14:16.543711       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2073479Z [cluster1] I0716 18:14:16.589458       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2075218Z [cluster1] I0716 18:14:16.589499       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2076675Z [cluster1] I0716 18:14:16.589509       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2079284Z [cluster1] I0716 18:14:16.603963       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2083997Z [cluster1] I0716 18:14:16.641684       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2088362Z [cluster1] I0716 18:14:16.769844       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2092056Z [cluster1] I0716 18:14:16.913182       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2096146Z [cluster1] I0716 18:14:16.926869       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2099472Z [cluster1] I0716 18:14:16.969930       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2104028Z [cluster1] I0716 18:14:16.969969       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.97.160\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n"  
2021-07-16T18:15:08.2108595Z [cluster1] E0716 18:14:17.099831       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.2112056Z [cluster1] I0716 18:14:17.174975       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2115468Z [cluster1] I0716 18:14:17.195359       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2118863Z [cluster1] I0716 18:14:17.340896       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2122261Z [cluster1] I0716 18:14:17.363879       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2126764Z [cluster1] E0716 18:14:17.536562       1 submariner_controller.go:204] controller_submariner "msg"="failed to update the Submariner status" "error"="Operation cannot be fulfilled on submariners.submariner.io \"submariner\": the object has been modified; please apply your changes to the latest version and try again" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2130025Z [cluster1] I0716 18:14:17.536608       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2131896Z [cluster1] I0716 18:14:17.536646       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2133366Z [cluster1] I0716 18:14:17.536657       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2136750Z [cluster1] I0716 18:14:17.557943       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2140435Z [cluster1] I0716 18:14:17.611004       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2143876Z [cluster1] I0716 18:14:18.035625       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2148434Z [cluster1] I0716 18:14:18.092853       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2151595Z [cluster1] I0716 18:14:18.598146       1 servicediscovery_controller.go:109] controller_servicediscovery "msg"="Reconciling ServiceDiscovery" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2155257Z [cluster1] I0716 18:14:18.676740       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-agent" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.2159324Z [cluster1] I0716 18:14:18.821472       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-agent-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2163203Z [cluster1] I0716 18:14:19.232890       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2167618Z [cluster1] I0716 18:14:19.240900       1 helpers.go:131] controller_servicediscovery "msg"="Updated existing Deployment" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Deployment.Name"="submariner-lighthouse-coredns" "Deployment.Namespace"="submariner-operator"
2021-07-16T18:15:08.2171810Z [cluster1] I0716 18:14:19.252645       1 helpers.go:233] controller_servicediscovery "msg"="Updated existing Service" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-lighthouse-coredns-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2175737Z [cluster1] I0716 18:14:19.655218       1 metrics.go:55] controller_servicediscovery "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2180315Z [cluster1] I0716 18:14:19.658455       1 servicediscovery_controller.go:471] controller_servicediscovery "msg"="coredns configmap has lighthouse configuration hence updating" "Request.Name"="service-discovery" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2185032Z [cluster1] I0716 18:14:19.658512       1 servicediscovery_controller.go:495] controller_servicediscovery "msg"="Updated coredns ConfigMap #lighthouse-start AUTO-GENERATED SECTION. DO NOT EDIT\nclusterset.local:53 {\n    forward . 100.1.97.160\n}\n#lighthouse-end\n.:53 {\n    errors\n    health {\n       lameduck 5s\n    }\n    ready\n    kubernetes cluster1.local in-addr.arpa ip6.arpa {\n       pods insecure\n       fallthrough in-addr.arpa ip6.arpa\n       ttl 30\n    }\n    prometheus :9153\n    forward . /etc/resolv.conf\n    cache 30\n    loop\n    reload\n    loadbalance\n}\n\n\n\n\n\n"  
2021-07-16T18:15:08.2188809Z [cluster1] I0716 18:14:36.051202       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2190576Z [cluster1] I0716 18:14:36.051266       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2192102Z [cluster1] I0716 18:14:36.051276       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2194743Z [cluster1] I0716 18:14:36.064432       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2198185Z [cluster1] I0716 18:14:36.069678       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2201723Z [cluster1] I0716 18:14:36.470950       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2205704Z [cluster1] I0716 18:14:36.476699       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2208618Z [cluster1] I0716 18:14:36.487624       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2210996Z [cluster1] I0716 18:14:36.487635       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2212748Z [cluster1] I0716 18:14:36.487669       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2214235Z [cluster1] I0716 18:14:36.487675       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2215702Z [cluster1] I0716 18:14:36.487680       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2217157Z [cluster1] I0716 18:14:36.487685       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2219774Z [cluster1] I0716 18:14:36.495882       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2223162Z [cluster1] I0716 18:14:36.498028       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2226733Z [cluster1] I0716 18:14:36.501045       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2230255Z [cluster1] I0716 18:14:36.504675       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2233687Z [cluster1] I0716 18:14:36.903347       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2237900Z [cluster1] I0716 18:14:36.906757       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2241855Z [cluster1] I0716 18:14:36.910177       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2245633Z [cluster1] I0716 18:14:36.912006       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2249777Z [cluster1] E0716 18:14:37.771937       1 controller.go:267] controller-runtime/manager/controller/broker "msg"="Reconciler error" "error"="invalid GlobalnetCidrRange: invalid CIDR address: " "name"="submariner-broker" "namespace"="submariner-operator" "reconciler group"="submariner.io" "reconciler kind"="Broker" 
2021-07-16T18:15:08.2252737Z [cluster1] I0716 18:14:41.130779       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2254489Z [cluster1] I0716 18:14:41.131093       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2255964Z [cluster1] I0716 18:14:41.131244       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2258912Z [cluster1] I0716 18:14:41.150326       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2262368Z [cluster1] I0716 18:14:41.171839       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2266340Z [cluster1] I0716 18:14:41.574779       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2270639Z [cluster1] I0716 18:14:41.580328       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2273896Z [cluster1] I0716 18:14:41.590102       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2275680Z [cluster1] I0716 18:14:41.590165       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2277151Z [cluster1] I0716 18:14:41.590175       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2279248Z [cluster1] I0716 18:14:41.603715       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2281009Z [cluster1] I0716 18:14:41.603862       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2282468Z [cluster1] I0716 18:14:41.603882       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2285215Z [cluster1] I0716 18:14:41.613973       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2289367Z [cluster1] I0716 18:14:41.622940       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2293215Z [cluster1] I0716 18:14:41.624097       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2298405Z [cluster1] I0716 18:14:41.638786       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2301885Z [cluster1] I0716 18:14:42.025558       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2305286Z [cluster1] I0716 18:14:42.031136       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2308689Z [cluster1] I0716 18:14:42.040668       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2312091Z [cluster1] I0716 18:14:42.047698       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2314961Z [cluster1] I0716 18:14:46.167952       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2316721Z [cluster1] I0716 18:14:46.168016       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2318177Z [cluster1] I0716 18:14:46.168027       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2320961Z [cluster1] I0716 18:14:46.174254       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2325189Z [cluster1] I0716 18:14:46.178731       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2328679Z [cluster1] I0716 18:14:46.580034       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2332494Z [cluster1] I0716 18:14:46.584732       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2335419Z [cluster1] I0716 18:14:46.593057       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2337192Z [cluster1] I0716 18:14:46.593252       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2338774Z [cluster1] I0716 18:14:46.593341       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2341448Z [cluster1] I0716 18:14:46.595659       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2343221Z [cluster1] I0716 18:14:46.596006       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2344679Z [cluster1] I0716 18:14:46.596160       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2347313Z [cluster1] I0716 18:14:46.601434       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2351334Z [cluster1] I0716 18:14:46.601984       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2354826Z [cluster1] I0716 18:14:46.605388       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2358683Z [cluster1] I0716 18:14:46.605940       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2362163Z [cluster1] I0716 18:14:47.009517       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2365786Z [cluster1] I0716 18:14:47.009994       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2369794Z [cluster1] I0716 18:14:47.019588       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2373267Z [cluster1] I0716 18:14:47.025422       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2377126Z [cluster1] I0716 18:14:51.187331       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2378907Z [cluster1] I0716 18:14:51.187397       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2380363Z [cluster1] I0716 18:14:51.187407       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2383002Z [cluster1] I0716 18:14:51.195308       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2387065Z [cluster1] I0716 18:14:51.201229       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2390530Z [cluster1] I0716 18:14:51.603222       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2393991Z [cluster1] I0716 18:14:51.609193       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2397995Z [cluster1] I0716 18:14:51.619548       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2400262Z [cluster1] I0716 18:14:51.619603       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2402070Z [cluster1] I0716 18:14:51.619614       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2404985Z [cluster1] I0716 18:14:51.637440       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2409429Z [cluster1] I0716 18:14:51.637503       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2411504Z [cluster1] I0716 18:14:51.637515       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2415013Z [cluster1] I0716 18:14:51.645777       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2419678Z [cluster1] I0716 18:14:51.646480       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2424760Z [cluster1] I0716 18:14:51.655882       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2428508Z [cluster1] I0716 18:14:51.656323       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2431934Z [cluster1] I0716 18:14:52.058129       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2435739Z [cluster1] I0716 18:14:52.059123       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2439188Z [cluster1] I0716 18:14:52.069762       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2442803Z [cluster1] I0716 18:14:52.106778       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2445825Z [cluster1] I0716 18:14:56.199820       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2447600Z [cluster1] I0716 18:14:56.200051       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2449060Z [cluster1] I0716 18:14:56.200137       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2453320Z [cluster1] I0716 18:14:56.209939       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2456841Z [cluster1] I0716 18:14:56.217729       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2460259Z [cluster1] I0716 18:14:56.619169       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2463641Z [cluster1] I0716 18:14:56.634317       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2466651Z [cluster1] I0716 18:14:56.652055       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2468426Z [cluster1] I0716 18:14:56.652102       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2469878Z [cluster1] I0716 18:14:56.652112       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2471963Z [cluster1] I0716 18:14:56.654006       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2473731Z [cluster1] I0716 18:14:56.654145       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2475778Z [cluster1] I0716 18:14:56.654166       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2478543Z [cluster1] I0716 18:14:56.671809       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2482354Z [cluster1] I0716 18:14:56.672585       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2486115Z [cluster1] I0716 18:14:56.677231       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2489624Z [cluster1] I0716 18:14:56.696777       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2493087Z [cluster1] I0716 18:14:57.082371       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2496481Z [cluster1] I0716 18:14:57.090948       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2499897Z [cluster1] I0716 18:14:57.112228       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2503261Z [cluster1] I0716 18:14:57.133430       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2506145Z [cluster1] I0716 18:15:01.227549       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2507899Z [cluster1] I0716 18:15:01.227622       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2509361Z [cluster1] I0716 18:15:01.227639       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2512075Z [cluster1] I0716 18:15:01.244809       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2515534Z [cluster1] I0716 18:15:01.250128       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2518964Z [cluster1] I0716 18:15:01.652366       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2523460Z [cluster1] I0716 18:15:01.657681       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2526478Z [cluster1] I0716 18:15:01.673414       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2528234Z [cluster1] I0716 18:15:01.673691       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2529826Z [cluster1] I0716 18:15:01.673838       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2531903Z [cluster1] I0716 18:15:01.673622       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2533664Z [cluster1] I0716 18:15:01.675292       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2535124Z [cluster1] I0716 18:15:01.675383       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2537725Z [cluster1] I0716 18:15:01.703575       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2541094Z [cluster1] I0716 18:15:01.705796       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2544532Z [cluster1] I0716 18:15:01.720585       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2548011Z [cluster1] I0716 18:15:01.721914       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2551451Z [cluster1] I0716 18:15:02.124455       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2554920Z [cluster1] I0716 18:15:02.125351       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2558317Z [cluster1] I0716 18:15:02.132485       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2562142Z [cluster1] I0716 18:15:02.132615       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2565897Z [cluster1] I0716 18:15:06.253714       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2567671Z [cluster1] I0716 18:15:06.253771       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2569110Z [cluster1] I0716 18:15:06.253782       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2571770Z [cluster1] I0716 18:15:06.265083       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2576012Z [cluster1] I0716 18:15:06.272782       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2579436Z [cluster1] I0716 18:15:06.674785       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2582838Z [cluster1] I0716 18:15:06.679862       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2585712Z [cluster1] I0716 18:15:06.701238       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2587457Z [cluster1] I0716 18:15:06.701283       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2588911Z [cluster1] I0716 18:15:06.701294       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2590983Z [cluster1] I0716 18:15:06.702433       1 submariner_controller.go:103] controller_submariner "msg"="Reconciling Submariner" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" 
2021-07-16T18:15:08.2592747Z [cluster1] I0716 18:15:06.702465       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="10.1.0.0/16" "type"="Cluster"
2021-07-16T18:15:08.2594209Z [cluster1] I0716 18:15:06.702476       1 submariner_networkdiscovery.go:77] controller_submariner "msg"="Using detected CIDR"  "CIDR"="100.1.0.0/16" "type"="Service"
2021-07-16T18:15:08.2596839Z [cluster1] I0716 18:15:06.719196       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2600319Z [cluster1] I0716 18:15:06.719911       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-gateway" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.2603933Z [cluster1] I0716 18:15:06.728274       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2607436Z [cluster1] I0716 18:15:06.729138       1 helpers.go:233] controller_submariner "msg"="Updated existing Service" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "Service.Name"="submariner-gateway-metrics" "Service.Namespace"="submariner-operator"
2021-07-16T18:15:08.2611816Z [cluster1] I0716 18:15:07.133827       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2615171Z [cluster1] I0716 18:15:07.137900       1 metrics.go:55] controller_submariner "msg"="Install prometheus-operator in your cluster to create ServiceMonitor objects" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "error"="no ServiceMonitor registered with the API"
2021-07-16T18:15:08.2619106Z [cluster1] I0716 18:15:07.141771       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.8205427Z [cluster1] I0716 18:15:07.147803       1 helpers.go.988101       1 helpers.go:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:08.8208186Z [submariner-operator]$ [cluster2] verify_subm_cr
2021-07-16T18:15:08.8209409Z [submariner-operator]$ [cluster2] verify_subm_cr
2021-07-16T18:15:08.8210888Z [submariner-operator]$ [cluster2] kubectl get submariner --namespace=submariner-operator
2021-07-16T18:15:08.8212718Z [submariner-operator]$ [cluster2] kubectl get submariner --namespace=submariner-operator
2021-07-16T18:15:08.8214185Z [submariner-operator]$ [cluster2] grep submariner
2021-07-16T18:15:08.8215852Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner --namespace=submariner-operator
2021-07-16T18:15:08.8217039Z [cluster2] submariner   70s
2021-07-16T18:15:08.8218276Z [submariner-operator]$ [cluster2] json_file=/tmp/submariner.cluster2.json
2021-07-16T18:15:08.8220044Z [submariner-operator]$ [cluster2] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:08.8221770Z [submariner-operator]$ [cluster2] tee /tmp/submariner.cluster2.json
2021-07-16T18:15:08.8223522Z [submariner-operator]$ [cluster2] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:08.8225532Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:08.8226749Z [cluster2] {
2021-07-16T18:15:08.8227472Z [cluster2]     "apiVersion": "submariner.io/v1alpha1",
2021-07-16T18:15:08.8228267Z [cluster2]     "kind": "Submariner",
2021-07-16T18:15:08.8229116Z [cluster2]     "metadata": {
2021-07-16T18:15:08.8230088Z [cluster2]         "creationTimestamp": "2021-07-16T18:13:58Z",
2021-07-16T18:15:08.8230868Z [cluster2]         "generation": 1,
2021-07-16T18:15:08.8231598Z [cluster2]         "managedFields": [
2021-07-16T18:15:08.8232219Z [cluster2]             {
2021-07-16T18:15:08.8232961Z [cluster2]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-16T18:15:08.8234029Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:08.8234755Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:08.8235388Z [cluster2]                     "f:spec": {
2021-07-16T18:15:08.8235974Z [cluster2]                         ".": {},
2021-07-16T18:15:08.8236588Z [cluster2]                         "f:broker": {},
2021-07-16T18:15:08.8237335Z [cluster2]                         "f:brokerK8sApiServer": {},
2021-07-16T18:15:08.8238281Z [cluster2]                         "f:brokerK8sApiServerToken": {},
2021-07-16T18:15:08.8239153Z [cluster2]                         "f:brokerK8sCA": {},
2021-07-16T18:15:08.8240021Z [cluster2]                         "f:brokerK8sRemoteNamespace": {},
2021-07-16T18:15:08.8240938Z [cluster2]                         "f:cableDriver": {},
2021-07-16T18:15:08.8241678Z [cluster2]                         "f:ceIPSecDebug": {},
2021-07-16T18:15:08.8242440Z [cluster2]                         "f:ceIPSecIKEPort": {},
2021-07-16T18:15:08.8243246Z [cluster2]                         "f:ceIPSecNATTPort": {},
2021-07-16T18:15:08.8244602Z [cluster2]                         "f:ceIPSecPSK": {},
2021-07-16T18:15:08.8245321Z [cluster2]                         "f:clusterCIDR": {},
2021-07-16T18:15:08.8246025Z [cluster2]                         "f:clusterID": {},
2021-07-16T18:15:08.8246704Z [cluster2]                         "f:colorCodes": {},
2021-07-16T18:15:08.8247540Z [cluster2]                         "f:connectionHealthCheck": {
2021-07-16T18:15:08.8248442Z [cluster2]                             ".": {},
2021-07-16T18:15:08.8249072Z [cluster2]                             "f:enabled": {},
2021-07-16T18:15:08.8249815Z [cluster2]                             "f:intervalSeconds": {},
2021-07-16T18:15:08.8250643Z [cluster2]                             "f:maxPacketLossCount": {}
2021-07-16T18:15:08.8251364Z [cluster2]                         },
2021-07-16T18:15:08.8251969Z [cluster2]                         "f:debug": {},
2021-07-16T18:15:08.8252646Z [cluster2]                         "f:namespace": {},
2021-07-16T18:15:08.8253330Z [cluster2]                         "f:natEnabled": {},
2021-07-16T18:15:08.8254014Z [cluster2]                         "f:repository": {},
2021-07-16T18:15:08.8254728Z [cluster2]                         "f:serviceCIDR": {},
2021-07-16T18:15:08.8255598Z [cluster2]                         "f:serviceDiscoveryEnabled": {},
2021-07-16T18:15:08.8256416Z [cluster2]                         "f:version": {}
2021-07-16T18:15:08.8257017Z [cluster2]                     }
2021-07-16T18:15:08.8257551Z [cluster2]                 },
2021-07-16T18:15:08.8258172Z [cluster2]                 "manager": "subctl",
2021-07-16T18:15:08.8258866Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:08.8259863Z [cluster2]                 "time": "2021-07-16T18:13:58Z"
2021-07-16T18:15:08.8260464Z [cluster2]             },
2021-07-16T18:15:08.8260978Z [cluster2]             {
2021-07-16T18:15:08.8261726Z [cluster2]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-16T18:15:08.8262566Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:08.8263244Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:08.8263880Z [cluster2]                     "f:status": {
2021-07-16T18:15:08.8264458Z [cluster2]                         ".": {},
2021-07-16T18:15:08.8265104Z [cluster2]                         "f:clusterCIDR": {},
2021-07-16T18:15:08.8265799Z [cluster2]                         "f:clusterID": {},
2021-07-16T18:15:08.8266481Z [cluster2]                         "f:colorCodes": {},
2021-07-16T18:15:08.8267216Z [cluster2]                         "f:deploymentInfo": {},
2021-07-16T18:15:08.8268089Z [cluster2]                         "f:gatewayDaemonSetStatus": {
2021-07-16T18:15:08.8268872Z [cluster2]                             ".": {},
2021-07-16T18:15:08.8269615Z [cluster2]                             "f:lastResourceVersion": {},
2021-07-16T18:15:08.8270727Z [cluster2]                             "f:mismatchedContainerImages": {},
2021-07-16T18:15:08.8271827Z [cluster2]                             "f:nonReadyContainerStates": {},
2021-07-16T18:15:08.8272667Z [cluster2]                             "f:status": {
2021-07-16T18:15:08.8273275Z [cluster2]                                 ".": {},
2021-07-16T18:15:08.8274083Z [cluster2]                                 "f:currentNumberScheduled": {},
2021-07-16T18:15:08.8275080Z [cluster2]                                 "f:desiredNumberScheduled": {},
2021-07-16T18:15:08.8276015Z [cluster2]                                 "f:numberAvailable": {},
2021-07-16T18:15:08.8276866Z [cluster2]                                 "f:numberMisscheduled": {},
2021-07-16T18:15:08.8277670Z [cluster2]                                 "f:numberReady": {},
2021-07-16T18:15:08.8278497Z [cluster2]                                 "f:observedGeneration": {},
2021-07-16T18:15:08.8279448Z [cluster2]                                 "f:updatedNumberScheduled": {}
2021-07-16T18:15:08.8280232Z [cluster2]                             }
2021-07-16T18:15:08.8280808Z [cluster2]                         },
2021-07-16T18:15:08.8281408Z [cluster2]                         "f:gateways": {},
2021-07-16T18:15:08.8282279Z [cluster2]                         "f:globalnetDaemonSetStatus": {
2021-07-16T18:15:08.8283101Z [cluster2]                             ".": {},
2021-07-16T18:15:08.8284043Z [cluster2]                             "f:mismatchedContainerImages": {}
2021-07-16T18:15:08.8284876Z [cluster2]                         },
2021-07-16T18:15:08.8285852Z [cluster2]                         "f:loadBalancerStatus": {},
2021-07-16T18:15:08.8286633Z [cluster2]                         "f:natEnabled": {},
2021-07-16T18:15:08.8327386Z [cluster2]                         "f:networkPlugin": {},
2021-07-16T18:15:08.8328367Z [cluster2]                         "f:routeAgentDaemonSetStatus": {
2021-07-16T18:15:08.8329210Z [cluster2]                             ".": {},
2021-07-16T18:15:08.8329990Z [cluster2]                             "f:lastResourceVersion": {},
2021-07-16T18:15:08.8330979Z [cluster2]                             "f:mismatchedContainerImages": {},
2021-07-16T18:15:08.8332040Z [cluster2]                             "f:nonReadyContainerStates": {},
2021-07-16T18:15:08.8332856Z [cluster2]                             "f:status": {
2021-07-16T18:15:08.8333470Z [cluster2]                                 ".": {},
2021-07-16T18:15:08.8334292Z [cluster2]                                 "f:currentNumberScheduled": {},
2021-07-16T18:15:08.8335301Z [cluster2]                                 "f:desiredNumberScheduled": {},
2021-07-16T18:15:08.8336209Z [cluster2]                                 "f:numberAvailable": {},
2021-07-16T18:15:08.8337043Z [cluster2]                                 "f:numberMisscheduled": {},
2021-07-16T18:15:08.8337854Z [cluster2]                                 "f:numberReady": {},
2021-07-16T18:15:08.8338670Z [cluster2]                                 "f:observedGeneration": {},
2021-07-16T18:15:08.8339603Z [cluster2]                                 "f:updatedNumberScheduled": {}
2021-07-16T18:15:08.8340426Z [cluster2]                             }
2021-07-16T18:15:08.8340993Z [cluster2]                         },
2021-07-16T18:15:08.8341617Z [cluster2]                         "f:serviceCIDR": {}
2021-07-16T18:15:08.8342253Z [cluster2]                     }
2021-07-16T18:15:08.8342890Z [cluster2]                 },
2021-07-16T18:15:08.8344155Z [cluster2]                 "manager": "submariner-operator",
2021-07-16T18:15:08.8345017Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:08.8345890Z [cluster2]                 "time": "2021-07-16T18:14:35Z"
2021-07-16T18:15:08.8346489Z [cluster2]             }
2021-07-16T18:15:08.8347005Z [cluster2]         ],
2021-07-16T18:15:08.8347618Z [cluster2]         "name": "submariner",
2021-07-16T18:15:08.8348615Z [cluster2]         "namespace": "submariner-operator",
2021-07-16T18:15:08.8349481Z [cluster2]         "resourceVersion": "2251",
2021-07-16T18:15:08.8351186Z [cluster2]         "selfLink": "/apis/submariner.io/v1alpha1/namespaces/submariner-operator/submariners/submariner",
2021-07-16T18:15:08.8352712Z [cluster2]         "uid": "95ae3111-f54b-40e8-acc9-f10b14121b67"
2021-07-16T18:15:08.8353476Z [cluster2]     },
2021-07-16T18:15:08.8354031Z [cluster2]     "spec": {
2021-07-16T18:15:08.8354605Z [cluster2]         "broker": "k8s",
2021-07-16T18:15:08.8355350Z [cluster2]         "brokerK8sApiServer": "172.18.0.8:6443",
2021-07-16T18:15:08.8517032Z [cluster2]         "brokerK8sApiServerToken": "***",
2021-07-16T18:15:08.9087018Z [cluster2]         "brokerK8sCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=",
2021-07-16T18:15:08.9663082Z [cluster2]         "brokerK8sRemoteNamespace": "submariner-k8s-broker",
2021-07-16T18:15:08.9667529Z [cluster2]         "cableDriver": "libreswan",
2021-07-16T18:15:08.9669079Z [cluster2]         "ceIPSecDebug": false,
2021-07-16T18:15:08.9670551Z [cluster2]         "ceIPSecIKEPort": 500,
2021-07-16T18:15:08.9672043Z [cluster2]         "ceIPSecNATTPort": 4500,
2021-07-16T18:15:08.9674233Z [cluster2]         "ceIPSecPSK": "Zhs+SumA5VO7u+aRnCe5Wjoau5QpxSYF1mxiCBvpnGHaZ/d40WAFSnV803hZx1RR",
2021-07-16T18:15:08.9676319Z [cluster2]         "clusterCIDR": "",
2021-07-16T18:15:08.9677762Z [cluster2]         "clusterID": "cluster2",
2021-07-16T18:15:08.9679191Z [cluster2]         "colorCodes": "blue",
2021-07-16T18:15:08.9680699Z [cluster2]         "connectionHealthCheck": {
2021-07-16T18:15:08.9682211Z [cluster2]             "enabled": true,
2021-07-16T18:15:08.9683736Z [cluster2]             "intervalSeconds": 1,
2021-07-16T18:15:08.9685283Z [cluster2]             "maxPacketLossCount": 5
2021-07-16T18:15:08.9686698Z [cluster2]         },
2021-07-16T18:15:08.9687990Z [cluster2]         "debug": false,
2021-07-16T18:15:08.9689955Z [cluster2]         "namespace": "submariner-operator",
2021-07-16T18:15:08.9691528Z [cluster2]         "natEnabled": false,
2021-07-16T18:15:08.9693469Z [cluster2]         "repository": "localhost:5000",
2021-07-16T18:15:08.9695058Z [cluster2]         "serviceCIDR": "",
2021-07-16T18:15:08.9696696Z [cluster2]         "serviceDiscoveryEnabled": true,
2021-07-16T18:15:08.9698222Z [cluster2]         "version": "local"
2021-07-16T18:15:08.9699488Z [cluster2]     },
2021-07-16T18:15:08.9700739Z [cluster2]     "status": {
2021-07-16T18:15:08.9702076Z [cluster2]         "clusterCIDR": "10.2.0.0/16",
2021-07-16T18:15:08.9703484Z [cluster2]         "clusterID": "cluster2",
2021-07-16T18:15:08.9704985Z [cluster2]         "colorCodes": "blue",
2021-07-16T18:15:08.9706528Z [cluster2]         "deploymentInfo": {},
2021-07-16T18:15:08.9708103Z [cluster2]         "gatewayDaemonSetStatus": {
2021-07-16T18:15:08.9710003Z [cluster2]             "lastResourceVersion": "2034",
2021-07-16T18:15:08.9712766Z [cluster2]             "mismatchedContainerImages": false,
2021-07-16T18:15:08.9714505Z [cluster2]             "nonReadyContainerStates": [],
2021-07-16T18:15:08.9716564Z [cluster2]             "status": {
2021-07-16T18:15:08.9718261Z [cluster2]                 "currentNumberScheduled": 1,
2021-07-16T18:15:08.9719921Z [cluster2]                 "desiredNumberScheduled": 1,
2021-07-16T18:15:08.9721716Z [cluster2]                 "numberAvailable": 1,
2021-07-16T18:15:08.9723480Z [cluster2]                 "numberMisscheduled": 0,
2021-07-16T18:15:08.9725074Z [cluster2]                 "numberReady": 1,
2021-07-16T18:15:08.9726563Z [cluster2]                 "observedGeneration": 1,
2021-07-16T18:15:08.9728169Z [cluster2]                 "updatedNumberScheduled": 1
2021-07-16T18:15:08.9729633Z [cluster2]             }
2021-07-16T18:15:08.9730873Z [cluster2]         },
2021-07-16T18:15:08.9732138Z [cluster2]         "gateways": [
2021-07-16T18:15:08.9733439Z [cluster2]             {
2021-07-16T18:15:08.9734763Z [cluster2]                 "connections": [
2021-07-16T18:15:08.9736103Z [cluster2]                     {
2021-07-16T18:15:08.9737418Z [cluster2]                         "endpoint": {
2021-07-16T18:15:08.9738851Z [cluster2]                             "backend": "libreswan",
2021-07-16T18:15:08.9740324Z [cluster2]                             "backend_config": {
2021-07-16T18:15:08.9742255Z [cluster2]                                 "natt-discovery-port": "4490",
2021-07-16T18:15:08.9744460Z [cluster2]                                 "preferred-server": "false"
2021-07-16T18:15:08.9745907Z [cluster2]                             },
2021-07-16T18:15:08.9749537Z [cluster2]                             "cable_name": "submariner-cable-cluster1-172-18-0-9",
2021-07-16T18:15:09.0119448Z [cluster2]                             "cluster_id": "cluster1",
2021-07-16T18:15:09.0120550Z [cluster2]                             "healthCheckIP": "10.1.0.1",
2021-07-16T18:15:09.0122046Z [cluster2]                             "hostname": "cluster1-worker",
2021-07-16T18:15:09.0122890Z [cluster2]                             "nat_enabled": false,
2021-07-16T18:15:09.0124326Z [cluster2]                             "private_ip": "172.18.0.9",
2021-07-16T18:15:09.0125019Z [cluster2]                             "public_ip": "40.71.64.254",
2021-07-16T18:15:09.0125688Z [cluster2]                             "subnets": [
2021-07-16T18:15:09.0126397Z [cluster2]                                 "100.1.0.0/16",
2021-07-16T18:15:09.0127019Z [cluster2]                                 "10.1.0.0/16"
2021-07-16T18:15:09.0127614Z [cluster2]                             ]
2021-07-16T18:15:09.0128287Z [cluster2]                         },
2021-07-16T18:15:09.0128887Z [cluster2]                         "latencyRTT": {
2021-07-16T18:15:09.0129866Z [cluster2]                             "average": "192.315µs",
2021-07-16T18:15:09.0130783Z [cluster2]                             "last": "173.305µs",
2021-07-16T18:15:09.0131635Z [cluster2]                             "max": "334.409µs",
2021-07-16T18:15:09.0132467Z [cluster2]                             "min": "140.504µs",
2021-07-16T18:15:09.0133278Z [cluster2]                             "stdDev": "46.936µs"
2021-07-16T18:15:09.0133909Z [cluster2]                         },
2021-07-16T18:15:09.0134560Z [cluster2]                         "status": "connected",
2021-07-16T18:15:09.0135295Z [cluster2]                         "statusMessage": "",
2021-07-16T18:15:09.0136110Z [cluster2]                         "usingIP": "172.18.0.9"
2021-07-16T18:15:09.0136733Z [cluster2]                     }
2021-07-16T18:15:09.0137345Z [cluster2]                 ],
2021-07-16T18:15:09.0137985Z [cluster2]                 "haStatus": "active",
2021-07-16T18:15:09.0138698Z [cluster2]                 "localEndpoint": {
2021-07-16T18:15:09.0139431Z [cluster2]                     "backend": "libreswan",
2021-07-16T18:15:09.0140102Z [cluster2]                     "backend_config": {
2021-07-16T18:15:09.0141369Z [cluster2]                         "natt-discovery-port": "4490",
2021-07-16T18:15:09.0143089Z [cluster2]                         "preferred-server": "false"
2021-07-16T18:15:09.0171323Z [cluster2]                     },
2021-07-16T18:15:09.0172781Z [cluster2]                     "cable_name": "submariner-cable-cluster2-172-18-0-5",
2021-07-16T18:15:09.0173777Z [cluster2]                     "cluster_id": "cluster2",
2021-07-16T18:15:09.0174435Z [cluster2]                     "healthCheckIP": "10.2.128.0",
2021-07-16T18:15:09.0175332Z [cluster2]                     "hostname": "cluster2-worker",
2021-07-16T18:15:09.0176012Z [cluster2]                     "nat_enabled": false,
2021-07-16T18:15:09.0176589Z [cluster2]                     "private_ip": "172.18.0.5",
2021-07-16T18:15:09.0177162Z [cluster2]                     "public_ip": "40.71.64.254",
2021-07-16T18:15:09.0177699Z [cluster2]                     "subnets": [
2021-07-16T18:15:09.0178221Z [cluster2]                         "100.2.0.0/16",
2021-07-16T18:15:09.0178707Z [cluster2]                         "10.2.0.0/16"
2021-07-16T18:15:09.0179161Z [cluster2]                     ]
2021-07-16T18:15:09.0179610Z [cluster2]                 },
2021-07-16T18:15:09.0180141Z [cluster2]                 "statusFailure": "",
2021-07-16T18:15:09.0180979Z [cluster2]                 "version": "devel-7cad5c5"
2021-07-16T18:15:09.0181524Z [cluster2]             }
2021-07-16T18:15:09.0181958Z [cluster2]         ],
2021-07-16T18:15:09.0182855Z [cluster2]         "globalnetDaemonSetStatus": {
2021-07-16T18:15:09.0183811Z [cluster2]             "mismatchedContainerImages": false
2021-07-16T18:15:09.0184504Z [cluster2]         },
2021-07-16T18:15:09.0185098Z [cluster2]         "loadBalancerStatus": {},
2021-07-16T18:15:09.0185769Z [cluster2]         "natEnabled": false,
2021-07-16T18:15:09.0186606Z [cluster2]         "networkPlugin": "weave-net",
2021-07-16T18:15:09.0187439Z [cluster2]         "routeAgentDaemonSetStatus": {
2021-07-16T18:15:09.0188430Z [cluster2]             "lastResourceVersion": "2084",
2021-07-16T18:15:09.0189327Z [cluster2]             "mismatchedContainerImages": false,
2021-07-16T18:15:09.0190239Z [cluster2]             "nonReadyContainerStates": [],
2021-07-16T18:15:09.0190933Z [cluster2]             "status": {
2021-07-16T18:15:09.0191606Z [cluster2]                 "currentNumberScheduled": 3,
2021-07-16T18:15:09.0192472Z [cluster2]                 "desiredNumberScheduled": 3,
2021-07-16T18:15:09.0193254Z [cluster2]                 "numberAvailable": 3,
2021-07-16T18:15:09.0193954Z [cluster2]                 "numberMisscheduled": 0,
2021-07-16T18:15:09.0194627Z [cluster2]                 "numberReady": 3,
2021-07-16T18:15:09.0195278Z [cluster2]                 "observedGeneration": 1,
2021-07-16T18:15:09.0196094Z [cluster2]                 "updatedNumberScheduled": 3
2021-07-16T18:15:09.0196732Z [cluster2]             }
2021-07-16T18:15:09.0197166Z [cluster2]         },
2021-07-16T18:15:09.0197674Z [cluster2]         "serviceCIDR": "100.2.0.0/16"
2021-07-16T18:15:09.0198184Z [cluster2]     }
2021-07-16T18:15:09.0198586Z [cluster2] }
2021-07-16T18:15:09.0199787Z [submariner-operator]$ [cluster2] validate_equals .metadata.namespace submariner-operator
2021-07-16T18:15:09.0201500Z [submariner-operator]$ [cluster2] validate_equals .metadata.namespace submariner-operator
2021-07-16T18:15:09.0203214Z [submariner-operator]$ [cluster2] jq -r -M .metadata.namespace /tmp/submariner.cluster2.json
2021-07-16T18:15:09.0223432Z [submariner-operator]$ [cluster2] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-16T18:15:09.0224967Z [submariner-operator]$ [cluster2] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-16T18:15:09.0226513Z [submariner-operator]$ [cluster2] jq -r -M .apiVersion /tmp/submariner.cluster2.json
2021-07-16T18:15:09.0227919Z [submariner-operator]$ [cluster2] validate_equals .kind Submariner
2021-07-16T18:15:09.0348396Z [submariner-operator]$ [cluster2] validate_equals .ki:85] controller_submariner "msg"="Updated existing DaemonSet" "Request.Name"="submariner" "Request.Namespace"="submariner-operator" "DaemonSet.Name"="submariner-routeagent" "DaemonSet.Namespace"="submariner-operator"
2021-07-16T18:15:09.0350761Z [submariner-operator]$ [cluster1] verify_subm_cr
2021-07-16T18:15:09.0351806Z [submariner-operator]$ [cluster1] verify_subm_cr
2021-07-16T18:15:09.0353148Z [submariner-operator]$ [cluster1] kubectl get submariner --namespace=submariner-operator
2021-07-16T18:15:09.0354744Z [submariner-operator]$ [cluster1] kubectl get submariner --namespace=submariner-operator
2021-07-16T18:15:09.0356497Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner --namespace=submariner-operator
2021-07-16T18:15:09.0357927Z [submariner-operator]$ [cluster1] grep submariner
2021-07-16T18:15:09.0358657Z [cluster1] submariner   76s
2021-07-16T18:15:09.0359757Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner.cluster1.json
2021-07-16T18:15:09.0361592Z [submariner-operator]$ [cluster1] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:09.0363196Z [submariner-operator]$ [cluster1] tee /tmp/submariner.cluster1.json
2021-07-16T18:15:09.0365403Z [submariner-operator]$ [cluster1] kubectl get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:09.0367331Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:09.0368569Z [cluster1] {
2021-07-16T18:15:09.0369192Z [cluster1]     "apiVersion": "submariner.io/v1alpha1",
2021-07-16T18:15:09.0369881Z [cluster1]     "kind": "Submariner",
2021-07-16T18:15:09.0370411Z [cluster1]     "metadata": {
2021-07-16T18:15:09.0371302Z [cluster1]         "creationTimestamp": "2021-07-16T18:13:52Z",
2021-07-16T18:15:09.0371959Z [cluster1]         "generation": 1,
2021-07-16T18:15:09.0372545Z [cluster1]         "managedFields": [
2021-07-16T18:15:09.0373058Z [cluster1]             {
2021-07-16T18:15:09.0373691Z [cluster1]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-16T18:15:09.0374408Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:09.0375009Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:09.0375518Z [cluster1]                     "f:spec": {
2021-07-16T18:15:09.0376012Z [cluster1]                         ".": {},
2021-07-16T18:15:09.0376523Z [cluster1]                         "f:broker": {},
2021-07-16T18:15:09.0377164Z [cluster1]                         "f:brokerK8sApiServer": {},
2021-07-16T18:15:09.0378009Z [cluster1]                         "f:brokerK8sApiServerToken": {},
2021-07-16T18:15:09.0378763Z [cluster1]                         "f:brokerK8sCA": {},
2021-07-16T18:15:09.0379543Z [cluster1]                         "f:brokerK8sRemoteNamespace": {},
2021-07-16T18:15:09.0380316Z [cluster1]                         "f:cableDriver": {},
2021-07-16T18:15:09.0380951Z [cluster1]                         "f:ceIPSecDebug": {},
2021-07-16T18:15:09.0381610Z [cluster1]                         "f:ceIPSecIKEPort": {},
2021-07-16T18:15:09.0382310Z [cluster1]                         "f:ceIPSecNATTPort": {},
2021-07-16T18:15:09.0382975Z [cluster1]                         "f:ceIPSecPSK": {},
2021-07-16T18:15:09.0383581Z [cluster1]                         "f:clusterCIDR": {},
2021-07-16T18:15:09.0384182Z [cluster1]                         "f:clusterID": {},
2021-07-16T18:15:09.0384765Z [cluster1]                         "f:colorCodes": {},
2021-07-16T18:15:09.0385493Z [cluster1]                         "f:connectionHealthCheck": {
2021-07-16T18:15:09.0386148Z [cluster1]                             ".": {},
2021-07-16T18:15:09.0386673Z [cluster1]                             "f:enabled": {},
2021-07-16T18:15:09.0387294Z [cluster1]                             "f:intervalSeconds": {},
2021-07-16T18:15:09.0388144Z [cluster1]                             "f:maxPacketLossCount": {}
2021-07-16T18:15:09.0388774Z [cluster1]                         },
2021-07-16T18:15:09.0389260Z [cluster1]                         "f:debug": {},
2021-07-16T18:15:09.0389824Z [cluster1]                         "f:namespace": {},
2021-07-16T18:15:09.0390407Z [cluster1]                         "f:natEnabled": {},
2021-07-16T18:15:09.0391007Z [cluster1]                         "f:repository": {},
2021-07-16T18:15:09.0391608Z [cluster1]                         "f:serviceCIDR": {},
2021-07-16T18:15:09.0392384Z [cluster1]                         "f:serviceDiscoveryEnabled": {},
2021-07-16T18:15:09.0393106Z [cluster1]                         "f:version": {}
2021-07-16T18:15:09.0393612Z [cluster1]                     }
2021-07-16T18:15:09.0394044Z [cluster1]                 },
2021-07-16T18:15:09.0394558Z [cluster1]                 "manager": "subctl",
2021-07-16T18:15:09.0395150Z [cluster1]                 "operation": "Update",
2021-07-16T18:15:09.0395933Z [cluster1]                 "time": "2021-07-16T18:13:52Z"
2021-07-16T18:15:09.0396424Z [cluster1]             },
2021-07-16T18:15:09.0396848Z [cluster1]             {
2021-07-16T18:15:09.0397484Z [cluster1]                 "apiVersion": "submariner.io/v1alpha1",
2021-07-16T18:15:09.0398202Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:09.0398793Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:09.0399303Z [cluster1]                     "f:status": {
2021-07-16T18:15:09.0399884Z [cluster1]                         ".": {},
2021-07-16T18:15:09.0400484Z [cluster1]                         "f:clusterCIDR": {},
2021-07-16T18:15:09.0401085Z [cluster1]                         "f:clusterID": {},
2021-07-16T18:15:09.0401673Z [cluster1]                         "f:colorCodes": {},
2021-07-16T18:15:09.0402302Z [cluster1]                         "f:deploymentInfo": {},
2021-07-16T18:15:09.0403083Z [cluster1]                         "f:gatewayDaemonSetStatus": {
2021-07-16T18:15:09.0404327Z [cluster1]                             ".": {},
2021-07-16T18:15:09.0404986Z [cluster1]                             "f:lastResourceVersion": {},
2021-07-16T18:15:09.0405870Z [cluster1]                             "f:mismatchedContainerImages": {},
2021-07-16T18:15:09.0406838Z [cluster1]                             "f:nonReadyContainerStates": {},
2021-07-16T18:15:09.0407568Z [cluster1]                             "f:status": {
2021-07-16T18:15:09.0408072Z [cluster1]                                 ".": {},
2021-07-16T18:15:09.0408779Z [cluster1]                                 "f:currentNumberScheduled": {},
2021-07-16T18:15:09.0409669Z [cluster1]                                 "f:desiredNumberScheduled": {},
2021-07-16T18:15:09.0410485Z [cluster1]                                 "f:numberAvailable": {},
2021-07-16T18:15:09.0411221Z [cluster1]                                 "f:numberMisscheduled": {},
2021-07-16T18:15:09.0411936Z [cluster1]                                 "f:numberReady": {},
2021-07-16T18:15:09.0412639Z [cluster1]                                 "f:observedGeneration": {},
2021-07-16T18:15:09.0413482Z [cluster1]                                 "f:updatedNumberScheduled": {}
2021-07-16T18:15:09.0414167Z [cluster1]                             }
2021-07-16T18:15:09.0414619Z [cluster1]                         },
2021-07-16T18:15:09.0415138Z [cluster1]                         "f:gateways": {},
2021-07-16T18:15:09.0415879Z [cluster1]                         "f:globalnetDaemonSetStatus": {
2021-07-16T18:15:09.0416597Z [cluster1]                             ".": {},
2021-07-16T18:15:09.0417318Z [cluster1]                             "f:mismatchedContainerImages": {}
2021-07-16T18:15:09.0418037Z [cluster1]                         },
2021-07-16T18:15:09.0418759Z [cluster1]                         "f:loadBalancerStatus": {},
2021-07-16T18:15:09.0419445Z [cluster1]                         "f:natEnabled": {},
2021-07-16T18:15:09.0420080Z [cluster1]                         "f:networkPlugin": {},
2021-07-16T18:15:09.0421027Z [cluster1]                         "f:routeAgentDaemonSetStatus": {
2021-07-16T18:15:09.0421766Z [cluster1]                             ".": {},
2021-07-16T18:15:09.0422519Z [cluster1]                             "f:lastResourceVersion": {},
2021-07-16T18:15:09.0423422Z [cluster1]                             "f:mismatchedContainerImages": {},
2021-07-16T18:15:09.0424365Z [cluster1]                             "f:nonReadyContainerStates": {},
2021-07-16T18:15:09.0425183Z [cluster1]                             "f:status": {
2021-07-16T18:15:09.0425680Z [cluster1]                                 ".": {},
2021-07-16T18:15:09.0426380Z [cluster1]                                 "f:currentNumberScheduled": {},
2021-07-16T18:15:09.0427282Z [cluster1]                                 "f:desiredNumberScheduled": {},
2021-07-16T18:15:09.0428071Z [cluster1]                                 "f:numberAvailable": {},
2021-07-16T18:15:09.0428821Z [cluster1]                                 "f:numberMisscheduled": {},
2021-07-16T18:15:09.0429522Z [cluster1]                                 "f:numberReady": {},
2021-07-16T18:15:09.0430236Z [cluster1]                                 "f:observedGeneration": {},
2021-07-16T18:15:09.0431067Z [cluster1]                                 "f:updatedNumberScheduled": {}
2021-07-16T18:15:09.0437653Z [cluster1]                             }
2021-07-16T18:15:09.0438152Z [cluster1]                         },
2021-07-16T18:15:09.0438693Z [cluster1]                         "f:serviceCIDR": {}
2021-07-16T18:15:09.0439421Z [cluster1]                     }
2021-07-16T18:15:09.0439859Z [cluster1]                 },
2021-07-16T18:15:09.0440919Z [cluster1]                 "manager": "submariner-operator",
2021-07-16T18:15:09.0441634Z [cluster1]                 "operation": "Update",
2021-07-16T18:15:09.0442403Z [cluster1]                 "time": "2021-07-16T18:14:16Z"
2021-07-16T18:15:09.0442873Z [cluster1]             }
2021-07-16T18:15:09.0443309Z [cluster1]         ],
2021-07-16T18:15:09.0443933Z [cluster1]         "name": "submariner",
2021-07-16T18:15:09.0444833Z [cluster1]         "namespace": "submariner-operator",
2021-07-16T18:15:09.0445598Z [cluster1]         "resourceVersion": "2355",
2021-07-16T18:15:09.0446895Z [cluster1]         "selfLink": "/apis/submariner.io/v1alpha1/namespaces/submariner-operator/submariners/submariner",
2021-07-16T18:15:09.0452026Z [cluster1]         "uid": "816aef07-6dc5-4ae5-aff3-b8dcef57204e"
2021-07-16T18:15:09.0453594Z [cluster1]     },
2021-07-16T18:15:09.0454135Z [cluster1]     "spec": {
2021-07-16T18:15:09.0454621Z [cluster1]         "broker": "k8s",
2021-07-16T18:15:09.0455270Z [cluster1]         "brokerK8sApiServer": "172.18.0.8:6443",
2021-07-16T18:15:09.0658011Z [cluster1]         "brokerK8sApiServerToken": "***",
2021-07-16T18:15:09.1226491Z [cluster1]         "brokerK8sCA": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=",
2021-07-16T18:15:09.1881770Z [cluster1]         "brokerK8sRemoteNamespace": "submariner-k8s-broker",
2021-07-16T18:15:09.1882904Z [cluster1]         "cableDriver": "libreswan",
2021-07-16T18:15:09.1884665Z [cluster1]         "ceIPSecDebug": false,
2021-07-16T18:15:09.1885546Z [cluster1]         "ceIPSecIKEPort": 500,
2021-07-16T18:15:09.1886325Z [cluster1]         "ceIPSecNATTPort": 4500,
2021-07-16T18:15:09.1888742Z [cluster1]         "ceIPSecPSK": "Zhs+SumA5VO7u+aRnCe5Wjoau5QpxSYF1mxiCBvpnGHaZ/d40WAFSnV803hZx1RR",
2021-07-16T18:15:09.1890117Z [cluster1]         "clusterCIDR": "",
2021-07-16T18:15:09.1890754Z [cluster1]         "clusterID": "cluster1",
2021-07-16T18:15:09.1891488Z [cluster1]         "colorCodes": "blue",
2021-07-16T18:15:09.1892293Z [cluster1]         "connectionHealthCheck": {
2021-07-16T18:15:09.1893033Z [cluster1]             "enabled": true,
2021-07-16T18:15:09.1893735Z [cluster1]             "intervalSeconds": 1,
2021-07-16T18:15:09.1894469Z [cluster1]             "maxPacketLossCount": 5
2021-07-16T18:15:09.1895160Z [cluster1]         },
2021-07-16T18:15:09.1895668Z [cluster1]         "debug": false,
2021-07-16T18:15:09.1896796Z [cluster1]         "namespace": "submariner-operator",
2021-07-16T18:15:09.1897549Z [cluster1]         "natEnabled": false,
2021-07-16T18:15:09.1898262Z [cluster1]         "repository": "localhost:5000",
2021-07-16T18:15:09.1899298Z [cluster1]         "serviceCIDR": "",
2021-07-16T18:15:09.1900146Z [cluster1]         "serviceDiscoveryEnabled": true,
2021-07-16T18:15:09.1900960Z [cluster1]         "version": "local"
2021-07-16T18:15:09.1901471Z [cluster1]     },
2021-07-16T18:15:09.1902002Z [cluster1]     "status": {
2021-07-16T18:15:09.1902590Z [cluster1]         "clusterCIDR": "10.1.0.0/16",
2021-07-16T18:15:09.1903307Z [cluster1]         "clusterID": "cluster1",
2021-07-16T18:15:09.1903951Z [cluster1]         "colorCodes": "blue",
2021-07-16T18:15:09.1905052Z [cluster1]         "deploymentInfo": {},
2021-07-16T18:15:09.1905839Z [cluster1]         "gatewayDaemonSetStatus": {
2021-07-16T18:15:09.1906767Z [cluster1]             "lastResourceVersion": "2019",
2021-07-16T18:15:09.1920211Z [cluster1]             "mismatchedContainerImages": false,
2021-07-16T18:15:09.1921289Z [cluster1]             "nonReadyContainerStates": [],
2021-07-16T18:15:09.1921998Z [cluster1]             "status": {
2021-07-16T18:15:09.1922701Z [cluster1]                 "currentNumberScheduled": 1,
2021-07-16T18:15:09.1923767Z [cluster1]                 "desiredNumberScheduled": 1,
2021-07-16T18:15:09.1924538Z [cluster1]                 "numberAvailable": 1,
2021-07-16T18:15:09.1925254Z [cluster1]                 "numberMisscheduled": 0,
2021-07-16T18:15:09.1925926Z [cluster1]                 "numberReady": 1,
2021-07-16T18:15:09.1926591Z [cluster1]                 "observedGeneration": 1,
2021-07-16T18:15:09.1927411Z [cluster1]                 "updatedNumberScheduled": 1
2021-07-16T18:15:09.1928050Z [cluster1]             }
2021-07-16T18:15:09.1928489Z [cluster1]         },
2021-07-16T18:15:09.1928953Z [cluster1]         "gateways": [
2021-07-16T18:15:09.1929431Z [cluster1]             {
2021-07-16T18:15:09.1929933Z [cluster1]                 "connections": [
2021-07-16T18:15:09.1930446Z [cluster1]                     {
2021-07-16T18:15:09.1930949Z [cluster1]                         "endpoint": {
2021-07-16T18:15:09.1931560Z [cluster1]                             "backend": "libreswan",
2021-07-16T18:15:09.1932163Z [cluster1]                             "backend_config": {
2021-07-16T18:15:09.1933366Z [cluster1]                                 "natt-discovery-port": "4490",
2021-07-16T18:15:09.1934352Z [cluster1]                                 "preferred-server": "false"
2021-07-16T18:15:09.1934958Z [cluster1]                             },
2021-07-16T18:15:09.1936407Z [cluster1]                             "cable_name": "submariner-cable-cluster2-172-18-0-5",
2021-07-16T18:15:09.1937419Z [cluster1]                             "cluster_id": "cluster2",
2021-07-16T18:15:09.2211267Z [cluster1]                             "healthCheckIP": "10.2.128.0",
2021-07-16T18:15:09.2213047Z [cluster1]                             "hostname": "cluster2-worker",
2021-07-16T18:15:09.2213911Z [cluster1]                             "nat_enabled": false,
2021-07-16T18:15:09.2214622Z [cluster1]                             "private_ip": "172.18.0.5",
2021-07-16T18:15:09.2215335Z [cluster1]                             "public_ip": "40.71.64.254",
2021-07-16T18:15:09.2215997Z [cluster1]                             "subnets": [
2021-07-16T18:15:09.2216607Z [cluster1]                                 "100.2.0.0/16",
2021-07-16T18:15:09.2217200Z [cluster1]                                 "10.2.0.0/16"
2021-07-16T18:15:09.2217781Z [cluster1]                             ]
2021-07-16T18:15:09.2218325Z [cluster1]                         },
2021-07-16T18:15:09.2218953Z [cluster1]                         "latencyRTT": {
2021-07-16T18:15:09.2219909Z [cluster1]                             "average": "195.359µs",
2021-07-16T18:15:09.2221844Z [cluster1]                             "last": "211.506µs",
2021-07-16T18:15:09.2222385Z [cluster1]                             "max": "1.539042ms",
2021-07-16T18:15:09.2223065Z [cluster1]                             "min": "100.303µs",
2021-07-16T18:15:09.2223777Z [cluster1]                             "stdDev": "241.493µs"
2021-07-16T18:15:09.2224515Z [cluster1]                         },
2021-07-16T18:15:09.2225060Z [cluster1]                         "status": "connected",
2021-07-16T18:15:09.2225689Z [cluster1]                         "statusMessage": "",
2021-07-16T18:15:09.2226280Z [cluster1]                         "usingIP": "172.18.0.5"
2021-07-16T18:15:09.2226779Z [cluster1]                     }
2021-07-16T18:15:09.2227208Z [cluster1]                 ],
2021-07-16T18:15:09.2227734Z [cluster1]                 "haStatus": "active",
2021-07-16T18:15:09.2228318Z [cluster1]                 "localEndpoint": {
2021-07-16T18:15:09.2228937Z [cluster1]                     "backend": "libreswan",
2021-07-16T18:15:09.2229522Z [cluster1]                     "backend_config": {
2021-07-16T18:15:09.2230591Z [cluster1]                         "natt-discovery-port": "4490",
2021-07-16T18:15:09.2231523Z [cluster1]                         "preferred-server": "false"
2021-07-16T18:15:09.2232129Z [cluster1]                     },
2021-07-16T18:15:09.2233262Z [cluster1]                     "cable_name": "submariner-cable-cluster1-172-18-0-9",
2021-07-16T18:15:09.2234248Z [cluster1]                     "cluster_id": "cluster1",
2021-07-16T18:15:09.2235093Z [cluster1]                     "healthCheckIP": "10.1.0.1",
2021-07-16T18:15:09.2235971Z [cluster1]                     "hostname": "cluster1-worker",
2021-07-16T18:15:09.2236640Z [cluster1]                     "nat_enabled": false,
2021-07-16T18:15:09.2237212Z [cluster1]                     "private_ip": "172.18.0.9",
2021-07-16T18:15:09.2237775Z [cluster1]                     "public_ip": "40.71.64.254",
2021-07-16T18:15:09.2238319Z [cluster1]                     "subnets": [
2021-07-16T18:15:09.2238815Z [cluster1]                         "100.1.0.0/16",
2021-07-16T18:15:09.2239295Z [cluster1]                         "10.1.0.0/16"
2021-07-16T18:15:09.2239742Z [cluster1]                     ]
2021-07-16T18:15:09.2240184Z [cluster1]                 },
2021-07-16T18:15:09.2240710Z [cluster1]                 "statusFailure": "",
2021-07-16T18:15:09.2241536Z [cluster1]                 "version": "devel-7cad5c5"
2021-07-16T18:15:09.2242074Z [cluster1]             }
2021-07-16T18:15:09.2242502Z [cluster1]         ],
2021-07-16T18:15:09.2243156Z [cluster1]         "globalnetDaemonSetStatus": {
2021-07-16T18:15:09.2244751Z [cluster1]             "mismatchedContainerImages": false
2021-07-16T18:15:09.2245448Z [cluster1]         },
2021-07-16T18:15:09.2246182Z [cluster1]         "loadBalancerStatus": {},
2021-07-16T18:15:09.2246862Z [cluster1]         "natEnabled": false,
2021-07-16T18:15:09.2247712Z [cluster1]         "networkPlugin": "weave-net",
2021-07-16T18:15:09.2248644Z [cluster1]         "routeAgentDaemonSetStatus": {
2021-07-16T18:15:09.2249610Z [cluster1]             "lastResourceVersion": "2077",
2021-07-16T18:15:09.2250500Z [cluster1]             "mismatchedContainerImages": false,
2021-07-16T18:15:09.2251405Z [cluster1]             "nonReadyContainerStates": [],
2021-07-16T18:15:09.2252106Z [cluster1]             "status": {
2021-07-16T18:15:09.2252772Z [cluster1]                 "currentNumberScheduled": 3,
2021-07-16T18:15:09.2253636Z [cluster1]                 "desiredNumberScheduled": 3,
2021-07-16T18:15:09.2254404Z [cluster1]                 "numberAvailable": 3,
2021-07-16T18:15:09.2255095Z [cluster1]                 "numberMisscheduled": 0,
2021-07-16T18:15:09.2255764Z [cluster1]                 "numberReady": 3,
2021-07-16T18:15:09.2256420Z [cluster1]                 "observedGeneration": 1,
2021-07-16T18:15:09.2257230Z [cluster1]                 "updatedNumberScheduled": 3
2021-07-16T18:15:09.2257862Z [cluster1]             }
2021-07-16T18:15:09.2258296Z [cluster1]         },
2021-07-16T18:15:09.2258798Z [cluster1]         "serviceCIDR": "100.1.0.0/16"
2021-07-16T18:15:09.2259309Z [cluster1]     }
2021-07-16T18:15:09.2259713Z [cluster1] }
2021-07-16T18:15:09.2260910Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-operator
2021-07-16T18:15:09.2262766Z [submariner-operator]$ [cluster1] validate_equals .metadata.namespace submariner-operator
2021-07-16T18:15:09.2264483Z [submariner-operator]$ [cluster1] jq -r -M .metadata.namespace /tmp/submariner.cluster1.json
2021-07-16T18:15:09.2266083Z [submariner-operator]$ [cluster1] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-16T18:15:09.2267529Z [submariner-operator]$ [cluster1] validate_equals .apiVersion submariner.io/v1alpha1
2021-07-16T18:15:09.2269057Z [submariner-operator]$ [cluster1] jq -r -M .apiVersion /tmp/submariner.cluster1.json
2021-07-16T18:15:09.2270591Z [submariner-operator]$ [cluster1] validate_equals .kind Submariner
2021-07-16T18:15:09.2271833Z [submariner-operator]$ [cluster1] validate_equals .kind Submariner
2021-07-16T18:15:09.2902904Z [snd Submariner
2021-07-16T18:15:09.2904092Z [submariner-operator]$ [cluster2] jq -r -M .kind /tmp/submariner.cluster2.json
2021-07-16T18:15:09.2905537Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariner
2021-07-16T18:15:09.2907019Z [submariner-operator]$ [cluster2] validate_equals .metadata.name submariner
2021-07-16T18:15:09.2908533Z [submariner-operator]$ [cluster2] jq -r -M .metadata.name /tmp/submariner.cluster2.json
2021-07-16T18:15:09.2910134Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443
2021-07-16T18:15:09.2911687Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443
2021-07-16T18:15:09.2913410Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sApiServer /tmp/submariner.cluster2.json
2021-07-16T18:15:09.3138566Z [submariner-operator]$ [cluster2] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-16T18:15:09.3432413Z [submariner-operator]$ [cluster2] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-16T18:15:09.3434957Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sApiServerToken /tmp/submariner.cluster2.json
2021-07-16T18:15:09.4173395Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4reubmariner-operator]$ [cluster1] jq -r -M .kind /tmp/submariner.cluster1.json
2021-07-16T18:15:09.4653969Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner
2021-07-16T18:15:09.4655411Z [submariner-operator]$ [cluster1] validate_equals .metadata.name submariner
2021-07-16T18:15:09.4656976Z [submariner-operator]$ [cluster1] jq -r -M .metadata.name /tmp/submariner.cluster1.json
2021-07-16T18:15:09.4658571Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443
2021-07-16T18:15:09.4660148Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sApiServer 172.18.0.8:6443
2021-07-16T18:15:09.4661886Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sApiServer /tmp/submariner.cluster1.json
2021-07-16T18:15:09.4967905Z [submariner-operator]$ [cluster1] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-16T18:15:09.5186194Z [submariner-operator]$ [cluster1] validate_not_equals .spec.brokerK8sApiServerToken ***
2021-07-16T18:15:09.5188716Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sApiServerToken /tmp/submariner.cluster1.json
2021-07-16T18:15:09.8554618Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWkdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
2021-07-16T18:15:09.9821513Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
2021-07-16T18:15:10.0477888Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sCA /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0479979Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-16T18:15:10.0482084Z [submariner-operator]$ [cluster2] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-16T18:15:10.0484441Z [submariner-operator]$ [cluster2] jq -r -M .spec.brokerK8sRemoteNamespace /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0486196Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecDebug false
2021-07-16T18:15:10.0487615Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecDebug false
2021-07-16T18:15:10.0489487Z [submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecDebug /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0491053Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecIKEPort 500
2021-07-16T18:15:10.0492460Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecIKEPort 500
2021-07-16T18:15:10.0494073Z [submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecIKEPort /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0498234Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-16T18:15:10.0499694Z [submariner-operator]$ [cluster2] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-16T18:15:10.0501339Z [submariner-operator]$ [cluster2] jq -r -M .spec.ceIPSecNATTPort /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0502929Z [submariner-operator]$ [cluster2] validate_equals .spec.repository localhost:5000
2021-07-16T18:15:10.0504341Z [submariner-operator]$ [cluster2] validate_equals .spec.repository localhost:5000
2021-07-16T18:15:10.0505967Z [submariner-operator]$ [cluster2] jq -r -M .spec.repository /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0507423Z [submariner-operator]$ [cluster2] validate_equals .spec.version local
2021-07-16T18:15:10.0508703Z [submariner-operator]$ [cluster2] validate_equals .spec.version local
2021-07-16T18:15:10.0510224Z [submariner-operator]$ [cluster2] jq -r -M .spec.version /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0511631Z [submariner-operator]$ [cluster2] validate_equals .spec.broker k8s
2021-07-16T18:15:10.0512852Z [submariner-operator]$ [cluster2] validate_equals .spec.broker k8s
2021-07-16T18:15:10.0514273Z [submariner-operator]$ [cluster2] jq -r -M .spec.broker /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0515849Z [submariner-operator]$ [cluster2] jq -r .spec.clusterID /tmp/submariner.cluster2.json
2021-07-16T18:15:10.0516919Z [cluster2] Generated cluster id: cluster2
2021-07-16T18:15:10.0518021Z [submariner-operator]$ [cluster2] validate_equals .spec.colorCodes blue
2021-07-16T18:15:10.1627860Z [submariner-operatorE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
2021-07-16T18:15:10.2378686Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sCA LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
2021-07-16T18:15:10.2911922Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sCA /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2914371Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-16T18:15:10.2916667Z [submariner-operator]$ [cluster1] validate_equals .spec.brokerK8sRemoteNamespace submariner-k8s-broker
2021-07-16T18:15:10.2918801Z [submariner-operator]$ [cluster1] jq -r -M .spec.brokerK8sRemoteNamespace /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2920556Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecDebug false
2021-07-16T18:15:10.2921962Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecDebug false
2021-07-16T18:15:10.2923702Z [submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecDebug /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2925282Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecIKEPort 500
2021-07-16T18:15:10.2926711Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecIKEPort 500
2021-07-16T18:15:10.2928324Z [submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecIKEPort /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2929911Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-16T18:15:10.2931360Z [submariner-operator]$ [cluster1] validate_equals .spec.ceIPSecNATTPort 4500
2021-07-16T18:15:10.2932991Z [submariner-operator]$ [cluster1] jq -r -M .spec.ceIPSecNATTPort /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2934587Z [submariner-operator]$ [cluster1] validate_equals .spec.repository localhost:5000
2021-07-16T18:15:10.2936402Z [submariner-operator]$ [cluster1] validate_equals .spec.repository localhost:5000
2021-07-16T18:15:10.2938148Z [submariner-operator]$ [cluster1] jq -r -M .spec.repository /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2939612Z [submariner-operator]$ [cluster1] validate_equals .spec.version local
2021-07-16T18:15:10.2941683Z [submariner-operator]$ [cluster1] validate_equals .spec.version local
2021-07-16T18:15:10.2943184Z [submariner-operator]$ [cluster1] jq -r -M .spec.version /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2945030Z [submariner-operator]$ [cluster1] validate_equals .spec.broker k8s
2021-07-16T18:15:10.2946464Z [submariner-operator]$ [cluster1] validate_equals .spec.broker k8s
2021-07-16T18:15:10.2948204Z [submariner-operator]$ [cluster1] jq -r -M .spec.broker /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2949819Z [submariner-operator]$ [cluster1] jq -r .spec.clusterID /tmp/submariner.cluster1.json
2021-07-16T18:15:10.2950869Z [cluster1] Generated cluster id: cluster1
2021-07-16T18:15:10.2951998Z [submariner-operator]$ [cluster1] validate_equals .spec.colorCodes blue
2021-07-16T18:15:11.1234979Z [submariner-operator]$ [cluster1] validate_eq]$ [cluster2] validate_equals .spec.colorCodes blue
2021-07-16T18:15:11.1236721Z [submariner-operator]$ [cluster2] jq -r -M .spec.colorCodes /tmp/submariner.cluster2.json
2021-07-16T18:15:11.1238179Z [submariner-operator]$ [cluster2] validate_equals .spec.debug false
2021-07-16T18:15:11.1239439Z [submariner-operator]$ [cluster2] validate_equals .spec.debug false
2021-07-16T18:15:11.1240841Z [submariner-operator]$ [cluster2] jq -r -M .spec.debug /tmp/submariner.cluster2.json
2021-07-16T18:15:11.1242435Z [submariner-operator]$ [cluster2] validate_equals .spec.namespace submariner-operator
2021-07-16T18:15:11.1244200Z [submariner-operator]$ [cluster2] validate_equals .spec.namespace submariner-operator
2021-07-16T18:15:11.1245834Z [submariner-operator]$ [cluster2] jq -r -M .spec.namespace /tmp/submariner.cluster2.json
2021-07-16T18:15:11.1247334Z [submariner-operator]$ [cluster2] validate_equals .spec.natEnabled false
2021-07-16T18:15:11.1248660Z [submariner-operator]$ [cluster2] validate_equals .spec.natEnabled false
2021-07-16T18:15:11.1250425Z [submariner-operator]$ [cluster2] jq -r -M .spec.natEnabled /tmp/submariner.cluster2.json
2021-07-16T18:15:11.1251839Z [submariner-operator]$ [cluster2] verify_subm_cr_status_with_retries
2021-07-16T18:15:11.1253032Z [submariner-operator]$ [cluster2] verify_subm_cr_status_with_retries
2021-07-16T18:15:11.1254244Z [submariner-operator]$ [cluster2] with_retries 5 verify_subm_cr_status_
2021-07-16T18:15:11.1255442Z [submariner-operator]$ [cluster2] with_retries 5 verify_subm_cr_status_
2021-07-16T18:15:11.1256545Z [submariner-operator]$ [cluster2] iteration=1
2021-07-16T18:15:11.1257596Z [submariner-operator]$ [cluster2] verify_subm_cr_status_
2021-07-16T18:15:11.1258624Z [submariner-operator]$ [cluster2] wait 8525
2021-07-16T18:15:11.1259637Z [submariner-operator]$ [cluster2] verify_subm_cr_status_
2021-07-16T18:15:11.1260725Z [submariner-operator]$ [cluster2] verify_subm_cr_status
2021-07-16T18:15:11.1261950Z [submariner-operator]$ [cluster2] verify_subm_cr_status
2021-07-16T18:15:11.1263242Z [submariner-operator]$ [cluster2] json_file=/tmp/submariner.cluster2.json
2021-07-16T18:15:11.1264227Z [cluster2] 
2021-07-16T18:15:11.1264773Z [cluster2] 
2021-07-16T18:15:11.1266148Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:11.1267833Z [submariner-operator]$ [cluster2] validate_equals .status.serviceCIDR 100.2.0.0/16
2021-07-16T18:15:11.1269254Z [submariner-operator]$ [cluster2] validate_equals .status.serviceCIDR 100.2.0.0/16
2021-07-16T18:15:11.1270840Z [submariner-operator]$ [cluster2] jq -r -M .status.serviceCIDR /tmp/submariner.cluster2.json
2021-07-16T18:15:11.1272389Z [submariner-operator]$ [cluster2] validate_equals .status.clusterCIDR 10.2.0.0/16
2021-07-16T18:15:11.1273826Z [submariner-operator]$ [cluster2] validate_equals .status.clusterCIDR 10.2.0.0/16
2021-07-16T18:15:11.1275379Z [submariner-operator]$ [cluster2] jq -r -M .status.clusterCIDR /tmp/submariner.cluster2.json
2021-07-16T18:15:11.1276711Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:15:11.1277678Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:15:11.1278771Z [submariner-operator]$ [cluster2] verify_subm_gateway_deployment
2021-07-16T18:15:11.1280114Z [submariner-operator]$ [cluster2] verify_subm_gateway_deployment
2021-07-16T18:15:11.1281708Z [submariner-operator]$ [cluster2] verify_daemonset submariner-gateway submariner-gateway
2021-07-16T18:15:11.1283375Z [submariner-operator]$ [cluster2] verify_daemonset submariner-gateway submariner-gateway
2021-07-16T18:15:11.1285028Z [submariner-operator]$ [cluster2] daemonset_app=submariner-gateway
2021-07-16T18:15:11.1286391Z [submariner-operator]$ [cluster2] daemonset_name=submariner-gateway
2021-07-16T18:15:11.1287826Z [submariner-operator]$ [cluster2] with_retries 60 daemonset_created
2021-07-16T18:15:11.1289039Z [submariner-operator]$ [cluster2] with_retries 60 daemonset_created
2021-07-16T18:15:11.1290153Z [submariner-operator]$ [cluster2] iteration=1
2021-07-16T18:15:11.1291202Z [submariner-operator]$ [cluster2] daemonset_created
2021-07-16T18:15:11.1292308Z [submariner-operator]$ [cluster2] daemonset_created
2021-07-16T18:15:11.1293105Z [cluster2] 
2021-07-16T18:15:11.1293894Z [submariner-operator]$ [cluster2] wait 8593
2021-07-16T18:15:11.1294841Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:15:11.1295977Z [submariner-operator]$ [cluster2] with_retries 120 daemonset_deployed
2021-07-16T18:15:11.1297232Z [submariner-operator]$ [cluster2] with_retries 120 daemonset_deployed
2021-07-16T18:15:11.1298340Z [submariner-operator]$ [cluster2] iteration=1
2021-07-16T18:15:11.1299450Z [submariner-operator]$ [cluster2] wait 8638
2021-07-16T18:15:11.1300476Z [submariner-operator]$ [cluster2] daemonset_deployed
2021-07-16T18:15:11.1301590Z [submariner-operator]$ [cluster2] daemonset_deployed
2021-07-16T18:15:11.1303296Z [submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-16T18:15:11.3453070Z [submariner-operator]$ [clustuals .spec.colorCodes blue
2021-07-16T18:15:11.3454606Z [submariner-operator]$ [cluster1] jq -r -M .spec.colorCodes /tmp/submariner.cluster1.json
2021-07-16T18:15:11.3456094Z [submariner-operator]$ [cluster1] validate_equals .spec.debug false
2021-07-16T18:15:11.3457448Z [submariner-operator]$ [cluster1] validate_equals .spec.debug false
2021-07-16T18:15:11.3458850Z [submariner-operator]$ [cluster1] jq -r -M .spec.debug /tmp/submariner.cluster1.json
2021-07-16T18:15:11.3460452Z [submariner-operator]$ [cluster1] validate_equals .spec.namespace submariner-operator
2021-07-16T18:15:11.3462055Z [submariner-operator]$ [cluster1] validate_equals .spec.namespace submariner-operator
2021-07-16T18:15:11.3463669Z [submariner-operator]$ [cluster1] jq -r -M .spec.namespace /tmp/submariner.cluster1.json
2021-07-16T18:15:11.3465148Z [submariner-operator]$ [cluster1] validate_equals .spec.natEnabled false
2021-07-16T18:15:11.3466487Z [submariner-operator]$ [cluster1] validate_equals .spec.natEnabled false
2021-07-16T18:15:11.3468015Z [submariner-operator]$ [cluster1] jq -r -M .spec.natEnabled /tmp/submariner.cluster1.json
2021-07-16T18:15:11.3469425Z [submariner-operator]$ [cluster1] verify_subm_cr_status_with_retries
2021-07-16T18:15:11.3470625Z [submariner-operator]$ [cluster1] verify_subm_cr_status_with_retries
2021-07-16T18:15:11.3471834Z [submariner-operator]$ [cluster1] with_retries 5 verify_subm_cr_status_
2021-07-16T18:15:11.3473037Z [submariner-operator]$ [cluster1] with_retries 5 verify_subm_cr_status_
2021-07-16T18:15:11.3474142Z [submariner-operator]$ [cluster1] iteration=1
2021-07-16T18:15:11.3475186Z [submariner-operator]$ [cluster1] verify_subm_cr_status_
2021-07-16T18:15:11.3476273Z [submariner-operator]$ [cluster1] verify_subm_cr_status_
2021-07-16T18:15:11.3477347Z [submariner-operator]$ [cluster1] verify_subm_cr_status
2021-07-16T18:15:11.3478740Z [submariner-operator]$ [cluster1] verify_subm_cr_status
2021-07-16T18:15:11.3480071Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner.cluster1.json
2021-07-16T18:15:11.3481254Z [submariner-operator]$ [cluster1] wait 8554
2021-07-16T18:15:11.3482006Z [cluster1] 
2021-07-16T18:15:11.3482558Z [cluster1] 
2021-07-16T18:15:11.3484158Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get submariner submariner --namespace=submariner-operator -o json
2021-07-16T18:15:11.3485867Z [submariner-operator]$ [cluster1] validate_equals .status.serviceCIDR 100.1.0.0/16
2021-07-16T18:15:11.3487383Z [submariner-operator]$ [cluster1] validate_equals .status.serviceCIDR 100.1.0.0/16
2021-07-16T18:15:11.3488979Z [submariner-operator]$ [cluster1] jq -r -M .status.serviceCIDR /tmp/submariner.cluster1.json
2021-07-16T18:15:11.3490522Z [submariner-operator]$ [cluster1] validate_equals .status.clusterCIDR 10.1.0.0/16
2021-07-16T18:15:11.3491920Z [submariner-operator]$ [cluster1] validate_equals .status.clusterCIDR 10.1.0.0/16
2021-07-16T18:15:11.3493477Z [submariner-operator]$ [cluster1] jq -r -M .status.clusterCIDR /tmp/submariner.cluster1.json
2021-07-16T18:15:11.3494800Z [submariner-operator]$ [cluster1] return 0
2021-07-16T18:15:11.3495764Z [submariner-operator]$ [cluster1] return 0
2021-07-16T18:15:11.3496879Z [submariner-operator]$ [cluster1] verify_subm_gateway_deployment
2021-07-16T18:15:11.3498233Z [submariner-operator]$ [cluster1] verify_subm_gateway_deployment
2021-07-16T18:15:11.3499679Z [submariner-operator]$ [cluster1] verify_daemonset submariner-gateway submariner-gateway
2021-07-16T18:15:11.3501353Z [submariner-operator]$ [cluster1] verify_daemonset submariner-gateway submariner-gateway
2021-07-16T18:15:11.3502832Z [submariner-operator]$ [cluster1] daemonset_app=submariner-gateway
2021-07-16T18:15:11.3504202Z [submariner-operator]$ [cluster1] daemonset_name=submariner-gateway
2021-07-16T18:15:11.3505478Z [submariner-operator]$ [cluster1] with_retries 60 daemonset_created
2021-07-16T18:15:11.3506709Z [submariner-operator]$ [cluster1] with_retries 60 daemonset_created
2021-07-16T18:15:11.3507825Z [submariner-operator]$ [cluster1] iteration=1
2021-07-16T18:15:11.3508807Z [submariner-operator]$ [cluster1] wait 8621
2021-07-16T18:15:11.3509966Z [submariner-operator]$ [cluster1] daemonset_created
2021-07-16T18:15:11.3511064Z [submariner-operator]$ [cluster1] daemonset_created
2021-07-16T18:15:11.3511874Z [cluster1] 
2021-07-16T18:15:11.3512645Z [submariner-operator]$ [cluster1] return 0
2021-07-16T18:15:11.3513784Z [submariner-operator]$ [cluster1] with_retries 120 daemonset_deployed
2021-07-16T18:15:11.3515024Z [submariner-operator]$ [cluster1] with_retries 120 daemonset_deployed
2021-07-16T18:15:11.3516156Z [submariner-operator]$ [cluster1] iteration=1
2021-07-16T18:15:11.3517235Z [submariner-operator]$ [cluster1] daemonset_deployed
2021-07-16T18:15:11.3518339Z [submariner-operator]$ [cluster1] daemonset_deployed
2021-07-16T18:15:11.3520070Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-16T18:15:13.3100675Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desireder2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-16T18:15:13.3103828Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-16T18:15:13.3106122Z [submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-16T18:15:13.3108198Z [submariner-operator]$ [cluster2] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-16T18:15:13.3110252Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-16T18:15:13.3111845Z [submariner-operator]$ [cluster2] return 0
2021-07-16T18:15:13.3112910Z [submariner-operator]$ [cluster2] verify_subm_gateway_pod
2021-07-16T18:15:13.3114026Z [submariner-operator]$ [cluster2] verify_subm_gateway_pod
2021-07-16T18:15:13.3115713Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-16T18:15:13.3118017Z [submariner-operator]$ [cluster2] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-16T18:15:13.3120335Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-16T18:15:13.3122155Z [cluster2] pod/submariner-gateway-gfcst condition met
2021-07-16T18:15:13.3124029Z [submariner-operator]$ [cluster2] subm_gateway_pod_name=$(kubectl get pods --namespace=$subm_ns -l app=$gateway_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-16T18:15:13.3126249Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:13.3128693Z [submariner-operator]$ [cluster2] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:13.3131130Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:13.3133453Z [submariner-operator]$ [cluster2] json_file=/tmp/submariner-gateway-gfcst.cluster2.json
2021-07-16T18:15:13.3135588Z [submariner-operator]$ [cluster2] kubectl get pod submariner-gateway-gfcst --namespace=submariner-operator -o json
2021-07-16T18:15:13.3137548Z [submariner-operator]$ [cluster2] tee /tmp/submariner-gateway-gfcst.cluster2.json
2021-07-16T18:15:13.3139550Z [submariner-operator]$ [cluster2] kubectl get pod submariner-gateway-gfcst --namespace=submariner-operator -o json
2021-07-16T18:15:13.3141757Z [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pod submariner-gateway-gfcst --namespace=submariner-operator -o json
2021-07-16T18:15:13.3142958Z [cluster2] {
2021-07-16T18:15:13.3143460Z [cluster2]     "apiVersion": "v1",
2021-07-16T18:15:13.3143969Z [cluster2]     "kind": "Pod",
2021-07-16T18:15:13.3144467Z [cluster2]     "metadata": {
2021-07-16T18:15:13.3145284Z [cluster2]         "creationTimestamp": "2021-07-16T18:14:12Z",
2021-07-16T18:15:13.3146299Z [cluster2]         "generateName": "submariner-gateway-",
2021-07-16T18:15:13.3146985Z [cluster2]         "labels": {
2021-07-16T18:15:13.3147774Z [cluster2]             "app": "submariner-gateway",
2021-07-16T18:15:13.3148788Z [cluster2]             "controller-revision-hash": "5fdb545c4c",
2021-07-16T18:15:13.3149964Z [cluster2]             "gateway.submariner.io/node": "cluster2-worker",
2021-07-16T18:15:13.3150921Z [cluster2]             "gateway.submariner.io/status": "active",
2021-07-16T18:15:13.3151943Z [cluster2]             "pod-template-generation": "1"
2021-07-16T18:15:13.3152594Z [cluster2]         },
2021-07-16T18:15:13.3153120Z [cluster2]         "managedFields": [
2021-07-16T18:15:13.3153643Z [cluster2]             {
2021-07-16T18:15:13.3154145Z [cluster2]                 "apiVersion": "v1",
2021-07-16T18:15:13.3154770Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:13.3155351Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:13.3155895Z [cluster2]                     "f:metadata": {
2021-07-16T18:15:13.3156610Z [cluster2]                         "f:generateName": {},
2021-07-16T18:15:13.3157189Z [cluster2]                         "f:labels": {
2021-07-16T18:15:13.3157690Z [cluster2]                             ".": {},
2021-07-16T18:15:13.3158170Z [cluster2]                             "f:app": {},
2021-07-16T18:15:13.3159110Z [cluster2]                             "f:controller-revision-hash": {},
2021-07-16T18:15:13.3160193Z [cluster2]                             "f:pod-template-generation": {}
2021-07-16T18:15:13.3160877Z [cluster2]                         },
2021-07-16T18:15:13.3161559Z [cluster2]                         "f:ownerReferences": {
2021-07-16T18:15:13.3162150Z [cluster2]                             ".": {},
2021-07-16T18:15:13.3163123Z [cluster2]                             "k:{\"uid\":\"7a85b5a5-ff5b-4fcd-b70d-0c44966555a6\"}": {
2021-07-16T18:15:13.3212426Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3213158Z [cluster2]                                 "f:apiVersion": {},
2021-07-16T18:15:13.3213853Z [cluster2]                                 "f:blockOwnerDeletion": {},
2021-07-16T18:15:13.3214561Z [cluster2]                                 "f:controller": {},
2021-07-16T18:15:13.3215114Z [cluster2]                                 "f:kind": {},
2021-07-16T18:15:13.3215642Z [cluster2]                                 "f:name": {},
2021-07-16T18:15:13.3216161Z [cluster2]                                 "f:uid": {}
2021-07-16T18:15:13.3216784Z [cluster2]                             }
2021-07-16T18:15:13.3217248Z [cluster2]                         }
2021-07-16T18:15:13.3217690Z [cluster2]                     },
2021-07-16T18:15:13.3218172Z [cluster2]                     "f:spec": {
2021-07-16T18:15:13.3218692Z [cluster2]                         "f:affinity": {
2021-07-16T18:15:13.3219211Z [cluster2]                             ".": {},
2021-07-16T18:15:13.3219753Z [cluster2]                             "f:nodeAffinity": {
2021-07-16T18:15:13.3220465Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3221726Z [cluster2]                                 "f:requiredDuringSchedulingIgnoredDuringExecution": {
2021-07-16T18:15:13.3222930Z [cluster2]                                     ".": {},
2021-07-16T18:15:13.3223569Z [cluster2]                                     "f:nodeSelectorTerms": {}
2021-07-16T18:15:13.3224185Z [cluster2]                                 }
2021-07-16T18:15:13.3224659Z [cluster2]                             },
2021-07-16T18:15:13.3225245Z [cluster2]                             "f:podAntiAffinity": {
2021-07-16T18:15:13.3227904Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3229486Z [cluster2]                                 "f:requiredDuringSchedulingIgnoredDuringExecution": {}
2021-07-16T18:15:13.3230797Z [cluster2]                             }
2021-07-16T18:15:13.3231249Z [cluster2]                         },
2021-07-16T18:15:13.3231778Z [cluster2]                         "f:containers": {
2021-07-16T18:15:13.3233101Z [cluster2]                             "k:{\"name\":\"submariner-gateway\"}": {
2021-07-16T18:15:13.3233750Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3234279Z [cluster2]                                 "f:command": {},
2021-07-16T18:15:13.3234806Z [cluster2]                                 "f:env": {
2021-07-16T18:15:13.3235316Z [cluster2]                                     ".": {},
2021-07-16T18:15:13.3235927Z [cluster2]                                     "k:{\"name\":\"BROKER_K8S_APISERVER\"}": {
2021-07-16T18:15:13.3236557Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3237068Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3237610Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3238122Z [cluster2]                                     },
2021-07-16T18:15:13.3238911Z [cluster2]                                     "k:{\"name\":\"BROKER_K8S_APISERVERTOKEN\"}": {
2021-07-16T18:15:13.3239579Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3240093Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3240636Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3241132Z [cluster2]                                     },
2021-07-16T18:15:13.3241740Z [cluster2]                                     "k:{\"name\":\"BROKER_K8S_CA\"}": {
2021-07-16T18:15:13.3242319Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3242826Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3243366Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3247466Z [cluster2]                                     },
2021-07-16T18:15:13.3248231Z [cluster2]                                     "k:{\"name\":\"BROKER_K8S_REMOTENAMESPACE\"}": {
2021-07-16T18:15:13.3248909Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3249439Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3249973Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3250484Z [cluster2]                                     },
2021-07-16T18:15:13.3251063Z [cluster2]                                     "k:{\"name\":\"CE_IPSEC_DEBUG\"}": {
2021-07-16T18:15:13.3251642Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3252313Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3252837Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3253343Z [cluster2]                                     },
2021-07-16T18:15:13.3253953Z [cluster2]                                     "k:{\"name\":\"CE_IPSEC_FORCEENCAPS\"}": {
2021-07-16T18:15:13.3254586Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3255115Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3255645Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3256156Z [cluster2]                                     },
2021-07-16T18:15:13.3256730Z [cluster2]                                     "k:{\"name\":\"CE_IPSEC_IKEPORT\"}": {
2021-07-16T18:15:13.3257330Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3257837Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3258379Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3258873Z [cluster2]                                     },
2021-07-16T18:15:13.3259464Z [cluster2]                                     "k:{\"name\":\"CE_IPSEC_NATTPORT\"}": {
2021-07-16T18:15:13.3260697Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3261212Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3261760Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3262261Z [cluster2]                                     },
2021-07-16T18:15:13.3262919Z [cluster2]                                     "k:{\"name\":\"CE_IPSEC_PREFERREDSERVER\"}": {
2021-07-16T18:15:13.3263577Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3264098Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3264931Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3265439Z [cluster2]                                     },
2021-07-16T18:15:13.3266008Z [cluster2]                                     "k:{\"name\":\"CE_IPSEC_PSK\"}": {
2021-07-16T18:15:13.3266572Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3267831Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3268532Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3269499Z [cluster2]                                     },
2021-07-16T18:15:13.3270074Z [cluster2]                                     "k:{\"name\":\"NODE_NAME\"}": {
2021-07-16T18:15:13.3270636Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3271160Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3271718Z [cluster2]                                         "f:valueFrom": {
2021-07-16T18:15:13.3281170Z [cluster2]                                             ".": {},
2021-07-16T18:15:13.3282015Z [cluster2]                                             "f:fieldRef": {
2021-07-16T18:15:13.3282581Z [cluster2]                                                 ".": {},
2021-07-16T18:15:13.3283151Z [cluster2]                                                 "f:apiVersion": {},
2021-07-16T18:15:13.3292809Z [cluster2]                                                 "f:fieldPath": {}
2021-07-16T18:15:13.3293474Z [cluster2]                                             }
2021-07-16T18:15:13.3293957Z [cluster2]                                         }
2021-07-16T18:15:13.3294444Z [cluster2]                                     },
2021-07-16T18:15:13.3294992Z [cluster2]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-16T18:15:13.3295553Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3296063Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3296819Z [cluster2]                                         "f:valueFrom": {
2021-07-16T18:15:13.3297361Z [cluster2]                                             ".": {},
2021-07-16T18:15:13.3297917Z [cluster2]                                             "f:fieldRef": {
2021-07-16T18:15:13.3298473Z [cluster2]                                                 ".": {},
2021-07-16T18:15:13.3299037Z [cluster2]                                                 "f:apiVersion": {},
2021-07-16T18:15:13.3299675Z [cluster2]                                                 "f:fieldPath": {}
2021-07-16T18:15:13.3300221Z [cluster2]                                             }
2021-07-16T18:15:13.3300717Z [cluster2]                                         }
2021-07-16T18:15:13.3301187Z [cluster2]                                     },
2021-07-16T18:15:13.3301792Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_BROKER\"}": {
2021-07-16T18:15:13.3302404Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3302928Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3303469Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3303962Z [cluster2]                                     },
2021-07-16T18:15:13.3304607Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_CABLEDRIVER\"}": {
2021-07-16T18:15:13.3305247Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3305769Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3306298Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3306803Z [cluster2]                                     },
2021-07-16T18:15:13.3307443Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_CLUSTERCIDR\"}": {
2021-07-16T18:15:13.3308081Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3308603Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3309135Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3309641Z [cluster2]                                     },
2021-07-16T18:15:13.3310251Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_CLUSTERID\"}": {
2021-07-16T18:15:13.3310885Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3311390Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3312018Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3312534Z [cluster2]                                     },
2021-07-16T18:15:13.3313155Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_COLORCODES\"}": {
2021-07-16T18:15:13.3313798Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3314307Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3314854Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3315357Z [cluster2]                                     },
2021-07-16T18:15:13.3315961Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_DEBUG\"}": {
2021-07-16T18:15:13.3316575Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3317083Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3317627Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3318127Z [cluster2]                                     },
2021-07-16T18:15:13.3318762Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_GLOBALCIDR\"}": {
2021-07-16T18:15:13.3319388Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3319912Z [cluster2]                                         "f:name": {}
2021-07-16T18:15:13.3320406Z [cluster2]                                     },
2021-07-16T18:15:13.3321124Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_HEALTHCHECKENABLED\"}": {
2021-07-16T18:15:13.3321916Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3322418Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3322966Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3323460Z [cluster2]                                     },
2021-07-16T18:15:13.3324694Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_HEALTHCHECKINTERVAL\"}": {
2021-07-16T18:15:13.3325424Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3325947Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3326488Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3326986Z [cluster2]                                     },
2021-07-16T18:15:13.3327873Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_HEALTHCHECKMAXPACKETLOSSCOUNT\"}": {
2021-07-16T18:15:13.3328755Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3329279Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3329810Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3330322Z [cluster2]                                     },
2021-07-16T18:15:13.3330932Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_NAMESPACE\"}": {
2021-07-16T18:15:13.3331577Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3332102Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3332628Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3333135Z [cluster2]                                     },
2021-07-16T18:15:13.3333753Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_NATENABLED\"}": {
2021-07-16T18:15:13.3334395Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3334900Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3335444Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3335961Z [cluster2]                                     },
2021-07-16T18:15:13.3336591Z [cluster2]                                     "k:{\"name\":\"SUBMARINER_SERVICECIDR\"}": {
2021-07-16T18:15:13.3337319Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3337835Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3338376Z [cluster2]                                         "f:value": {}
2021-07-16T18:15:13.3338874Z [cluster2]                                     }
2021-07-16T18:15:13.3339350Z [cluster2]                                 },
2021-07-16T18:15:13.3339846Z [cluster2]                                 "f:image": {},
2021-07-16T18:15:13.3340624Z [cluster2]                                 "f:imagePullPolicy": {},
2021-07-16T18:15:13.3341285Z [cluster2]                                 "f:name": {},
2021-07-16T18:15:13.3341805Z [cluster2]                                 "f:ports": {
2021-07-16T18:15:13.3342323Z [cluster2]                                     ".": {},
2021-07-16T18:15:13.3342982Z [cluster2]                                     "k:{\"containerPort\":4490,\"protocol\":\"UDP\"}": {
2021-07-16T18:15:13.3343649Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3344236Z [cluster2]                                         "f:containerPort": {},
2021-07-16T18:15:13.3344876Z [cluster2]                                         "f:hostPort": {},
2021-07-16T18:15:13.3345434Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3346007Z [cluster2]                                         "f:protocol": {}
2021-07-16T18:15:13.3347736Z [cluster2]                                     },
2021-07-16T18:15:13.3348395Z [cluster2]                                     "k:{\"containerPort\":4500,\"protocol\":\"UDP\"}": {
2021-07-16T18:15:13.3349173Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3349755Z [cluster2]                                         "f:containerPort": {},
2021-07-16T18:15:13.3350396Z [cluster2]                                         "f:hostPort": {},
2021-07-16T18:15:13.3351986Z [cluster2]                                         "f:name": {},
2021-07-16T18:15:13.3352584Z [cluster2]                                         "f:protocol": {}
2021-07-16T18:15:13.3353122Z [cluster2]                                     }
2021-07-16T18:15:13.3353582Z [cluster2]                                 },
2021-07-16T18:15:13.3354120Z [cluster2]                                 "f:resources": {},
2021-07-16T18:15:13.3354759Z [cluster2]                                 "f:securityContext": {
2021-07-16T18:15:13.3355361Z [cluster2]                                     ".": {},
2021-07-16T18:15:13.3356078Z [cluster2]                                     "f:allowPrivilegeEscalation": {},
2021-07-16T18:15:13.3356892Z [cluster2]                                     "f:capabilities": {
2021-07-16T18:15:13.3357453Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3357964Z [cluster2]                                         "f:add": {},
2021-07-16T18:15:13.3358498Z [cluster2]                                         "f:drop": {}
2021-07-16T18:15:13.3358990Z [cluster2]                                     },
2021-07-16T18:15:13.3359545Z [cluster2]                                     "f:privileged": {},
2021-07-16T18:15:13.3360295Z [cluster2]                                     "f:readOnlyRootFilesystem": {},
2021-07-16T18:15:13.3361070Z [cluster2]                                     "f:runAsNonRoot": {}
2021-07-16T18:15:13.3361619Z [cluster2]                                 },
2021-07-16T18:15:13.3362306Z [cluster2]                                 "f:terminationMessagePath": {},
2021-07-16T18:15:13.3363243Z [cluster2]                                 "f:terminationMessagePolicy": {},
2021-07-16T18:15:13.3364165Z [cluster2]                                 "f:volumeMounts": {
2021-07-16T18:15:13.3364738Z [cluster2]                                     ".": {},
2021-07-16T18:15:13.3365337Z [cluster2]                                     "k:{\"mountPath\":\"/etc/ipsec.d\"}": {
2021-07-16T18:15:13.3365946Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3366599Z [cluster2]                                         "f:mountPath": {},
2021-07-16T18:15:13.3367181Z [cluster2]                                         "f:name": {}
2021-07-16T18:15:13.3367672Z [cluster2]                                     },
2021-07-16T18:15:13.3368289Z [cluster2]                                     "k:{\"mountPath\":\"/var/lib/ipsec/nss\"}": {
2021-07-16T18:15:13.3368907Z [cluster2]                                         ".": {},
2021-07-16T18:15:13.3369442Z [cluster2]                                         "f:mountPath": {},
2021-07-16T18:15:13.3370026Z [cluster2]                                         "f:name": {}
2021-07-16T18:15:13.3370524Z [cluster2]                                     }
2021-07-16T18:15:13.3371001Z [cluster2]                                 }
2021-07-16T18:15:13.3371454Z [cluster2]                             }
2021-07-16T18:15:13.3371917Z [cluster2]                         },
2021-07-16T18:15:13.3372638Z [cluster2]                         "f:dnsPolicy": {},
2021-07-16T18:15:13.3373329Z [cluster2]                         "f:enableServiceLinks": {},
2021-07-16T18:15:13.3374032Z [cluster2]                         "f:hostNetwork": {},
2021-07-16T18:15:13.3374649Z [cluster2]                         "f:nodeSelector": {
2021-07-16T18:15:13.3375212Z [cluster2]                             ".": {},
2021-07-16T18:15:13.3375804Z [cluster2]                             "f:submariner.io/gateway": {}
2021-07-16T18:15:13.3386250Z [cluster2]                         },
2021-07-16T18:15:13.3386834Z [cluster2]                         "f:restartPolicy": {},
2021-07-16T18:15:13.3387675Z [cluster2]                         "f:schedulerName": {},
2021-07-16T18:15:13.3388344Z [cluster2]                         "f:securityContext": {},
2021-07-16T18:15:13.3389035Z [cluster2]                         "f:serviceAccount": {},
2021-07-16T18:15:13.3389760Z [cluster2]                         "f:serviceAccountName": {},
2021-07-16T18:15:13.3390690Z [cluster2]                         "f:terminationGracePeriodSeconds": {},
2021-07-16T18:15:13.3391566Z [cluster2]                         "f:tolerations": {},
2021-07-16T18:15:13.3392130Z [cluster2]                         "f:volumes": {
2021-07-16T18:15:13.3392637Z [cluster2]                             ".": {},
2021-07-16T18:15:13.3393168Z [cluster2]                             "k:{\"name\":\"ipsecd\"}": {
2021-07-16T18:15:13.3393706Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3394232Z [cluster2]                                 "f:emptyDir": {},
2021-07-16T18:15:13.3394795Z [cluster2]                                 "f:name": {}
2021-07-16T18:15:13.3395284Z [cluster2]                             },
2021-07-16T18:15:13.3395816Z [cluster2]                             "k:{\"name\":\"ipsecnss\"}": {
2021-07-16T18:15:13.3396368Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3396891Z [cluster2]                                 "f:emptyDir": {},
2021-07-16T18:15:13.3397447Z [cluster2]                                 "f:name": {}
2021-07-16T18:15:13.3398745Z [cluster2]                             }
2021-07-16T18:15:13.3399373Z [cluster2]                         }
2021-07-16T18:15:13.3399822Z [cluster2]                     }
2021-07-16T18:15:13.3400424Z [cluster2]                 },
2021-07-16T18:15:13.3401982Z [cluster2]                 "manager": "kube-controller-manager",
2021-07-16T18:15:13.3402917Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:13.3404431Z [cluster2]                 "time": "2021-07-16T18:14:12Z"
2021-07-16T18:15:13.3405475Z [cluster2]             },
2021-07-16T18:15:13.3406085Z [cluster2]             {
2021-07-16T18:15:13.3406592Z [cluster2]                 "apiVersion": "v1",
2021-07-16T18:15:13.3407211Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:13.3407792Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:13.3408313Z [cluster2]                     "f:status": {
2021-07-16T18:15:13.3408852Z [cluster2]                         "f:conditions": {
2021-07-16T18:15:13.3409670Z [cluster2]                             "k:{\"type\":\"ContainersReady\"}": {
2021-07-16T18:15:13.3410312Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3410882Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-16T18:15:13.3411614Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:13.3412261Z [cluster2]                                 "f:status": {},
2021-07-16T18:15:13.3412806Z [cluster2]                                 "f:type": {}
2021-07-16T18:15:13.3413290Z [cluster2]                             },
2021-07-16T18:15:13.3413865Z [cluster2]                             "k:{\"type\":\"Initialized\"}": {
2021-07-16T18:15:13.3414432Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3415017Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-16T18:15:13.3415738Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:13.3416384Z [cluster2]                                 "f:status": {},
2021-07-16T18:15:13.3416917Z [cluster2]                                 "f:type": {}
2021-07-16T18:15:13.3417398Z [cluster2]                             },
2021-07-16T18:15:13.3417923Z [cluster2]                             "k:{\"type\":\"Ready\"}": {
2021-07-16T18:15:13.3418447Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3419026Z [cluster2]                                 "f:lastProbeTime": {},
2021-07-16T18:15:13.3419735Z [cluster2]                                 "f:lastTransitionTime": {},
2021-07-16T18:15:13.3420478Z [cluster2]                                 "f:status": {},
2021-07-16T18:15:13.3421013Z [cluster2]                                 "f:type": {}
2021-07-16T18:15:13.3421496Z [cluster2]                             }
2021-07-16T18:15:13.3421957Z [cluster2]                         },
2021-07-16T18:15:13.3423031Z [cluster2]                         "f:containerStatuses": {},
2021-07-16T18:15:13.3423851Z [cluster2]                         "f:hostIP": {},
2021-07-16T18:15:13.3424386Z [cluster2]                         "f:phase": {},
2021-07-16T18:15:13.3424918Z [cluster2]                         "f:podIP": {},
2021-07-16T18:15:13.3425442Z [cluster2]                         "f:podIPs": {
2021-07-16T18:15:13.3425955Z [cluster2]                             ".": {},
2021-07-16T18:15:13.3426466Z [cluster2]                             "k:{\"ip\":\"172.18.0.5\"}": {
2021-07-16T18:15:13.3427321Z [cluster2]                                 ".": {},
2021-07-16T18:15:13.3427979Z [cluster2]                                 "f:ip": {}
2021-07-16T18:15:13.3428451Z [cluster2]                             }
2021-07-16T18:15:13.3428918Z [cluster2]                         },
2021-07-16T18:15:13.3429426Z [cluster2]                         "f:startTime": {}
2021-07-16T18:15:13.3429939Z [cluster2]                     }
2021-07-16T18:15:13.3430369Z [cluster2]                 },
2021-07-16T18:15:13.3430893Z [cluster2]                 "manager": "kubelet",
2021-07-16T18:15:13.3431910Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:13.3433000Z [cluster2]                 "time": "2021-07-16T18:14:30Z"
2021-07-16T18:15:13.3433496Z [cluster2]             },
2021-07-16T18:15:13.3433918Z [cluster2]             {
2021-07-16T18:15:13.3434440Z [cluster2]                 "apiVersion": "v1",
2021-07-16T18:15:13.3443079Z [cluster2]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:13.3444197Z [cluster2]                 "fieldsV1": {
2021-07-16T18:15:13.3444752Z [cluster2]                     "f:metadata": {
2021-07-16T18:15:13.3445300Z [cluster2]                         "f:labels": {
2021-07-16T18:15:13.3445984Z [cluster2]                             "f:gateway.submariner.io/node": {},
2021-07-16T18:15:13.3446865Z [cluster2]                             "f:gateway.submariner.io/status": {}
2021-07-16T18:15:13.3447521Z [cluster2]                         }
2021-07-16T18:15:13.3447977Z [cluster2]                     }
2021-07-16T18:15:13.3448575Z [cluster2]                 },
2021-07-16T18:15:13.3449595Z [cluster2]                 "manager": "submariner-gateway",
2021-07-16T18:15:13.3450308Z [cluster2]                 "operation": "Update",
2021-07-16T18:15:13.3451057Z [cluster2]                 "time": "2021-07-16T18:14:30Z"
2021-07-16T18:15:13.3451543Z [cluster2]             }
2021-07-16T18:15:13.3452627Z [cluster2]         ],
2021-07-16T18:15:13.3454276Z [cluster2]         "name": "submariner-gateway-gfcst",
2021-07-16T18:15:13.3455506Z [cluster2]         "namespace": "submariner-operator",
2021-07-16T18:15:13.3457020Z [cluster2]         "ownerReferences": [
2021-07-16T18:15:13.3457592Z [cluster2]             {
2021-07-16T18:15:13.3458132Z [cluster2]                 "apiVersion": "apps/v1",
2021-07-16T18:15:13.3458846Z [cluster2]                 "blockOwnerDeletion": true,
2021-07-16T18:15:13.3459516Z [cluster2]                 "controller": true,
2021-07-16T18:15:13.3460104Z [cluster2]                 "kind": "DaemonSet",
2021-07-16T18:15:13.3460987Z [cluster2]                 "name": "submariner-gateway",
2021-07-16T18:15:13.3462042Z [cluster2]                 "uid": "7a85b5a5-ff5b-4fcd-b70d-0c44966555a6"
2021-07-16T18:15:13.3462717Z [cluster2]             }
2021-07-16T18:15:13.3463147Z [cluster2]         ],
2021-07-16T18:15:13.3463684Z [cluster2]         "resourceVersion": "2051",
2021-07-16T18:15:13.3464988Z [cluster2]         "selfLink": "/api/v1/namespaces/submariner-operator/pods/submariner-gateway-gfcst",
2021-07-16T18:15:13.3466315Z [cluster2]         "uid": "d130b7a6-9472-460d-8ec7-f3885c301092"
2021-07-16T18:15:13.3467040Z [cluster2]     },
2021-07-16T18:15:13.3467857Z [cluster2]     "spec": {
2021-07-16T18:15:13.3468483Z [cluster2]         "affinity": {
2021-07-16T18:15:13.3469056Z [cluster2]             "nodeAffinity": {
2021-07-16T18:15:13.3470291Z [cluster2]                 "requiredDuringSchedulingIgnoredDuringExecution": {
2021-07-16T18:15:13.3471604Z [cluster2]                     "nodeSelectorTerms": [
2021-07-16T18:15:13.3472195Z [cluster2]                         {
2021-07-16T18:15:13.3472728Z [cluster2]                             "matchFields": [
2021-07-16T18:15:13.3473256Z [cluster2]                                 {
2021-07-16T18:15:13.3473839Z [cluster2]                                     "key": "metadata.name",
2021-07-16T18:15:13.3474469Z [cluster2]                                     "operator": "In",
2021-07-16T18:15:13.3475020Z [cluster2]                                     "values": [
2021-07-16T18:15:13.3475875Z [cluster2]                                         "cluster2-worker"
2021-07-16T18:15:13.3476452Z [cluster2]                                     ]
2021-07-16T18:15:13.3476925Z [cluster2]                                 }
2021-07-16T18:15:13.3477379Z [cluster2]                             ]
2021-07-16T18:15:13.3477840Z [cluster2]                         }
2021-07-16T18:15:13.3478280Z [cluster2]                     ]
2021-07-16T18:15:13.3478724Z [cluster2]                 }
2021-07-16T18:15:13.3479153Z [cluster2]             },
2021-07-16T18:15:13.3479715Z [cluster2]             "podAntiAffinity": {
2021-07-16T18:15:13.3480995Z [cluster2]                 "requiredDuringSchedulingIgnoredDuringExecution": [
2021-07-16T18:15:13.3482147Z [cluster2]                     {
2021-07-16T18:15:13.3482700Z [cluster2]                         "labelSelector": {
2021-07-16T18:15:13.3483316Z [cluster2]                             "matchLabels": {
2021-07-16T18:15:13.3484392Z [cluster2]                                 "app": "submariner-gateway"
2021-07-16T18:15:13.3484999Z [cluster2]                             }
2021-07-16T18:15:13.3485463Z [cluster2]                         },
2021-07-16T18:15:13.3486118Z [cluster2]                         "topologyKey": "kubernetes.io/hostname"
2021-07-16T18:15:13.3486770Z [cluster2]                     }
2021-07-16T18:15:13.3487209Z [cluster2]                 ]
2021-07-16T18:15:13.3487630Z [cluster2]             }
2021-07-16T18:15:13.3488062Z [cluster2]         },
2021-07-16T18:15:13.3488656Z [cluster2]         "containers": [
2021-07-16T18:15:13.3489157Z [cluster2]             {
2021-07-16T18:15:13.3489616Z [cluster2]                 "command": [
2021-07-16T18:15:13.3490184Z [cluster2]                     "submariner.sh"
2021-07-16T18:15:13.3490690Z [cluster2]                 ],
2021-07-16T18:15:13.3491152Z [cluster2]                 "env": [
2021-07-16T18:15:13.3491601Z [cluster2]                     {
2021-07-16T18:15:13.3492195Z [cluster2]                         "name": "SUBMARINER_NAMESPACE",
2021-07-16T18:15:13.3493146Z [cluster2]                         "value": "submariner-operator"
2021-07-16T18:15:13.3493779Z [cluster2]                     },
2021-07-16T18:15:13.3494226Z [cluster2]                     {
2021-07-16T18:15:13.3494811Z [cluster2]                         "name": "SUBMARINER_CLUSTERCIDR",
2021-07-16T18:15:13.3495442Z [cluster2]                         "value": "10.2.0.0/16"
2021-07-16T18:15:13.3495909Z [cluster2]                     },
2021-07-16T18:15:13.3496357Z [cluster2]                     {
2021-07-16T18:15:13.3496941Z [cluster2]                         "name": "SUBMARINER_SERVICECIDR",
2021-07-16T18:15:13.3497577Z [cluster2]                         "value": "100.2.0.0/16"
2021-07-16T18:15:13.3498047Z [cluster2]                     },
2021-07-16T18:15:13.3498492Z [cluster2]                     {
2021-07-16T18:15:13.3499066Z [cluster2]                         "name": "SUBMARINER_GLOBALCIDR"
2021-07-16T18:15:13.3499639Z [cluster2]                     },
2021-07-16T18:15:13.3500171Z [cluster2]                     {
2021-07-16T18:15:13.3500738Z [cluster2]                         "name": "SUBMARINER_CLUSTERID",
2021-07-16T18:15:13.3501384Z [cluster2]                         "value": "cluster2"
2021-07-16T18:15:13.3501880Z [cluster2]                     },
2021-07-16T18:15:13.3502320Z [cluster2]                     {
2021-07-16T18:15:13.3502895Z [cluster2]                         "name": "SUBMARINER_COLORCODES",
2021-07-16T18:15:13.3503527Z [cluster2]                         "value": "blue"
2021-07-16T18:15:13.3504006Z [cluster2]                     },
2021-07-16T18:15:13.3504453Z [cluster2]                     {
2021-07-16T18:15:13.3505011Z [cluster2]                         "name": "SUBMARINER_DEBUG",
2021-07-16T18:15:13.3505591Z [cluster2]                         "value": "false"
2021-07-16T18:15:13.3506087Z [cluster2]                     },
2021-07-16T18:15:13.3506518Z [cluster2]                     {
2021-07-16T18:15:13.3507107Z [cluster2]                         "name": "SUBMARINER_NATENABLED",
2021-07-16T18:15:13.3507726Z [cluster2]                         "value": "false"
2021-07-16T18:15:13.3508222Z [cluster2]                     },
2021-07-16T18:15:13.3508650Z [cluster2]                     {
2021-07-16T18:15:13.3509212Z [cluster2]                         "name": "SUBMARINER_BROKER",
2021-07-16T18:15:13.3510699Z [cluster2]                         "value": "k8s"
2021-07-16T18:15:13.3511350Z [cluster2]                     },
2021-07-16T18:15:13.3511801Z [cluster2]                     {
2021-07-16T18:15:13.3512391Z [cluster2]                         "name": "SUBMARINER_CABLEDRIVER",
2021-07-16T18:15:13.3513060Z [cluster2]                         "value": "libreswan"
2021-07-16T18:15:13.3513567Z [cluster2]                     },
2021-07-16T18:15:13.3514011Z [cluster2]                     {
2021-07-16T18:15:13.3514562Z [cluster2]                         "name": "BROKER_K8S_APISERVER",
2021-07-16T18:15:13.3515166Z [cluster2]                         "value": "172.18.0.8:6443"
2021-07-16T18:15:13.3516232Z [cluster2]                     },
2021-07-16T18:15:13.3516702Z [cluster2]                     {
2021-07-16T18:15:13.3517302Z [cluster2]                         "name": "BROKER_K8S_APISERVERTOKEN",
2021-07-16T18:15:13.3673862Z [cluster2]                         "value": "***"
2021-07-16T18:15:13.3674433Z [cluster2]                     },
2021-07-16T18:15:13.3674877Z [cluster2]                     {
2021-07-16T18:15:13.3676508Z [cluster2]                         "name": "BROKER_K8S_REMOTENAMESPACE",
2021-07-16T18:15:13.3678146Z [cluster2]                         "value": "submariner-k8s-broker"
2021-07-16T18:15:13.3678811Z [cluster2]                     },
2021-07-16T18:15:13.3679250Z [cluster2]                     {
2021-07-16T18:15:13.3679784Z [cluster2]                         "name": "BROKER_K8S_CA",
2021-07-16T18:15:13.4226358Z [cluster2]                         "value": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EY3hOakU0TURZMU4xb1hEVE14TURjeE5ERTRNRFkxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTTBSClUwU3RsQjBtaldFOGVuSWUxK0Q4SFEvS0JtUkwxMnZzd2haZ3RFYXkvZ0ZWT2lKNEZnaFBIcHNMS0lHVEFQWkYKYlpINXh0V3RYWGVTakVsVFdTUlhjUm84NVVvV2wwSWk0cTFwMWNLcDI5M2IwdTFUZUZSbFU5b3pYVzBWQnJnSAp5TUdUNHRCNUVPVm5odDBFUVNQQjlrT2FVN1JHbnc0T3NuU25WN2JXaG5FYUZrdGlrRjQ5N25hSjNocEZrNVUvClgvcHVtYnRIdkFaM1p2em1UbEdxbVVrZ2ltZUtYSHNjOHJCdk0ybzhmc3daWCs2dUVhYW1wQi9sQ3ppR3YydTIKakdqTVNLK0h5N0dCTmtVWkVYY0ZZczFVM0VuQTJRaVpWNDBnb2U3UkhydzF3UFd1blcxcjhhSlFzTm5ZYk9FRgpadFRtc1BxQzRBMHBYZzdRVTVjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFEQnIyYXM3TXBTc0RUUTRoWDUvQktTb214dDgKcTNKWVpWeXU5MlpXZUNxMFZ3TEpJRFV2dEQ2alFnbkk2OEJsMTV0dUVRdUdnVEZxTGtlMk1GbzZtWkFkd2F6UAo0RWFBYk4rekdreUd1WVFxM1FyNXJuelRTZWE2Ym5hQm1Kd2tQQnNkSnYrT2RHUVhjNHFRMC9JMzBrVy9KU09RCndId3BIanVkakRCaHY2aWdvbFAvODNVaDBrMVZzYmhLRitVbWxkd21VRHBRSzA3c1ZJb0lSSTcvcDFudm8vWnIKREZIYjk3TC9vVklFRXdScE81SEhaSURJSzJTVG5OTXc2VUYyeXF5WGtwOHZvcFhOZWQxaFhadUhTOVN5b1JHLwpCbmNuSGZsRmpVMmE2UkhBK0NVRlFVN1JSSWVZWkEwcjEvMVRoUDZvelhtVGZSUTRueFh3QWcxZWdwbz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo="
2021-07-16T18:15:13.4777346Z [cluster2]                     },
2021-07-16T18:15:13.4778301Z [cluster2]                     {
2021-07-16T18:15:13.4779310Z [cluster2]                         "name": "CE_IPSEC_PSK",
2021-07-16T18:15:13.4780786Z [cluster2]                         "value": "Zhs+SumA5VO7u+aRnCe5Wjoau5QpxSYF1mxiCBvpnGHaZ/d40WAFSnV803hZx1RR"
2021-07-16T18:15:13.4781973Z [cluster2]                     },
2021-07-16T18:15:13.4782411Z [cluster2]                     {
2021-07-16T18:15:13.4783720Z [cluster2]                         "name": "CE_IPSEC_DEBUG",
2021-07-16T18:15:13.4784447Z [cluster2]                         "value": "false"
2021-07-16T18:15:13.4784948Z [cluster2]                     },
2021-07-16T18:15:13.4785671Z [cluster2]                     {
2021-07-16T18:15:13.4786491Z [cluster2]                         "name": "SUBMARINER_HEALTHCHECKENABLED",
2021-07-16T18:15:13.4787204Z [cluster2]                         "value": "true"
2021-07-16T18:15:13.4787682Z [cluster2]                     },
2021-07-16T18:15:13.4788127Z [cluster2]                     {
2021-07-16T18:15:13.4788790Z [cluster2]                         "name": "SUBMARINER_HEALTHCHECKINTERVAL",
2021-07-16T18:15:13.4789495Z [cluster2]                         "value": "1"
2021-07-16T18:15:13.4789967Z [cluster2]                     },
2021-07-16T18:15:13.4790415Z [cluster2]                     {
2021-07-16T18:15:13.4791251Z [cluster2]                         "name": "SUBMARINER_HEALTHCHECKMAXPACKETLOSSCOUNT",
2021-07-16T18:15:13.4792096Z [cluster2]                         "value": "5"
2021-07-16T18:15:13.4792572Z [cluster2]                     },
2021-07-16T18:15:13.4793007Z [cluster2]                     {
2021-07-16T18:15:13.4793521Z [cluster2]                         "name": "NODE_NAME",
2021-07-16T18:15:13.4794078Z [cluster2]                         "valueFrom": {
2021-07-16T18:15:13.4794644Z [cluster2]                             "fieldRef": {
2021-07-16T18:15:13.4795221Z [cluster2]                                 "apiVersion": "v1",
2021-07-16T18:15:13.4795907Z [cluster2]                                 "fieldPath": "spec.nodeName"
2021-07-16T18:15:13.4796500Z [cluster2]                             }
2021-07-16T18:15:13.4796952Z [cluster2]                         }
2021-07-16T18:15:13.4797743Z [cluster2]                     },
2021-07-16T18:15:13.4798832Z [cluster2]                     {
2021-07-16T18:15:13.4799961Z [cluster2]                         "name": "POD_NAME",
2021-07-16T18:15:13.4800528Z [cluster2]                         "valueFrom": {
2021-07-16T18:15:13.4801089Z [cluster2]                             "fieldRef": {
2021-07-16T18:15:13.4801670Z [cluster2]                                 "apiVersion": "v1",
2021-07-16T18:15:13.4802357Z [cluster2]                                 "fieldPath": "metadata.name"
2021-07-16T18:15:13.4802956Z [cluster2]                             }
2021-07-16T18:15:13.4803413Z [cluster2]                         }
2021-07-16T18:15:13.4804004Z [cluster2]                     },
2021-07-16T18:15:13.4804445Z [cluster2]                     {
2021-07-16T18:15:13.4804994Z [cluster2]                         "name": "CE_IPSEC_IKEPORT",
2021-07-16T18:15:13.4805545Z [cluster2]                         "value": "500"
2021-07-16T18:15:13.4806029Z [cluster2]                     },
2021-07-16T18:15:13.4806468Z [cluster2]                     {
2021-07-16T18:15:13.4807024Z [cluster2]                         "name": "CE_IPSEC_NATTPORT",
2021-07-16T18:15:13.4807585Z [cluster2]                         "value": "4500"
2021-07-16T18:15:13.4808081Z [cluster2]                     },
2021-07-16T18:15:13.4808518Z [cluster2]                     {
2021-07-16T18:15:13.4809137Z [cluster2]                         "name": "CE_IPSEC_PREFERREDSERVER",
2021-07-16T18:15:13.4809796Z [cluster2]                         "value": "false"
2021-07-16T18:15:13.4810422Z [cluster2]                     },
2021-07-16T18:15:13.4810875Z [cluster2]                     {
2021-07-16T18:15:13.4811435Z [cluster2]                         "name": "CE_IPSEC_FORCEENCAPS",
2021-07-16T18:15:13.4812056Z [cluster2]                         "value": "false"
2021-07-16T18:15:13.4812538Z [cluster2]                     }
2021-07-16T18:15:13.4812982Z [cluster2]                 ],
2021-07-16T18:15:13.4814710Z [cluster2]                 "image": "localhost:5000/submariner-gateway:local",
2021-07-16T18:15:13.4815557Z [cluster2]                 "imagePullPolicy": "Always",
2021-07-16T18:15:13.4816469Z [cluster2]                 "name": "submariner-gateway",
2021-07-16T18:15:13.4817102Z [cluster2]                 "ports": [
2021-07-16T18:15:13.4817580Z [cluster2]                     {
2021-07-16T18:15:13.4818290Z [cluster2]                         "containerPort": 4500,
2021-07-16T18:15:13.4819056Z [cluster2]                         "hostPort": 4500,
2021-07-16T18:15:13.4819881Z [cluster2]                         "name": "cable-encaps",
2021-07-16T18:15:13.4820501Z [cluster2]                         "protocol": "UDP"
2021-07-16T18:15:13.4820998Z [cluster2]                     },
2021-07-16T18:15:13.4821439Z [cluster2]                     {
2021-07-16T18:15:13.4821971Z [cluster2]                         "containerPort": 4490,
2021-07-16T18:15:13.4822581Z [cluster2]                         "hostPort": 4490,
2021-07-16T18:15:13.4823382Z [cluster2]                         "name": "natt-discovery",
2021-07-16T18:15:13.4823994Z [cluster2]                         "protocol": "UDP"
2021-07-16T18:15:13.4824498Z [cluster2]                     }
2021-07-16T18:15:13.4824928Z [cluster2]                 ],
2021-07-16T18:15:13.4825431Z [cluster2]                 "resources": {},
2021-07-16T18:15:13.4826033Z [cluster2]                 "securityContext": {
2021-07-16T18:15:13.4826848Z [cluster2]                     "allowPrivilegeEscalation": true,
2021-07-16T18:15:13.4827615Z [cluster2]                     "capabilities": {
2021-07-16T18:15:13.4828156Z [cluster2]                         "add": [
2021-07-16T18:15:13.4828641Z [cluster2]                             "net_admin"
2021-07-16T18:15:13.4829134Z [cluster2]                         ],
2021-07-16T18:15:13.4829613Z [cluster2]                         "drop": [
2021-07-16T18:15:13.4830091Z [cluster2]                             "all"
2021-07-16T18:15:13.4830559Z [cluster2]                         ]
2021-07-16T18:15:13.4831121Z [cluster2]                     },
2021-07-16T18:15:13.4831667Z [cluster2]                     "privileged": true,
2021-07-16T18:15:13.4832410Z [cluster2]                     "readOnlyRootFilesystem": false,
2021-07-16T18:15:13.4833182Z [cluster2]                     "runAsNonRoot": false
2021-07-16T18:15:13.4833703Z [cluster2]                 },
2021-07-16T18:15:13.4834694Z [cluster2]                 "terminationMessagePath": "/dev/termination-log",
2021-07-16T18:15:13.4835710Z [cluster2]                 "terminationMessagePolicy": "File",
2021-07-16T18:15:13.4836488Z [cluster2]                 "volumeMounts": [
2021-07-16T18:15:13.4837009Z [cluster2]                     {
2021-07-16T18:15:13.4837563Z [cluster2]                         "mountPath": "/etc/ipsec.d",
2021-07-16T18:15:13.4838211Z [cluster2]                         "name": "ipsecd"
2021-07-16T18:15:13.4838695Z [cluster2]                     },
2021-07-16T18:15:13.4839140Z [cluster2]                     {
2021-07-16T18:15:13.4839704Z [cluster2]                         "mountPath": "/var/lib/ipsec/nss",
2021-07-16T18:15:13.4840348Z [cluster2]                         "name": "ipsecnss"
2021-07-16T18:15:13.4840843Z [cluster2]                     },
2021-07-16T18:15:13.4841288Z [cluster2]                     {
2021-07-16T18:15:13.4842022Z [cluster2]                         "mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
2021-07-16T18:15:13.4843315Z [cluster2]                         "name": "submariner-gateway-token-pj2z6",
2021-07-16T18:15:13.4844531Z [cluster2]                         "readOnly": true
2021-07-16T18:15:13.4845024Z [cluster2]                     }
2021-07-16T18:15:13.4845464Z [cluster2]                 ]
2021-07-16T18:15:13.4845890Z [cluster2]             }
2021-07-16T18:15:13.4846736Z [cluster2]         ],
2021-07-16T18:15:13.4847439Z [cluster2]         "dnsPolicy": "ClusterFirst",
2021-07-16T18:15:13.4848182Z [cluster2]         "enableServiceLinks": true,
2021-07-16T18:15:13.4848856Z [cluster2]         "hostNetwork": true,
2021-07-16T18:15:13.4850344Z [cluster2]         "nodeName": "cluster2-worker",
2021-07-16T18:15:13.4851018Z [cluster2]         "nodeSelector": {
2021-07-16T18:15:13.4851662Z [cluster2]             "submariner.io/gateway": "true"
2021-07-16T18:15:13.4852232Z [cluster2]         },
2021-07-16T18:15:13.4852694Z [cluster2]         "priority": 0,
2021-07-16T18:15:13.4853287Z [cluster2]         "restartPolicy": "Always",
2021-07-16T18:15:13.4854244Z [cluster2]         "schedulerName": "default-scheduler",
2021-07-16T18:15:13.4855007Z [cluster2]         "securityContext": {},
2021-07-16T18:15:13.4855942Z [cluster2]         "serviceAccount": "submariner-gateway",
2021-07-16T18:15:13.4858085Z [cluster2]         "serviceAccountName": "submariner-gateway",
2021-07-16T18:15:13.4859122Z [cluster2]         "terminationGracePeriodSeconds": 1,
2021-07-16T18:15:13.4859952Z [cluster2]         "tolerations": [
2021-07-16T18:15:13.4860459Z [cluster2]             {
2021-07-16T18:15:13.4860962Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4861470Z [cluster2]             },
2021-07-16T18:15:13.4861895Z [cluster2]             {
2021-07-16T18:15:13.4862427Z [cluster2]                 "effect": "NoExecute",
2021-07-16T18:15:13.4863351Z [cluster2]                 "key": "node.kubernetes.io/not-ready",
2021-07-16T18:15:13.4864064Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4864559Z [cluster2]             },
2021-07-16T18:15:13.4864989Z [cluster2]             {
2021-07-16T18:15:13.4865510Z [cluster2]                 "effect": "NoExecute",
2021-07-16T18:15:13.4866980Z [cluster2]                 "key": "node.kubernetes.io/unreachable",
2021-07-16T18:15:13.4867883Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4868374Z [cluster2]             },
2021-07-16T18:15:13.4868805Z [cluster2]             {
2021-07-16T18:15:13.4869330Z [cluster2]                 "effect": "NoSchedule",
2021-07-16T18:15:13.4870335Z [cluster2]                 "key": "node.kubernetes.io/disk-pressure",
2021-07-16T18:15:13.4871229Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4871739Z [cluster2]             },
2021-07-16T18:15:13.4872151Z [cluster2]             {
2021-07-16T18:15:13.4872690Z [cluster2]                 "effect": "NoSchedule",
2021-07-16T18:15:13.4873671Z [cluster2]                 "key": "node.kubernetes.io/memory-pressure",
2021-07-16T18:15:13.4874446Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4874954Z [cluster2]             },
2021-07-16T18:15:13.4875374Z [cluster2]             {
2021-07-16T18:15:13.4875911Z [cluster2]                 "effect": "NoSchedule",
2021-07-16T18:15:13.4876851Z [cluster2]                 "key": "node.kubernetes.io/pid-pressure",
2021-07-16T18:15:13.4878140Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4878792Z [cluster2]             },
2021-07-16T18:15:13.4879229Z [cluster2]             {
2021-07-16T18:15:13.4879755Z [cluster2]                 "effect": "NoSchedule",
2021-07-16T18:15:13.4880531Z [cluster2]                 "key": "node.kubernetes.io/unschedulable",
2021-07-16T18:15:13.4881269Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4881771Z [cluster2]             },
2021-07-16T18:15:13.4882201Z [cluster2]             {
2021-07-16T18:15:13.4882759Z [cluster2]                 "effect": "NoSchedule",
2021-07-16T18:15:13.4883919Z [cluster2]                 "key": "node.kubernetes.io/network-unavailable",
2021-07-16T18:15:13.4884724Z [cluster2]                 "operator": "Exists"
2021-07-16T18:15:13.4885360Z [cluster2]             }
2021-07-16T18:15:13.4885771Z [cluster2]         ],
2021-07-16T18:15:13.4886237Z [cluster2]         "volumes": [
2021-07-16T18:15:13.4886690Z [cluster2]             {
2021-07-16T18:15:13.4887184Z [cluster2]                 "emptyDir": {},
2021-07-16T18:15:13.4887724Z [cluster2]                 "name": "ipsecd"
2021-07-16T18:15:13.4888207Z [cluster2]             },
2021-07-16T18:15:13.4888628Z [cluster2]             {
2021-07-16T18:15:13.4889122Z [cluster2]                 "emptyDir": {},
2021-07-16T18:15:13.4889685Z [cluster2]                 "name": "ipsecnss"
2021-07-16T18:15:13.4890167Z [cluster2]             },
2021-07-16T18:15:13.4891192Z [cluster2]             {
2021-07-16T18:15:13.4892621Z [cluster2]                 "name": "submariner-gateway-token-pj2z6",
2021-07-16T18:15:13.4893446Z [cluster2]                 "secret": {
2021-07-16T18:15:13.4893994Z [cluster2]                     "defaultMode": 420,
2021-07-16T18:15:13.4895103Z [cluster2]                     "secretName": "submariner-gateway-token-pj2z6"
2021-07-16T18:15:13.4895924Z [cluster2]                 }
2021-07-16T18:15:13.4896367Z [cluster2]             }
2021-07-16T18:15:13.4896780Z [cluster2]         ]
2021-07-16T18:15:13.4897205Z [cluster2]     },
2021-07-16T18:15:13.4897639Z [cluster2]     "status": {
2021-07-16T18:15:13.4898158Z [cluster2]         "conditions": [
2021-07-16T18:15:13.4898643Z [cluster2]             {
2021-07-16T18:15:13.4899184Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:13.4900108Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:14:12Z",
2021-07-16T18:15:13.4900775Z [cluster2]                 "status": "True",
2021-07-16T18:15:13.4901350Z [cluster2]                 "type": "Initialized"
2021-07-16T18:15:13.4902590Z [cluster2]             },
2021-07-16T18:15:13.4903028Z [cluster2]             {
2021-07-16T18:15:13.4903566Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:13.4904536Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:14:30Z",
2021-07-16T18:15:13.4905209Z [cluster2]                 "status": "True",
2021-07-16T18:15:13.4905744Z [cluster2]                 "type": "Ready"
2021-07-16T18:15:13.4906226Z [cluster2]             },
2021-07-16T18:15:13.4906641Z [cluster2]             {
2021-07-16T18:15:13.4907190Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:13.4908486Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:14:30Z",
2021-07-16T18:15:13.4909577Z [cluster2]                 "status": "True",
2021-07-16T18:15:13.4910202Z [cluster2]                 "type": "ContainersReady"
2021-07-16T18:15:13.4910769Z [cluster2]             },
2021-07-16T18:15:13.4911186Z [cluster2]             {
2021-07-16T18:15:13.4912028Z [cluster2]                 "lastProbeTime": null,
2021-07-16T18:15:13.4913254Z [cluster2]                 "lastTransitionTime": "2021-07-16T18:14:12Z",
2021-07-16T18:15:13.4913935Z [cluster2]                 "status": "True",
2021-07-16T18:15:13.4914536Z [cluster2]                 "type": "PodScheduled"
2021-07-16T18:15:13.4915055Z [cluster2]             }
2021-07-16T18:15:13.4915478Z [cluster2]         ],
2021-07-16T18:15:13.4916030Z [cluster2]         "containerStatuses": [
2021-07-16T18:15:13.4916597Z [cluster2]             {
2021-07-16T18:15:13.4919135Z [cluster2]                 "containerID": "containerd://86db556503d6a4fde26bc6585ec1b45df01b80fe0173800a3ef370c2511b91e5",
2021-07-16T18:15:13.4921519Z [cluster2]                 "image": "localhost:5000/submariner-gateway:local",
2021-07-16T18:15:13.4923458Z [cluster2]                 "imageID": "localhost:5000/submariner-gateway@sha256:c57547568ee3a8d71547861c0c114d06078afd8749343c1a18593277e3e2266a",
2021-07-16T18:15:13.4925181Z [cluster2]                 "lastState": {},
2021-07-16T18:15:13.4926069Z [cluster2]                 "name": "submariner-gateway",
2021-07-16T18:15:13.4926699Z [cluster2]                 "ready": true,
2021-07-16T18:15:13.4927265Z [cluster2]                 "restartCount": 0,
2021-07-16T18:15:13.4927979Z [cluster2]                 "started": true,
2021-07-16T18:15:13.4928499Z [cluster2]                 "state": {
2021-07-16T18:15:13.4928993Z [cluster2]                     "running": {
2021-07-16T18:15:13.4929781Z [cluster2]                         "startedAt": "2021-07-16T18:14:29Z"
2021-07-16T18:15:13.4930326Z [cluster2]                     }
2021-07-16T18:15:13.4930754Z [cluster2]                 }
2021-07-16T18:15:13.4931188Z [cluster2]             }
2021-07-16T18:15:13.4931606Z [cluster2]         ],
2021-07-16T18:15:13.4932086Z [cluster2]         "hostIP": "172.18.0.5",
2021-07-16T18:15:13.4932614Z [cluster2]         "phase": "Running",
2021-07-16T18:15:13.4933149Z [cluster2]         "podIP": "172.18.0.5",
2021-07-16T18:15:13.4933635Z [cluster2]         "podIPs": [
2021-07-16T18:15:13.4934663Z [cluster2]             {
2021-07-16T18:15:13.6904049Z [cluster2]                 "ip": "172NumberScheduled}
2021-07-16T18:15:13.6906017Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get DaemonSet -n submariner-operator -o jsonpath={.status.desiredNumberScheduled}
2021-07-16T18:15:13.6907751Z [submariner-operator]$ [cluster1] wait 8656
2021-07-16T18:15:13.6909313Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-16T18:15:13.6911204Z [submariner-operator]$ [cluster1] kubectl get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-16T18:15:13.6913260Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get DaemonSet -n submariner-operator -o jsonpath={.status.numberReady}
2021-07-16T18:15:13.6914763Z [submariner-operator]$ [cluster1] return 0
2021-07-16T18:15:13.6915824Z [submariner-operator]$ [cluster1] verify_subm_gateway_pod
2021-07-16T18:15:13.6916929Z [submariner-operator]$ [cluster1] verify_subm_gateway_pod
2021-07-16T18:15:13.6918626Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-16T18:15:13.6920805Z [submariner-operator]$ [cluster1] kubectl wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-16T18:15:13.6923113Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 wait --for=condition=Ready pods -l app=submariner-gateway --timeout=120s --namespace=submariner-operator
2021-07-16T18:15:13.6925181Z [cluster1] pod/submariner-gateway-wh7jp condition met
2021-07-16T18:15:13.6926924Z [submariner-operator]$ [cluster1] subm_gateway_pod_name=$(kubectl get pods --namespace=$subm_ns -l app=$gateway_deployment_name -o=jsonpath='{.items..metadata.name}')
2021-07-16T18:15:13.6929149Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:13.6931592Z [submariner-operator]$ [cluster1] kubectl get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:13.6934023Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pods --namespace=submariner-operator -l app=submariner-gateway -o=jsonpath={.items..metadata.name}
2021-07-16T18:15:13.6936410Z [submariner-operator]$ [cluster1] json_file=/tmp/submariner-gateway-wh7jp.cluster1.json
2021-07-16T18:15:13.6938555Z [submariner-operator]$ [cluster1] kubectl get pod submariner-gateway-wh7jp --namespace=submariner-operator -o json
2021-07-16T18:15:13.6940514Z [submariner-operator]$ [cluster1] tee /tmp/submariner-gateway-wh7jp.cluster1.json
2021-07-16T18:15:13.6942633Z [submariner-operator]$ [cluster1] kubectl get pod submariner-gateway-wh7jp --namespace=submariner-operator -o json
2021-07-16T18:15:13.6944797Z [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get pod submariner-gateway-wh7jp --namespace=submariner-operator -o json
2021-07-16T18:15:13.6946132Z [cluster1] {
2021-07-16T18:15:13.6946616Z [cluster1]     "apiVersion": "v1",
2021-07-16T18:15:13.6947145Z [cluster1]     "kind": "Pod",
2021-07-16T18:15:13.6947626Z [cluster1]     "metadata": {
2021-07-16T18:15:13.6948468Z [cluster1]         "creationTimestamp": "2021-07-16T18:13:52Z",
2021-07-16T18:15:13.6949470Z [cluster1]         "generateName": "submariner-gateway-",
2021-07-16T18:15:13.6950145Z [cluster1]         "labels": {
2021-07-16T18:15:13.6950927Z [cluster1]             "app": "submariner-gateway",
2021-07-16T18:15:13.6952076Z [cluster1]             "controller-revision-hash": "765bb9989f",
2021-07-16T18:15:13.6953233Z [cluster1]             "gateway.submariner.io/node": "cluster1-worker",
2021-07-16T18:15:13.6954175Z [cluster1]             "gateway.submariner.io/status": "active",
2021-07-16T18:15:13.6955207Z [cluster1]             "pod-template-generation": "1"
2021-07-16T18:15:13.6955839Z [cluster1]         },
2021-07-16T18:15:13.6956367Z [cluster1]         "managedFields": [
2021-07-16T18:15:13.6956891Z [cluster1]             {
2021-07-16T18:15:13.6957389Z [cluster1]                 "apiVersion": "v1",
2021-07-16T18:15:13.6958014Z [cluster1]                 "fieldsType": "FieldsV1",
2021-07-16T18:15:13.6958588Z [cluster1]                 "fieldsV1": {
2021-07-16T18:15:13.6959124Z [cluster1]                     "f:metadata": {
2021-07-16T18:15:13.6959707Z [cluster1]                         "f:generateName": {},
2021-07-16T18:15:13.6960294Z [cluster1]                         "f:labels": {
2021-07-16T18:15:13.6960783Z [cluster1]                             ".": {},
2021-07-16T18:15:13.6961275Z [cluster1]                             "f:app": {},
2021-07-16T18:15:13.6987125Z [cluster1]                             "f:controller-revision-hash": {},
2021-07-16T18:15:13.6988646Z [cluster1]                             "f:pod-template-generation": {}
2021-07-16T18:15:13.6989472Z [cluster1]                         },
2021-07-16T18:15:13.6990205Z [cluster1]                         "f:ownerReferences": {
2021-07-16T18:15:13.6990931Z [cluster1]                             ".": {},
2021-07-16T18:15:13.6992113Z [cluster1]                             "k:{\"uid\":\"60a1865a-c19b-4f77-93c5-587615cc97f4\"}": {
2021-07-16T18:15:13.7005700Z [cluster1]                                 ".": {},
2021-07-16T18:15:13.7006528Z [cluster1]                                 "f:apiVersion": {},
2021-07-16T18:15:13.7007681Z [cluster1]                                 "f:blockOwnerDeletion": {},
2021-07-16T18:15:13.7008506Z [cluster1]                                 "f:controller": {},
2021-07-16T18:15:13.7009150Z [cluster1]                                 "f:kind": {},
2021-07-16T18:15:13.7009775Z [cluster1]                                 "f:name": {},
2021-07-16T18:15:13.7010385Z [cluster1]                                 "f:uid": {}
2021-07-16T18:15:13.7010947Z [cluster1]                             }
2021-07-16T18:15:13.7011502Z [cluster1]                         }
2021-07-16T18:15:13.7012044Z [cluster1]                     },
2021-07-16T18:15:13.7012615Z [cluster1]                     "f:spec": {
2021-07-16T18:15:13.7013344Z [cluster1]                         "f:affinity": {
2021-07-16T18:15:13.7013948Z [cluster1]                             ".": {},
2021-07-16T18:15:13.7014601Z [cluster1]                             "f:nodeAffinity": {
2021-07-16T18:15:13.7015236Z [cluster1]                                 ".": {},
2021-07-16T18:15:13.7016537Z [cluster1]                                 "f:requiredDuringSchedulingIgnoredDuringExecution": {
2021-07-16T18:15:13.7017837Z [cluster1]                                     ".": {},
2021-07-16T18:15:13.7018555Z [cluster1]                                     "f:nodeSelectorTerms": {}
2021-07-16T18:15:13.7019278Z [cluster1]                                 }
2021-07-16T18:15:13.7019824Z [cluster1]                             },
2021-07-16T18:15:13.7020512Z [cluster1]                             "f:podAntiAffinity": {
2021-07-16T18:15:13.7021323Z [cluster1]                                 ".": {},
2021-07-16T18:15:13.7022609Z [cluster1]                                 "f:requiredDuringSchedulingIgnoredDuringExecution": {}
2021-07-16T18:15:13.7023907Z [cluster1]                             }
2021-07-16T18:15:13.7024465Z [cluster1]                         },
2021-07-16T18:15:13.7025073Z [cluster1]                         "f:containers": {
2021-07-16T18:15:13.7026268Z [cluster1]                             "k:{\"name\":\"submariner-gateway\"}": {
2021-07-16T18:15:13.7027027Z [cluster1]                                 ".": {},
2021-07-16T18:15:13.7027652Z [cluster1]                                 "f:command": {},
2021-07-16T18:15:13.7028285Z [cluster1]                                 "f:env": {
2021-07-16T18:15:13.7028869Z [cluster1]                                     ".": {},
2021-07-16T18:15:13.7029591Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_APISERVER\"}": {
2021-07-16T18:15:13.7030305Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7030924Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7031561Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7032157Z [cluster1]                                     },
2021-07-16T18:15:13.7032910Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_APISERVERTOKEN\"}": {
2021-07-16T18:15:13.7033675Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7034280Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7034916Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7035511Z [cluster1]                                     },
2021-07-16T18:15:13.7036175Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_CA\"}": {
2021-07-16T18:15:13.7036852Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7037458Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7038203Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7038853Z [cluster1]                                     },
2021-07-16T18:15:13.7039599Z [cluster1]                                     "k:{\"name\":\"BROKER_K8S_REMOTENAMESPACE\"}": {
2021-07-16T18:15:13.7040360Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7041060Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7041719Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7042332Z [cluster1]                                     },
2021-07-16T18:15:13.7042991Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_DEBUG\"}": {
2021-07-16T18:15:13.7043804Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7044416Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7045065Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7045679Z [cluster1]                                     },
2021-07-16T18:15:13.7046384Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_FORCEENCAPS\"}": {
2021-07-16T18:15:13.7047115Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7047733Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7048356Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7049057Z [cluster1]                                     },
2021-07-16T18:15:13.7049737Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_IKEPORT\"}": {
2021-07-16T18:15:13.7050431Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7051095Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7051718Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7052508Z [cluster1]                                     },
2021-07-16T18:15:13.7053200Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_NATTPORT\"}": {
2021-07-16T18:15:13.7053891Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7054505Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7055128Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7055737Z [cluster1]                                     },
2021-07-16T18:15:13.7056489Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_PREFERREDSERVER\"}": {
2021-07-16T18:15:13.7057232Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7057846Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7058460Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7059061Z [cluster1]                                     },
2021-07-16T18:15:13.7059730Z [cluster1]                                     "k:{\"name\":\"CE_IPSEC_PSK\"}": {
2021-07-16T18:15:13.7060384Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7061001Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7061643Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7062237Z [cluster1]                                     },
2021-07-16T18:15:13.7062892Z [cluster1]                                     "k:{\"name\":\"NODE_NAME\"}": {
2021-07-16T18:15:13.7063547Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7064162Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7064825Z [cluster1]                                         "f:valueFrom": {
2021-07-16T18:15:13.7065471Z [cluster1]                                             ".": {},
2021-07-16T18:15:13.7066122Z [cluster1]                                             "f:fieldRef": {
2021-07-16T18:15:13.7066778Z [cluster1]                                                 ".": {},
2021-07-16T18:15:13.7067438Z [cluster1]                                                 "f:apiVersion": {},
2021-07-16T18:15:13.7068163Z [cluster1]                                                 "f:fieldPath": {}
2021-07-16T18:15:13.7068811Z [cluster1]                                             }
2021-07-16T18:15:13.7069508Z [cluster1]                                         }
2021-07-16T18:15:13.7070114Z [cluster1]                                     },
2021-07-16T18:15:13.7070764Z [cluster1]                                     "k:{\"name\":\"POD_NAME\"}": {
2021-07-16T18:15:13.7071419Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7072004Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7072670Z [cluster1]                                         "f:valueFrom": {
2021-07-16T18:15:13.7073331Z [cluster1]                                             ".": {},
2021-07-16T18:15:13.7073968Z [cluster1]                                             "f:fieldRef": {
2021-07-16T18:15:13.7074620Z [cluster1]                                                 ".": {},
2021-07-16T18:15:13.7075288Z [cluster1]                                                 "f:apiVersion": {},
2021-07-16T18:15:13.7076003Z [cluster1]                                                 "f:fieldPath": {}
2021-07-16T18:15:13.7076666Z [cluster1]                                             }
2021-07-16T18:15:13.7116760Z [cluster1]                                         }
2021-07-16T18:15:13.7117976Z [cluster1]                                     },
2021-07-16T18:15:13.7118721Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_BROKER\"}": {
2021-07-16T18:15:13.7119542Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7120155Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7121007Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7121618Z [cluster1]                                     },
2021-07-16T18:15:13.7122346Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_CABLEDRIVER\"}": {
2021-07-16T18:15:13.7123099Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7123819Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7124478Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7125084Z [cluster1]                                     },
2021-07-16T18:15:13.7125803Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_CLUSTERCIDR\"}": {
2021-07-16T18:15:13.7126543Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7127151Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7127860Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7128483Z [cluster1]                                     },
2021-07-16T18:15:13.7129186Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_CLUSTERID\"}": {
2021-07-16T18:15:13.7129913Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7130532Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7131155Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7131768Z [cluster1]                                     },
2021-07-16T18:15:13.7132484Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_COLORCODES\"}": {
2021-07-16T18:15:13.7133219Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7133838Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7134462Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7135070Z [cluster1]                                     },
2021-07-16T18:15:13.7135767Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_DEBUG\"}": {
2021-07-16T18:15:13.7136456Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7137072Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7137700Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7138311Z [cluster1]                                     },
2021-07-16T18:15:13.7139140Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_GLOBALCIDR\"}": {
2021-07-16T18:15:13.7139888Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7140504Z [cluster1]                                         "f:name": {}
2021-07-16T18:15:13.7141112Z [cluster1]                                     },
2021-07-16T18:15:13.7141944Z [cluster1]                                     "k:{\"name\":\"SUBMARINER_HEALTHCHECKENABLED\"}": {
2021-07-16T18:15:13.7142775Z [cluster1]                                         ".": {},
2021-07-16T18:15:13.7143374Z [cluster1]                                         "f:name": {},
2021-07-16T18:15:13.7144006Z [cluster1]                                         "f:value": {}
2021-07-16T18:15:13.7144647Z [cluster1]                                     },
2021-07-16T18:15:13.7145551Z [cluster1]