Skip to content

Instantly share code, notes, and snippets.

@anfredette

anfredette/make-deploy.txt

Created February 16, 2022 17:38
Show Gist options
  • Save anfredette/d8be2639ba2ec8dc8e1359d0474401e3 to your computer and use it in GitHub Desktop.
Save anfredette/d8be2639ba2ec8dc8e1359d0474401e3 to your computer and use it in GitHub Desktop.
Download ZIP
make-deploy.txt
Raw
make-deploy.txt
jordan:~/go/src/github.com/submariner-io/submariner-operator$ make deploy using=lighthouse DEPLOY_ARGS='--globalnet --cable_driver vxlan'
docker network create -d bridge kind
Error response from daemon: network with name kind already exists
make: [Makefile.dapper:42: deploy] Error 1 (ignored)
./.dapper -- make --debug=b deploy
Sending build context to Docker daemon 498.2MB
Step 1/8 : ARG BASE_BRANCH
Step 2/8 : FROM quay.io/submariner/shipyard-dapper-base:${BASE_BRANCH}
---> ee8080f4e530
Step 3/8 : ARG PROJECT
---> Using cache
---> c423414c6c8a
Step 4/8 : ENV DAPPER_ENV="QUAY_USERNAME QUAY_PASSWORD CLUSTERS_ARGS DEPLOY_ARGS CLEANUP_ARGS E2E_ARGS RELEASE_ARGS MAKEFLAGS FOCUS SKIP PLUGIN E2E_TESTDIR GITHUB_USER GITHUB_TOKEN" DAPPER_SOURCE=/go/src/github.com/submariner-io/${PROJECT} DAPPER_DOCKER_SOCKET=true
---> Using cache
---> 186ac4d56308
Step 5/8 : ENV DAPPER_OUTPUT=${DAPPER_SOURCE}/output
---> Using cache
---> 0f0fe974c772
Step 6/8 : WORKDIR ${DAPPER_SOURCE}
---> Using cache
---> 327e2e6efc7c
Step 7/8 : ENTRYPOINT ["/opt/shipyard/scripts/entry"]
---> Using cache
---> 6dcc8b26fa3a
Step 8/8 : CMD ["sh"]
---> Using cache
---> 4b5d2ce00c22
Successfully built 4b5d2ce00c22
Successfully tagged submariner-operator:devel
[15:15:55.051] [submariner-operator]$ trap chown -R 1000:1000 . exit
[15:15:55.052] [submariner-operator]$ mkdir -p bin dist output
[15:15:55.054] [submariner-operator]$ make --debug=b deploy
GNU Make 4.3
Built for x86_64-redhat-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Makefile:143: warning: overriding recipe for target 'e2e'
/opt/shipyard/Makefile.inc:139: warning: ignoring old recipe for target 'e2e'
Updating makefiles....
Updating goal targets....
File 'deploy' does not exist.
File 'clusters' does not exist.
Must remake target 'clusters'.
/opt/shipyard/scripts/clusters.sh --settings /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
Running with: k8s_version=1.20.7@sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9, olm_version=v0.18.3, olm=false, globalnet=false, prometheus=false, registry_inmemory=true, settings=/go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml, timeout=5m
[15:15:55.711] [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
[15:15:55.713] [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
[15:15:55.714] [submariner-operator]$ script_name=utils
[15:15:55.715] [submariner-operator]$ exec_name=clusters.sh
[15:15:55.717] [submariner-operator]$ rm -rf /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs
[15:15:55.719] [submariner-operator]$ mkdir -p /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs
[15:15:55.721] [submariner-operator]$ load_settings
[15:15:55.722] [submariner-operator]$ load_settings
[15:15:55.723] [submariner-operator]$ load_settings_from_yaml
[15:15:55.725] [submariner-operator]$ load_settings_from_yaml
[15:15:55.726] [submariner-operator]$ declare -ga clusters
[15:15:55.727] [submariner-operator]$ declare -gA cluster_cni cluster_nodes cluster_subm
[15:15:55.728] [submariner-operator]$ _yq .clusters | keys | .[]
[15:15:55.729] [submariner-operator]$ _yq .clusters | keys | .[]
[15:15:55.731] [submariner-operator]$ output=$(yq e "$*" "${settings}")
[15:15:55.732] [submariner-operator]$ yq e .clusters | keys | .[] /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:55.779] [submariner-operator]$ broker=$(_yq ".clusters[] | select(. | has(\"broker\")) | path | .[-1] // \"${clusters[0]}\"")
[15:15:55.781] [submariner-operator]$ _yq .clusters[] | select(. | has("broker")) | path | .[-1] // "cluster1"
[15:15:55.782] [submariner-operator]$ _yq .clusters[] | select(. | has("broker")) | path | .[-1] // "cluster1"
[15:15:55.783] [submariner-operator]$ output=$(yq e "$*" "${settings}")
[15:15:55.785] [submariner-operator]$ yq e .clusters[] | select(. | has("broker")) | path | .[-1] // "cluster1" /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:55.831] [submariner-operator]$ [cluster1] cluster_cni["${cluster}"]=$(_yq ".clusters.${cluster}.cni // .cni")
[15:15:55.833] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.cni // .cni
[15:15:55.834] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.cni // .cni
[15:15:55.835] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:15:55.837] [submariner-operator]$ [cluster1] yq e .clusters.cluster1.cni // .cni /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:55.884] [submariner-operator]$ [cluster1] cluster_nodes["${cluster}"]=$(_yq ".clusters.${cluster}.nodes // .nodes")
[15:15:55.885] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.nodes // .nodes
[15:15:55.887] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.nodes // .nodes
[15:15:55.888] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:15:55.889] [submariner-operator]$ [cluster1] yq e .clusters.cluster1.nodes // .nodes /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:55.935] [submariner-operator]$ [cluster1] cluster_subm["${cluster}"]=$(_yq ".clusters.${cluster}.submariner")
[15:15:55.937] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.submariner
[15:15:55.938] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.submariner
[15:15:55.939] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:15:55.941] [submariner-operator]$ [cluster1] yq e .clusters.cluster1.submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:55.987] [submariner-operator]$ [cluster1] cluster_subm["${cluster}"]=$(_yq ".submariner")
[15:15:55.989] [submariner-operator]$ [cluster1] _yq .submariner
[15:15:55.990] [submariner-operator]$ [cluster1] _yq .submariner
[15:15:55.991] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:15:55.993] [submariner-operator]$ [cluster1] yq e .submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:56.039] [submariner-operator]$ [cluster2] cluster_cni["${cluster}"]=$(_yq ".clusters.${cluster}.cni // .cni")
[15:15:56.041] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.cni // .cni
[15:15:56.042] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.cni // .cni
[15:15:56.043] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:15:56.045] [submariner-operator]$ [cluster2] yq e .clusters.cluster2.cni // .cni /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:56.091] [submariner-operator]$ [cluster2] cluster_nodes["${cluster}"]=$(_yq ".clusters.${cluster}.nodes // .nodes")
[15:15:56.093] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.nodes // .nodes
[15:15:56.094] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.nodes // .nodes
[15:15:56.095] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:15:56.097] [submariner-operator]$ [cluster2] yq e .clusters.cluster2.nodes // .nodes /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:56.143] [submariner-operator]$ [cluster2] cluster_subm["${cluster}"]=$(_yq ".clusters.${cluster}.submariner")
[15:15:56.145] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.submariner
[15:15:56.146] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.submariner
[15:15:56.148] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:15:56.149] [submariner-operator]$ [cluster2] yq e .clusters.cluster2.submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:56.195] [submariner-operator]$ [cluster2] cluster_subm["${cluster}"]=$(_yq ".submariner")
[15:15:56.196] [submariner-operator]$ [cluster2] _yq .submariner
[15:15:56.198] [submariner-operator]$ [cluster2] _yq .submariner
[15:15:56.199] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:15:56.201] [submariner-operator]$ [cluster2] yq e .submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:15:56.247] [submariner-operator]$ [cluster2] cat <<EOM
Cluster settings::
broker - ${broker@Q}
clusters - ${clusters[*]@Q}
cni - $(typeset -p cluster_cni | cut -f 2- -d=)
nodes per cluster - $(typeset -p cluster_nodes | cut -f 2- -d=)
install submariner - $(typeset -p cluster_subm | cut -f 2- -d=)
EOM
[15:15:56.249] [submariner-operator]$ [cluster2] typeset -p cluster_cni
[15:15:56.250] [submariner-operator]$ [cluster2] cut -f 2- -d=
[15:15:56.252] [submariner-operator]$ [cluster2] typeset -p cluster_nodes
[15:15:56.254] [submariner-operator]$ [cluster2] cut -f 2- -d=
[15:15:56.256] [submariner-operator]$ [cluster2] typeset -p cluster_subm
[15:15:56.258] [submariner-operator]$ [cluster2] cut -f 2- -d=
Cluster settings::
broker - 'cluster1'
clusters - 'cluster1' 'cluster2'
cni - ([cluster2]="weave" [cluster1]="weave" )
nodes per cluster - ([cluster2]="control-plane worker" [cluster1]="control-plane worker" )
install submariner - ([cluster2]="true" [cluster1]="true" )
[15:15:56.260] [submariner-operator]$ [cluster2] run_local_registry
[15:15:56.261] [submariner-operator]$ [cluster2] run_local_registry
[15:15:56.263] [submariner-operator]$ [cluster2] registry_running
[15:15:56.264] [submariner-operator]$ [cluster2] registry_running
[15:15:56.265] [submariner-operator]$ [cluster2] docker ps --filter name=^/?kind-registry$
[15:15:56.266] [submariner-operator]$ [cluster2] grep kind-registry
95ae4893edf5 registry:2 "/entrypoint.sh /etc…" 14 minutes ago Up 14 minutes 127.0.0.1:5000->5000/tcp kind-registry
[15:15:56.456] [submariner-operator]$ [cluster2] return 0
Local registry kind-registry already running.
[15:15:56.457] [submariner-operator]$ [cluster2] declare_cidrs
[15:15:56.458] [submariner-operator]$ [cluster2] declare_cidrs
[15:15:56.460] [submariner-operator]$ [cluster2] declare -gA cluster_CIDRs service_CIDRs global_CIDRs
[15:15:56.461] [submariner-operator]$ [cluster2] i=1
[15:15:56.462] [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
[15:15:56.463] [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
[15:15:56.465] [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.1.0.0/16
[15:15:56.466] [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.1.0.0/16
[15:15:56.467] [submariner-operator]$ [cluster1] i=$(("$i"+1))
[15:15:56.469] [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
[15:15:56.470] [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
[15:15:56.471] [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.2.0.0/16
[15:15:56.473] [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.2.0.0/16
[15:15:56.474] [submariner-operator]$ [cluster2] i=$(("$i"+1))
[15:15:56.475] [submariner-operator]$ [cluster2] wait 536
[15:15:56.475] [submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster
[15:15:56.477] [submariner-operator]$ [cluster2] run_all_clusters with_retries 3 create_kind_cluster
[15:15:56.478] [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 with_retries 3 create_kind_cluster
[15:15:56.479] [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 with_retries 3 create_kind_cluster
[15:15:56.481] [submariner-operator]$ [cluster2] declare -A pids
[15:15:56.482] [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
[15:15:56.484] [submariner-operator]$ [cluster1] set -o pipefail
[15:15:56.484] [submariner-operator]$ [cluster1] pids[cluster1]=559
[15:15:56.485] [submariner-operator]$ [cluster1] with_retries 3 create_kind_cluster
[15:15:56.486] [submariner-operator]$ [cluster2] set -o pipefail
[15:15:56.486] [submariner-operator]$ [cluster2] pids[cluster2]=567
[15:15:56.487] [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
[15:15:56.487] [submariner-operator]$ [cluster2] with_retries 3 create_kind_cluster
[15:15:56.487] [submariner-operator]$ [cluster2] wait 567
[15:15:56.487] [submariner-operator]$ [cluster1] with_retries
[15:15:56.489] [submariner-operator]$ [cluster1] iteration=1
[15:15:56.489] [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
[15:15:56.489] [submariner-operator]$ [cluster2] with_retries
[15:15:56.490] [submariner-operator]$ [cluster2] iteration=1
[15:15:56.491] [submariner-operator]$ [cluster1] wait 607
[15:15:56.491] [submariner-operator]$ [cluster1] create_kind_cluster
[15:15:56.493] [submariner-operator]$ [cluster1] create_kind_cluster
[15:15:56.493] [submariner-operator]$ [cluster2] wait 617
[15:15:56.493] [submariner-operator]$ [cluster2] create_kind_cluster
[15:15:56.494] [submariner-operator]$ [cluster1] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:56.494] [submariner-operator]$ [cluster2] create_kind_cluster
[15:15:56.495] [submariner-operator]$ [cluster1] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:56.495] [submariner-operator]$ [cluster2] export KUBECONFIG=/go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:56.497] [submariner-operator]$ [cluster2] rm -f /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:56.497] [submariner-operator]$ [cluster1] kind get clusters
[15:15:56.498] [submariner-operator]$ [cluster1] grep -q ^cluster1$
[15:15:56.499] [submariner-operator]$ [cluster2] kind get clusters
[15:15:56.500] [submariner-operator]$ [cluster2] grep -q ^cluster2$
[cluster2] KIND cluster already exists, skipping its creation...
[cluster1] KIND cluster already exists, skipping its creation...
[15:15:56.921] [submariner-operator]$ [cluster2] kind export kubeconfig --name=cluster2
[15:15:56.921] [submariner-operator]$ [cluster1] kind export kubeconfig --name=cluster1
[cluster1] Set kubectl context to "kind-cluster1"
[cluster2] Set kubectl context to "kind-cluster2"
[15:15:59.290] [submariner-operator]$ [cluster1] kind_fixup_config
[15:15:59.291] [submariner-operator]$ [cluster2] kind_fixup_config
[15:15:59.292] [submariner-operator]$ [cluster1] kind_fixup_config
[15:15:59.292] [submariner-operator]$ [cluster2] kind_fixup_config
[15:15:59.293] [submariner-operator]$ [cluster1] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster1-control-plane
[15:15:59.294] [submariner-operator]$ [cluster2] docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}} cluster2-control-plane
[15:15:59.295] [submariner-operator]$ [cluster1] head -n 1
[15:15:59.295] [submariner-operator]$ [cluster2] head -n 1
[15:15:59.486] [submariner-operator]$ [cluster1] sed -i -- s/server: .*/server: https:\/\/172.18.0.7:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:59.486] [submariner-operator]$ [cluster2] sed -i -- s/server: .*/server: https:\/\/172.18.0.6:6443/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:59.488] [submariner-operator]$ [cluster2] sed -i -- s/user: kind-.*/user: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:59.488] [submariner-operator]$ [cluster1] sed -i -- s/user: kind-.*/user: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:59.491] [submariner-operator]$ [cluster2] sed -i -- s/name: kind-.*/name: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:59.491] [submariner-operator]$ [cluster1] sed -i -- s/name: kind-.*/name: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:59.493] [submariner-operator]$ [cluster2] sed -i -- s/cluster: kind-.*/cluster: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:59.493] [submariner-operator]$ [cluster1] sed -i -- s/cluster: kind-.*/cluster: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:59.495] [submariner-operator]$ [cluster2] sed -i -- s/current-context: .*/current-context: cluster2/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:59.495] [submariner-operator]$ [cluster1] sed -i -- s/current-context: .*/current-context: cluster1/g /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:59.497] [submariner-operator]$ [cluster2] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2
[15:15:59.497] [submariner-operator]$ [cluster1] chmod a+r /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1
[15:15:59.499] [submariner-operator]$ [cluster2] return
[15:15:59.499] [submariner-operator]$ [cluster1] return
[15:15:59.501] [submariner-operator]$ [cluster2] return 0
[15:15:59.501] [submariner-operator]$ [cluster1] return 0
[15:15:59.503] [submariner-operator]$ [cluster2] wait 559
[15:15:59.504] [submariner-operator]$ [cluster2] print_clusters_message
[15:15:59.506] [submariner-operator]$ [cluster2] print_clusters_message
[15:15:59.507] [submariner-operator]$ [cluster2] cat <<EOM
Your virtual cluster(s) are deployed and working properly and can be accessed with:
export KUBECONFIG=\$(find \$(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
$ kubectl config use-context cluster1 # or cluster2, cluster3..
To clean everthing up, just run: make clean-clusters
EOM
Your virtual cluster(s) are deployed and working properly and can be accessed with:
export KUBECONFIG=$(find $(git rev-parse --show-toplevel)/output/kubeconfigs/ -type f -printf %p:)
$ kubectl config use-context cluster1 # or cluster2, cluster3..
To clean everthing up, just run: make clean-clusters
Successfully remade target file 'clusters'.
File 'preload-images' does not exist.
File 'images' does not exist.
Prerequisite 'bin/submariner-operator' is newer than target 'package/Dockerfile.submariner-operator'.
File 'FORCE_IMAGE' does not exist.
Must remake target 'FORCE_IMAGE'.
Successfully remade target file 'FORCE_IMAGE'.
Prerequisite 'FORCE_IMAGE' of target 'package/.image.submariner-operator' does not exist.
Must remake target 'package/.image.submariner-operator'.
/opt/shipyard/scripts/build_image.sh -i submariner-operator -f package/Dockerfile.submariner-operator -h package/.image.submariner-operator
flags:WARN flag short name (h) already defined
[15:15:59.770] [submariner-operator]$ set -e
[15:15:59.772] [submariner-operator]$ cache_image=quay.io/submariner/submariner-operator:devel
[15:15:59.773] [submariner-operator]$ cache_flag=
[15:15:59.774] [submariner-operator]$ cache_flag=--cache-from quay.io/submariner/submariner-operator:devel
[15:15:59.775] [submariner-operator]$ docker image ls -q quay.io/submariner/submariner-operator:devel
[15:15:59.979] [submariner-operator]$ grep FROM package/Dockerfile.submariner-operator
[15:15:59.980] [submariner-operator]$ cut -f2 -d
[15:15:59.982] [submariner-operator]$ grep -v scratch
[15:15:59.984] [submariner-operator]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel
[15:16:00.530] [submariner-operator]$ docker buildx use buildx_builder
error: failed to find instance "buildx_builder": open /root/.docker/buildx/instances/buildx_builder: no such file or directory
[15:16:01.070] [submariner-operator]$ docker buildx create --name buildx_builder --use
buildx_builder
[15:16:01.670] [submariner-operator]$ docker buildx build --load -t quay.io/submariner/submariner-operator:dev --cache-from quay.io/submariner/submariner-operator:devel -f package/Dockerfile.submariner-operator --iidfile package/.image.submariner-operator --platform linux/amd64 --build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel .
[+] Building 0.4s (8/8) FINISHED
=> [internal] load build definition from Dockerfile.submariner-operator 0.0s
=> => transferring dockerfile: 230B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> importing cache manifest from quay.io/submariner/submariner-operator:devel 0.1s
=> [internal] load build context 0.1s
=> => transferring context: 10.08MB 0.1s
=> CACHED [1/1] COPY bin/submariner-operator /submariner-operator 0.0s
=> exporting to oci image format 0.1s
=> => exporting layers 0.0s
=> => exporting manifest sha256:263cbc5c1ae1ae808cf49c8701839ddf35f4d05764a2908b5a17ef3f57916cb7 0.0s
=> => exporting config sha256:957c8b4ef55a2045cfd11b15e76d9cd0cbb52a6c80ec738bd2363f9c67d3cb48 0.0s
=> => sending tarball 0.1s
=> importing to docker 0.0s
=> exporting cache 0.0s
=> => preparing build cache for export 0.0s
[15:16:02.831] [submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev quay.io/submariner/submariner-operator:devel
Successfully remade target file 'package/.image.submariner-operator'.
File 'packagemanifests' does not exist.
File 'kustomization' does not exist.
File 'is-semantic-version' does not exist.
Must remake target 'is-semantic-version'.
Successfully remade target file 'is-semantic-version'.
File 'manifests' does not exist.
Must remake target 'manifests'.
/go/src/github.com/submariner-io/submariner-operator/bin/controller-gen "crd:crdVersions=v1,trivialVersions=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
Successfully remade target file 'manifests'.
Must remake target 'kustomization'.
/go/src/github.com/submariner-io/submariner-operator/bin/operator-sdk generate kustomize manifests -q
(cd config/manifests && /go/src/github.com/submariner-io/submariner-operator/bin/kustomize edit set image controller=quay.io/submariner/submariner-operator:devel-82bf5d4 && \
/go/src/github.com/submariner-io/submariner-operator/bin/kustomize edit set image repo=quay.io/submariner)
sed -e 's/${VERSION}/0.12.0/g' config/bundle/kustomization.template.yaml > config/bundle/kustomization.yaml
sed -e 's/${REPLACES_OP}/remove/g' -e 's/${FROM_VERSION}/0.0.0/g' \
config/bundle/patches/submariner.csv.template.yaml > config/bundle/patches/submariner.csv.config.yaml
(cd config/bundle && \
/go/src/github.com/submariner-io/submariner-operator/bin/kustomize edit add annotation createdAt:"2022-02-16 15:16:09" -f)
Successfully remade target file 'kustomization'.
Must remake target 'packagemanifests'.
(/go/src/github.com/submariner-io/submariner-operator/bin/kustomize build /go/src/github.com/submariner-io/submariner-operator/config/manifests \
| /go/src/github.com/submariner-io/submariner-operator/bin/operator-sdk generate packagemanifests -q --version 0.12.0 --channel=alpha-0.12 --default-channel)
(cd config/bundle && /go/src/github.com/submariner-io/submariner-operator/bin/kustomize edit add resource ../../packagemanifests/0.12.0/submariner.clusterserviceversion.yaml)
/go/src/github.com/submariner-io/submariner-operator/bin/kustomize build config/bundle/ --load_restrictor=LoadRestrictionsNone --output packagemanifests/0.12.0/submariner.clusterserviceversion.yaml
sed -i -e 's/$(SHORT_VERSION)/0.12/g' packagemanifests/0.12.0/submariner.clusterserviceversion.yaml
sed -i -e 's/$(VERSION)/devel-82bf5d4/g' packagemanifests/0.12.0/submariner.clusterserviceversion.yaml
mv packagemanifests/0.12.0/submariner.clusterserviceversion.yaml packagemanifests/0.12.0/submariner.v0.12.0.clusterserviceversion.yaml
Successfully remade target file 'packagemanifests'.
Prerequisite 'packagemanifests' of target 'package/Dockerfile.submariner-operator-index' does not exist.
Must remake target 'package/Dockerfile.submariner-operator-index'.
Successfully remade target file 'package/Dockerfile.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner.v0.12.0.clusterserviceversion.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-lighthouse-coredns_v1_serviceaccount.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-lighthouse-coredns_rbac.authorization.k8s.io_v1_clusterrole.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-gateway_rbac.authorization.k8s.io_v1_role.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-lighthouse-agent_rbac.authorization.k8s.io_v1_clusterrolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-gateway_rbac.authorization.k8s.io_v1_clusterrolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-globalnet_rbac.authorization.k8s.io_v1_rolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-networkplugin-syncer_rbac.authorization.k8s.io_v1_clusterrole.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-routeagent_rbac.authorization.k8s.io_v1_clusterrolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner.io_brokers.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-networkplugin-syncer_v1_serviceaccount.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-gateway_rbac.authorization.k8s.io_v1_rolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner.io_submariners.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-lighthouse-coredns_rbac.authorization.k8s.io_v1_clusterrolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-gateway_rbac.authorization.k8s.io_v1_clusterrole.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-globalnet_rbac.authorization.k8s.io_v1_role.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-routeagent_v1_serviceaccount.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-operator_v1_serviceaccount.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-lighthouse-agent_v1_serviceaccount.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-routeagent_rbac.authorization.k8s.io_v1_role.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-globalnet_rbac.authorization.k8s.io_v1_clusterrolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-routeagent_rbac.authorization.k8s.io_v1_clusterrole.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-routeagent_rbac.authorization.k8s.io_v1_rolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-globalnet_rbac.authorization.k8s.io_v1_clusterrole.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-lighthouse-agent_rbac.authorization.k8s.io_v1_clusterrole.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-networkplugin-syncer_rbac.authorization.k8s.io_v1_clusterrolebinding.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-gateway_v1_serviceaccount.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/0.12.0/submariner-globalnet_v1_serviceaccount.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'packagemanifests/submariner.package.yaml' is newer than target 'package/.image.submariner-operator-index'.
Prerequisite 'FORCE_IMAGE' of target 'package/.image.submariner-operator-index' does not exist.
Must remake target 'package/.image.submariner-operator-index'.
/opt/shipyard/scripts/build_image.sh -i submariner-operator-index -f package/Dockerfile.submariner-operator-index -h package/.image.submariner-operator-index
flags:WARN flag short name (h) already defined
[15:16:11.715] [submariner-operator]$ set -e
[15:16:11.717] [submariner-operator]$ cache_image=quay.io/submariner/submariner-operator-index:devel
[15:16:11.718] [submariner-operator]$ cache_flag=
[15:16:11.720] [submariner-operator]$ cache_flag=--cache-from quay.io/submariner/submariner-operator-index:devel
[15:16:11.721] [submariner-operator]$ docker image ls -q quay.io/submariner/submariner-operator-index:devel
[15:16:11.923] [submariner-operator]$ grep FROM package/Dockerfile.submariner-operator-index
[15:16:11.924] [submariner-operator]$ cut -f2 -d
[15:16:11.925] [submariner-operator]$ grep -v scratch
[15:16:11.928] [submariner-operator]$ cache_flag+= --cache-from quay.io/operator-framework/upstream-registry-builder
[15:16:11.929] [submariner-operator]$ docker pull quay.io/operator-framework/upstream-registry-builder
Using default tag: latest
latest: Pulling from operator-framework/upstream-registry-builder
Digest: sha256:70043025cb6b3cdce4800140428de4a18726aa93898bedf97fcd371d3240d9ee
Status: Image is up to date for quay.io/operator-framework/upstream-registry-builder:latest
quay.io/operator-framework/upstream-registry-builder:latest
[15:16:12.392] [submariner-operator]$ buildargs_flag=--build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel
[15:16:12.935] [submariner-operator]$ docker buildx use buildx_builder
[15:16:13.484] [submariner-operator]$ docker buildx build --load -t quay.io/submariner/submariner-operator-index:dev --cache-from quay.io/submariner/submariner-operator-index:devel --cache-from quay.io/operator-framework/upstream-registry-builder -f package/Dockerfile.submariner-operator-index --iidfile package/.image.submariner-operator-index --platform linux/amd64 --build-arg BUILDKIT_INLINE_CACHE=1 --build-arg BASE_BRANCH=devel .
[+] Building 2.8s (16/16) FINISHED
=> [internal] load build definition from Dockerfile.submariner-operator-index 0.0s
=> => transferring dockerfile: 637B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for quay.io/operator-framework/upstream-registry-builder:latest 0.1s
=> importing cache manifest from quay.io/submariner/submariner-operator-index:devel 0.3s
=> importing cache manifest from quay.io/operator-framework/upstream-registry-builder 0.2s
=> [internal] load build context 0.1s
=> => transferring context: 653.37kB 0.0s
=> CACHED [builder 1/3] FROM quay.io/operator-framework/upstream-registry-builder@sha256:70043025cb6b3cdce4800140428de4a18726aa93898bedf97fcd371d3240d9ee 0.0s
=> => resolve quay.io/operator-framework/upstream-registry-builder@sha256:70043025cb6b3cdce4800140428de4a18726aa93898bedf97fcd371d3240d9ee 0.0s
=> [builder 2/3] COPY packagemanifests manifests 0.1s
=> [builder 3/3] RUN find manifests -name *.clusterserviceversion.yaml -exec sed -i 's/replaces: submariner.v0.0.0//g' {} + && /bin/initializer -o ./bundles.db 0.4s
=> CACHED [stage-1 1/4] COPY --from=builder /etc/nsswitch.conf /etc/nsswitch.conf 0.0s
=> [stage-1 2/4] COPY --from=builder /bundles.db /bundles.db 0.0s
=> [stage-1 3/4] COPY --from=builder /bin/registry-server /registry-server 0.1s
=> [stage-1 4/4] COPY --from=builder /bin/grpc_health_probe /bin/grpc_health_probe 0.0s
=> exporting to oci image format 0.4s
=> => exporting layers 0.0s
=> => exporting manifest sha256:8ba4f13d9b3b4357248e75d6dcb38bcce3ed3ece3a33852dc00288fa8211c6b1 0.0s
=> => exporting config sha256:88f916cc70026c72515c8fe7a4bd3ecd0a9faed030ee32112831bc1f5cf6e2c7 0.0s
=> => sending tarball 0.3s
=> importing to docker 0.3s
=> exporting cache 0.0s
=> => preparing build cache for export 0.0s
[15:16:16.943] [submariner-operator]$ docker tag quay.io/submariner/submariner-operator-index:dev quay.io/submariner/submariner-operator-index:devel
Successfully remade target file 'package/.image.submariner-operator-index'.
File 'build' does not exist.
Must remake target 'build'.
Successfully remade target file 'build'.
Must remake target 'images'.
Successfully remade target file 'images'.
Must remake target 'preload-images'.
set -e;\
. /opt/shipyard/scripts/lib/deploy_funcs;\
. /opt/shipyard/scripts/lib/debug_functions;\
for image in submariner-operator submariner-operator-index submariner-gateway submariner-globalnet submariner-route-agent lighthouse-agent lighthouse-coredns; do\
import_image quay.io/submariner/${image};\
done
[15:16:17.161] [submariner-operator]$ import_image quay.io/submariner/submariner-operator
[15:16:17.162] [submariner-operator]$ import_image quay.io/submariner/submariner-operator
[15:16:17.164] [submariner-operator]$ docker tag quay.io/submariner/submariner-operator:dev localhost:5000/submariner-operator:local
[15:16:17.356] [submariner-operator]$ docker push localhost:5000/submariner-operator:local
The push refers to repository [localhost:5000/submariner-operator]
2e151bb90cb3: Layer already exists
local: digest: sha256:e27590ee9a73320a00c2c218c0e1ae48b4a6c7134897bf9c420bc4f1bde75724 size: 528
[15:16:17.554] [submariner-operator]$ import_image quay.io/submariner/submariner-operator-index
[15:16:17.556] [submariner-operator]$ import_image quay.io/submariner/submariner-operator-index
[15:16:17.558] [submariner-operator]$ docker tag quay.io/submariner/submariner-operator-index:dev localhost:5000/submariner-operator-index:local
[15:16:17.757] [submariner-operator]$ docker push localhost:5000/submariner-operator-index:local
The push refers to repository [localhost:5000/submariner-operator-index]
a4863cbe7937: Pushed
4ba58a0ff872: Layer already exists
9791ba34055c: Pushed
577c9bc3375c: Layer already exists
local: digest: sha256:87f60d8efdb9183b6371ffb85af4b5a626af2bf67d56c3394d4eb365dd062e02 size: 1158
[15:16:18.447] [submariner-operator]$ import_image quay.io/submariner/submariner-gateway
[15:16:18.448] [submariner-operator]$ import_image quay.io/submariner/submariner-gateway
[15:16:18.450] [submariner-operator]$ docker tag quay.io/submariner/submariner-gateway:dev localhost:5000/submariner-gateway:local
[15:16:18.648] [submariner-operator]$ docker push localhost:5000/submariner-gateway:local
The push refers to repository [localhost:5000/submariner-gateway]
388424b7942f: Layer already exists
08111fff0ede: Layer already exists
cfee32a0e20c: Layer already exists
4f21bd15cf0e: Layer already exists
local: digest: sha256:0bde7086869885e4967022d17ca3eb1f08921e27aafc5b642ac908559386c0da size: 1159
[15:16:18.853] [submariner-operator]$ import_image quay.io/submariner/submariner-globalnet
[15:16:18.855] [submariner-operator]$ import_image quay.io/submariner/submariner-globalnet
[15:16:18.856] [submariner-operator]$ docker tag quay.io/submariner/submariner-globalnet:dev localhost:5000/submariner-globalnet:local
[15:16:19.057] [submariner-operator]$ docker push localhost:5000/submariner-globalnet:local
The push refers to repository [localhost:5000/submariner-globalnet]
3f547651d2bd: Pushed
474b8677ec5e: Pushed
b6b7b602321e: Pushed
3009d96daa95: Layer already exists
cfee32a0e20c: Layer already exists
4f21bd15cf0e: Layer already exists
local: digest: sha256:9a46cde1f2af234f1bb022fa35e0f2f76e3d06b034ff0bc0971965a882ac2145 size: 1574
[15:16:19.578] [submariner-operator]$ import_image quay.io/submariner/submariner-route-agent
[15:16:19.580] [submariner-operator]$ import_image quay.io/submariner/submariner-route-agent
[15:16:19.581] [submariner-operator]$ docker tag quay.io/submariner/submariner-route-agent:dev localhost:5000/submariner-route-agent:local
[15:16:19.778] [submariner-operator]$ docker push localhost:5000/submariner-route-agent:local
The push refers to repository [localhost:5000/submariner-route-agent]
c910a0876fb0: Layer already exists
b4df79ea567f: Layer already exists
7c45d471bc4c: Layer already exists
4675c0d24297: Layer already exists
cfee32a0e20c: Layer already exists
4f21bd15cf0e: Layer already exists
local: digest: sha256:e5796dfbe7526012e871aee3a7c723523b1aae0b29f477c2fbc2daa5c6491d0c size: 1575
[15:16:19.980] [submariner-operator]$ import_image quay.io/submariner/lighthouse-agent
[15:16:19.981] [submariner-operator]$ import_image quay.io/submariner/lighthouse-agent
[15:16:19.983] [submariner-operator]$ docker tag quay.io/submariner/lighthouse-agent:dev localhost:5000/lighthouse-agent:local
[15:16:20.176] [submariner-operator]$ docker push localhost:5000/lighthouse-agent:local
The push refers to repository [localhost:5000/lighthouse-agent]
da8ee070c959: Layer already exists
ca89753c22d1: Layer already exists
local: digest: sha256:3477b732d49231b4cf24b5cac628682c9174596737710c57983561df03533534 size: 735
[15:16:20.387] [submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns
[15:16:20.388] [submariner-operator]$ import_image quay.io/submariner/lighthouse-coredns
[15:16:20.390] [submariner-operator]$ docker tag quay.io/submariner/lighthouse-coredns:dev localhost:5000/lighthouse-coredns:local
[15:16:20.586] [submariner-operator]$ docker push localhost:5000/lighthouse-coredns:local
The push refers to repository [localhost:5000/lighthouse-coredns]
0e7183faf640: Layer already exists
b583fb46a3ac: Layer already exists
local: digest: sha256:06f1e34afec3aa015ff0b7b7ffdf5ea55eb5eedd1c8c3f08cbe66062d338c4fb size: 739
Successfully remade target file 'preload-images'.
Must remake target 'deploy'.
/opt/shipyard/scripts/deploy.sh --globalnet --cable_driver vxlan --service_discovery --settings /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml --deploytool_broker_args '--components service-discovery,connectivity'
Running with: globalnet='true', deploytool='operator', deploytool_broker_args='--components service-discovery,connectivity', deploytool_submariner_args='', settings='/go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml', timeout=5m, image_tag=local, cable_driver=vxlan, service_discovery=true
[15:16:20.845] [submariner-operator]$ source /opt/shipyard/scripts/lib/utils
[15:16:20.846] [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
[15:16:20.847] [submariner-operator]$ script_name=utils
[15:16:20.849] [submariner-operator]$ exec_name=deploy.sh
[15:16:20.850] [submariner-operator]$ source /opt/shipyard/scripts/lib/deploy_funcs
[15:16:20.852] [submariner-operator]$ . /opt/shipyard/scripts/lib/source_only
[15:16:20.853] [submariner-operator]$ script_name=deploy_funcs
[15:16:20.854] [submariner-operator]$ exec_name=deploy.sh
[15:16:20.857] [submariner-operator]$ dd if=/dev/urandom count=64 bs=8
64+0 records in
64+0 records out
512 bytes copied, 5.8368e-05 s, 8.8 MB/s
[15:16:20.858] [submariner-operator]$ LC_CTYPE=C tr -dc a-zA-Z0-9
[15:16:20.859] [submariner-operator]$ fold -w 64
[15:16:20.861] [submariner-operator]$ head -n 1
[15:16:20.863] [submariner-operator]$ load_settings
[15:16:20.864] [submariner-operator]$ load_settings
[15:16:20.865] [submariner-operator]$ load_settings_from_yaml
[15:16:20.867] [submariner-operator]$ load_settings_from_yaml
[15:16:20.868] [submariner-operator]$ declare -ga clusters
[15:16:20.869] [submariner-operator]$ declare -gA cluster_cni cluster_nodes cluster_subm
[15:16:20.870] [submariner-operator]$ _yq .clusters | keys | .[]
[15:16:20.872] [submariner-operator]$ _yq .clusters | keys | .[]
[15:16:20.873] [submariner-operator]$ output=$(yq e "$*" "${settings}")
[15:16:20.875] [submariner-operator]$ yq e .clusters | keys | .[] /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:20.921] [submariner-operator]$ broker=$(_yq ".clusters[] | select(. | has(\"broker\")) | path | .[-1] // \"${clusters[0]}\"")
[15:16:20.923] [submariner-operator]$ _yq .clusters[] | select(. | has("broker")) | path | .[-1] // "cluster1"
[15:16:20.924] [submariner-operator]$ _yq .clusters[] | select(. | has("broker")) | path | .[-1] // "cluster1"
[15:16:20.926] [submariner-operator]$ output=$(yq e "$*" "${settings}")
[15:16:20.927] [submariner-operator]$ yq e .clusters[] | select(. | has("broker")) | path | .[-1] // "cluster1" /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:20.973] [submariner-operator]$ [cluster1] cluster_cni["${cluster}"]=$(_yq ".clusters.${cluster}.cni // .cni")
[15:16:20.975] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.cni // .cni
[15:16:20.976] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.cni // .cni
[15:16:20.978] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:16:20.979] [submariner-operator]$ [cluster1] yq e .clusters.cluster1.cni // .cni /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.027] [submariner-operator]$ [cluster1] cluster_nodes["${cluster}"]=$(_yq ".clusters.${cluster}.nodes // .nodes")
[15:16:21.029] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.nodes // .nodes
[15:16:21.030] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.nodes // .nodes
[15:16:21.031] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:16:21.033] [submariner-operator]$ [cluster1] yq e .clusters.cluster1.nodes // .nodes /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.079] [submariner-operator]$ [cluster1] cluster_subm["${cluster}"]=$(_yq ".clusters.${cluster}.submariner")
[15:16:21.080] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.submariner
[15:16:21.082] [submariner-operator]$ [cluster1] _yq .clusters.cluster1.submariner
[15:16:21.083] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:16:21.085] [submariner-operator]$ [cluster1] yq e .clusters.cluster1.submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.131] [submariner-operator]$ [cluster1] cluster_subm["${cluster}"]=$(_yq ".submariner")
[15:16:21.132] [submariner-operator]$ [cluster1] _yq .submariner
[15:16:21.134] [submariner-operator]$ [cluster1] _yq .submariner
[15:16:21.135] [submariner-operator]$ [cluster1] output=$(yq e "$*" "${settings}")
[15:16:21.137] [submariner-operator]$ [cluster1] yq e .submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.183] [submariner-operator]$ [cluster2] cluster_cni["${cluster}"]=$(_yq ".clusters.${cluster}.cni // .cni")
[15:16:21.185] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.cni // .cni
[15:16:21.186] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.cni // .cni
[15:16:21.187] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:16:21.189] [submariner-operator]$ [cluster2] yq e .clusters.cluster2.cni // .cni /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.238] [submariner-operator]$ [cluster2] cluster_nodes["${cluster}"]=$(_yq ".clusters.${cluster}.nodes // .nodes")
[15:16:21.240] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.nodes // .nodes
[15:16:21.241] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.nodes // .nodes
[15:16:21.242] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:16:21.244] [submariner-operator]$ [cluster2] yq e .clusters.cluster2.nodes // .nodes /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.290] [submariner-operator]$ [cluster2] cluster_subm["${cluster}"]=$(_yq ".clusters.${cluster}.submariner")
[15:16:21.292] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.submariner
[15:16:21.293] [submariner-operator]$ [cluster2] _yq .clusters.cluster2.submariner
[15:16:21.294] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:16:21.296] [submariner-operator]$ [cluster2] yq e .clusters.cluster2.submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.342] [submariner-operator]$ [cluster2] cluster_subm["${cluster}"]=$(_yq ".submariner")
[15:16:21.343] [submariner-operator]$ [cluster2] _yq .submariner
[15:16:21.345] [submariner-operator]$ [cluster2] _yq .submariner
[15:16:21.346] [submariner-operator]$ [cluster2] output=$(yq e "$*" "${settings}")
[15:16:21.347] [submariner-operator]$ [cluster2] yq e .submariner /go/src/github.com/submariner-io/submariner-operator/.shipyard.e2e.yml
[15:16:21.394] [submariner-operator]$ [cluster2] cat <<EOM
Cluster settings::
broker - ${broker@Q}
clusters - ${clusters[*]@Q}
cni - $(typeset -p cluster_cni | cut -f 2- -d=)
nodes per cluster - $(typeset -p cluster_nodes | cut -f 2- -d=)
install submariner - $(typeset -p cluster_subm | cut -f 2- -d=)
EOM
[15:16:21.396] [submariner-operator]$ [cluster2] typeset -p cluster_cni
[15:16:21.397] [submariner-operator]$ [cluster2] cut -f 2- -d=
[15:16:21.400] [submariner-operator]$ [cluster2] typeset -p cluster_nodes
[15:16:21.401] [submariner-operator]$ [cluster2] cut -f 2- -d=
[15:16:21.404] [submariner-operator]$ [cluster2] typeset -p cluster_subm
[15:16:21.405] [submariner-operator]$ [cluster2] cut -f 2- -d=
Cluster settings::
broker - 'cluster1'
clusters - 'cluster1' 'cluster2'
cni - ([cluster2]="weave" [cluster1]="weave" )
nodes per cluster - ([cluster2]="control-plane worker" [cluster1]="control-plane worker" )
install submariner - ([cluster2]="true" [cluster1]="true" )
[15:16:21.408] [submariner-operator]$ [cluster2] declare_cidrs
[15:16:21.410] [submariner-operator]$ [cluster2] declare_cidrs
[15:16:21.412] [submariner-operator]$ [cluster2] declare -gA cluster_CIDRs service_CIDRs global_CIDRs
[15:16:21.414] [submariner-operator]$ [cluster2] i=1
[15:16:21.416] [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
[15:16:21.419] [submariner-operator]$ [cluster1] add_cluster_cidrs 1 cluster1
[15:16:21.421] [submariner-operator]$ [cluster1] val=0
[15:16:21.424] [submariner-operator]$ [cluster1] cluster_CIDRs[cluster1]=10.0.0.0/16
[15:16:21.426] [submariner-operator]$ [cluster1] service_CIDRs[cluster1]=100.0.0.0/16
[15:16:21.428] [submariner-operator]$ [cluster1] global_CIDRs[cluster1]=242.254.1.0/24
[15:16:21.430] [submariner-operator]$ [cluster1] i=$(("$i"+1))
[15:16:21.432] [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
[15:16:21.434] [submariner-operator]$ [cluster2] add_cluster_cidrs 2 cluster2
[15:16:21.436] [submariner-operator]$ [cluster2] val=0
[15:16:21.438] [submariner-operator]$ [cluster2] cluster_CIDRs[cluster2]=10.0.0.0/16
[15:16:21.440] [submariner-operator]$ [cluster2] service_CIDRs[cluster2]=100.0.0.0/16
[15:16:21.442] [submariner-operator]$ [cluster2] global_CIDRs[cluster2]=242.254.2.0/24
[15:16:21.443] [submariner-operator]$ [cluster2] i=$(("$i"+1))
[15:16:21.445] [submariner-operator]$ [cluster2] declare_kubeconfig
[15:16:21.447] [submariner-operator]$ [cluster2] declare_kubeconfig
[15:16:21.449] [submariner-operator]$ [cluster2] source /opt/shipyard/scripts/lib/kubecfg
[15:16:21.451] [submariner-operator]$ [cluster2] export KUBECONFIG
[15:16:21.453] [submariner-operator]$ [cluster2] KUBECONFIG=$(find "${DAPPER_OUTPUT:-output}"/kubeconfigs -type f -printf %p: || :)
[15:16:21.455] [submariner-operator]$ [cluster2] find /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs -type f -printf %p:
[15:16:21.459] [submariner-operator]$ [cluster2] bash -c curl -Ls https://get.submariner.io | VERSION=devel DESTDIR=/go/bin bash
Installing subctl version devel
OS detected: linux
Architecture detected: amd64
Download URL: https://github.com/submariner-io/submariner-operator/releases/download/subctl-devel/subctl-devel-linux-amd64.tar.xz
Downloading...
subctl-devel-linux-amd64 has been installed as /go/bin/subctl
This provides subctl version: devel
please update your path (and consider adding it to ~/.profile):
export PATH=$PATH:/go/bin
[15:16:22.628] [submariner-operator]$ [cluster2] load_deploytool operator
[15:16:22.630] [submariner-operator]$ [cluster2] load_deploytool operator
Will deploy submariner using operator
[15:16:22.632] [submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/deploy_operator
[15:16:22.633] [submariner-operator]$ [cluster2] . /opt/shipyard/scripts/lib/source_only
[15:16:22.635] [submariner-operator]$ [cluster2] script_name=deploy_operator
[15:16:22.636] [submariner-operator]$ [cluster2] exec_name=deploy.sh
[15:16:22.638] [submariner-operator]$ [cluster2] deploytool_prereqs
[15:16:22.640] [submariner-operator]$ [cluster2] deploytool_prereqs
[15:16:22.643] [submariner-operator]$ [cluster2] subctl version
subctl version: devel-82bf5d4
[15:16:22.659] [submariner-operator]$ [cluster2] run_if_defined pre_deploy
[15:16:22.661] [submariner-operator]$ [cluster2] run_if_defined pre_deploy
[15:16:22.662] [submariner-operator]$ [cluster2] type -t pre_deploy
[15:16:22.664] [submariner-operator]$ [cluster2] grep -q function
[15:16:22.667] [submariner-operator]$ [cluster2] return 0
[15:16:22.668] [submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster
[15:16:22.670] [submariner-operator]$ [cluster2] run_subm_clusters prepare_cluster
[15:16:22.671] [submariner-operator]$ [cluster2] declare -a subm_clusters
[15:16:22.676] [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 prepare_cluster
[15:16:22.678] [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 prepare_cluster
[15:16:22.680] [submariner-operator]$ [cluster2] declare -A pids
[15:16:22.682] [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
[15:16:22.684] [submariner-operator]$ [cluster1] set -o pipefail
[15:16:22.684] [submariner-operator]$ [cluster1] pids[cluster1]=3337
[15:16:22.686] [submariner-operator]$ [cluster1] prepare_cluster
[15:16:22.686] [submariner-operator]$ [cluster2] set -o pipefail
[15:16:22.686] [submariner-operator]$ [cluster2] pids[cluster2]=3344
[15:16:22.688] [submariner-operator]$ [cluster2] prepare_cluster
[15:16:22.688] [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
[15:16:22.689] [submariner-operator]$ [cluster2] wait 3344
[15:16:22.688] [submariner-operator]$ [cluster1] prepare_cluster
[15:16:22.690] [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
[15:16:22.690] [submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
[15:16:22.690] [submariner-operator]$ [cluster2] prepare_cluster
[15:16:22.692] [submariner-operator]$ [cluster1] kubectl label node cluster1-worker submariner.io/gateway=true --overwrite
[15:16:22.692] [submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
[15:16:22.694] [submariner-operator]$ [cluster2] kubectl label node cluster2-worker submariner.io/gateway=true --overwrite
[15:16:22.694] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 label node cluster1-worker submariner.io/gateway=true --overwrite
[15:16:22.696] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 label node cluster2-worker submariner.io/gateway=true --overwrite
[cluster1] node/cluster1-worker not labeled
[cluster2] node/cluster2-worker not labeled
[15:16:23.178] [submariner-operator]$ [cluster2] wait 3337
[15:16:23.179] [submariner-operator]$ [cluster2] with_context cluster1 setup_broker
[15:16:23.181] [submariner-operator]$ [cluster2] with_context cluster1 setup_broker
[15:16:23.183] [submariner-operator]$ [cluster1] setup_broker
[15:16:23.185] [submariner-operator]$ [cluster1] setup_broker
[15:16:23.186] [submariner-operator]$ [cluster1] gn=--globalnet
[15:16:23.188] [submariner-operator]$ [cluster1] sd=--components service-discovery,connectivity
Installing broker...
[15:16:23.190] [submariner-operator]$ [cluster1] subctlrepver=
[15:16:23.191] [submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local
[15:16:23.193] [submariner-operator]$ [cluster1] cd /go/src/github.com/submariner-io/submariner-operator/output
[15:16:23.194] [output]$ [cluster1] subctl deploy-broker --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --repository localhost:5000 --version local --globalnet --components service-discovery,connectivity --components service-discovery,connectivity
✓ Setting up broker RBAC
✓ Deploying the Submariner operator
✓ Created Lighthouse service accounts and roles
✓ Deploying the broker
✓ The broker has been deployed
✓ Creating broker-info.subm file
⚠ Reusing IPsec PSK from existing broker-info.subm
✓ Backed up previous broker-info.subm to broker-info.subm.2022-02-16T15_16_33Z
[15:16:33.058] [submariner-operator]$ [cluster2] install_subm_all_clusters
[15:16:33.060] [submariner-operator]$ [cluster2] install_subm_all_clusters
[15:16:33.062] [submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm
[15:16:33.063] [submariner-operator]$ [cluster2] run_subm_clusters subctl_install_subm
[15:16:33.065] [submariner-operator]$ [cluster2] declare -a subm_clusters
[15:16:33.069] [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 subctl_install_subm
[15:16:33.071] [submariner-operator]$ [cluster2] run_parallel cluster1 cluster2 cluster1 cluster2 subctl_install_subm
[15:16:33.073] [submariner-operator]$ [cluster2] declare -A pids
[15:16:33.075] [submariner-operator]$ [cluster2] eval echo cluster1 cluster2
[15:16:33.078] [submariner-operator]$ [cluster1] pids[cluster1]=3506
[15:16:33.078] [submariner-operator]$ [cluster1] set -o pipefail
[15:16:33.079] [submariner-operator]$ [cluster1] subctl_install_subm
[15:16:33.080] [submariner-operator]$ [cluster2] set -o pipefail
[15:16:33.080] [submariner-operator]$ [cluster2] pids[cluster2]=3513
[15:16:33.081] [submariner-operator]$ [cluster1] sed /\[cluster1]/!s/^/[cluster1] /
[15:16:33.081] [submariner-operator]$ [cluster2] subctl_install_subm
[15:16:33.081] [submariner-operator]$ [cluster2] wait 3513
[15:16:33.081] [submariner-operator]$ [cluster1] subctl_install_subm
[15:16:33.083] [submariner-operator]$ [cluster1] subctlrepver=
[15:16:33.083] [submariner-operator]$ [cluster2] sed /\[cluster2]/!s/^/[cluster2] /
[15:16:33.083] [submariner-operator]$ [cluster2] subctl_install_subm
[15:16:33.085] [submariner-operator]$ [cluster1] subctlrepver=--repository localhost:5000 --version local
[15:16:33.085] [submariner-operator]$ [cluster2] subctlrepver=
[15:16:33.086] [submariner-operator]$ [cluster1] PASS_CIDR_ARGS=false
[15:16:33.086] [submariner-operator]$ [cluster2] subctlrepver=--repository localhost:5000 --version local
[15:16:33.088] [submariner-operator]$ [cluster2] PASS_CIDR_ARGS=false
[15:16:33.088] [submariner-operator]$ [cluster1] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster1 --clusterid cluster1 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr 242.254.1.0/24 --natt=false --cable-driver vxlan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm
[15:16:33.090] [submariner-operator]$ [cluster2] subctl join --kubeconfig /go/src/github.com/submariner-io/submariner-operator/output/kubeconfigs/kind-config-cluster2 --clusterid cluster2 --repository localhost:5000 --version local --nattport 4500 --ikeport 500 --colorcodes blue --globalnet-cidr 242.254.2.0/24 --natt=false --cable-driver vxlan /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm
[cluster1] Flag --colorcodes has been deprecated, --colorcodes has no effect and is deprecated
[cluster2] Flag --colorcodes has been deprecated, --colorcodes has no effect and is deprecated
[cluster1] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.7:6443
[cluster2] * /go/src/github.com/submariner-io/submariner-operator/output/broker-info.subm says broker is at: https://172.18.0.7:6443
[cluster1] * There are 1 labeled nodes in the cluster:
[cluster1] - cluster1-worker
[cluster1] • Discovering network details ...
[cluster2] * There are 1 labeled nodes in the cluster:
[cluster2] - cluster2-worker
[cluster2] • Discovering network details ...
[cluster1] Network plugin: weave-net
[cluster1] Service CIDRs: [100.1.0.0/16]
[cluster1] Cluster CIDRs: [10.1.0.0/16]
[cluster1] ✓ Discovering network details
[cluster2] Network plugin: weave-net
[cluster2] Service CIDRs: [100.2.0.0/16]
[cluster2] Cluster CIDRs: [10.2.0.0/16]
[cluster2] ✓ Discovering network details
[cluster1] • Retrieving Globalnet information from the Broker ...
[cluster2] • Retrieving Globalnet information from the Broker ...
[cluster1] ✓ Retrieving Globalnet information from the Broker
[cluster1] • Validating Globalnet configuration ...
[cluster1] ⚠ Validating Globalnet configuration
[cluster1] ⚠ Globalnet is not enabled on the Broker - ignoring the specified global CIDR
[cluster1] • Deploying the Submariner operator ...
[cluster2] ✓ Retrieving Globalnet information from the Broker
[cluster2] • Validating Globalnet configuration ...
[cluster2] ⚠ Validating Globalnet configuration
[cluster2] ⚠ Globalnet is not enabled on the Broker - ignoring the specified global CIDR
[cluster2] • Deploying the Submariner operator ...
[cluster1] ✓ Deploying the Submariner operator
[cluster1] ✓ Created Lighthouse service accounts and roles
[cluster1] • Creating SA for cluster ...
[cluster2] ✓ Deploying the Submariner operator
[cluster2] ✓ Created Lighthouse service accounts and roles
[cluster2] • Creating SA for cluster ...
[cluster1] ✓ Creating SA for cluster
[cluster1] • Deploying Submariner ...
[cluster2] ✓ Creating SA for cluster
[cluster2] • Deploying Submariner ...
[cluster1] ✓ Deploying Submariner
[cluster1] ✓ Submariner is up and running
[cluster2] ✓ Deploying Submariner
[cluster2] ✓ Submariner is up and running
[15:18:50.121] [submariner-operator]$ [cluster2] wait 3506
[15:18:50.125] [submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status
[15:18:50.127] [submariner-operator]$ [cluster2] with_context cluster2 with_retries 30 verify_gw_status
[15:18:50.129] [submariner-operator]$ [cluster2] with_retries 30 verify_gw_status
[15:18:50.130] [submariner-operator]$ [cluster2] with_retries
[15:18:50.132] [submariner-operator]$ [cluster2] iteration=1
[15:18:50.136] [submariner-operator]$ [cluster2] wait 3621
[15:18:50.136] [submariner-operator]$ [cluster2] verify_gw_status
[15:18:50.137] [submariner-operator]$ [cluster2] verify_gw_status
[15:18:50.138] [submariner-operator]$ [cluster2] sleep_duration=6
[15:18:50.140] [submariner-operator]$ [cluster2] subctl show connections
[15:18:50.142] [submariner-operator]$ [cluster2] grep the server could not find the requested resource
[15:18:50.181] [submariner-operator]$ [cluster2] subctl show connections
[15:18:50.182] [submariner-operator]$ [cluster2] grep connected
✓ Showing Connections
cluster1-worker cluster1 172.18.0.4 no libreswan 100.1.0.0/16, 10.1.0.0/16 connected 129.733µs
✗ Showing Connections
✗ There are no gateways detected
[15:18:50.220] [submariner-operator]$ [cluster2] return 0
[15:18:50.221] [submariner-operator]$ [cluster2] return 0
[15:18:50.223] [submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1
[15:18:50.224] [submariner-operator]$ [cluster2] with_context cluster2 connectivity_tests cluster1
[15:18:50.226] [submariner-operator]$ [cluster2] connectivity_tests cluster1
[15:18:50.227] [submariner-operator]$ [cluster2] connectivity_tests
[15:18:50.229] [submariner-operator]$ [cluster2] target_cluster=cluster1
[15:18:50.230] [submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest
[15:18:50.232] [submariner-operator]$ [cluster2] import_image quay.io/submariner/nettest
[15:18:50.235] [submariner-operator]$ [cluster2] docker tag quay.io/submariner/nettest:dev localhost:5000/nettest:local
[15:18:50.453] [submariner-operator]$ [cluster2] docker push localhost:5000/nettest:local
The push refers to repository [localhost:5000/nettest]
89a7ef98a887: Layer already exists
c87409e17378: Layer already exists
76edbd51e58f: Layer already exists
8d3ac3489996: Layer already exists
local: digest: sha256:b71e974e62ff97b617c09ece6b9b6d6e549a3443cff09aede8d4057305a555e8 size: 1155
[15:18:50.659] [submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
[15:18:50.662] [submariner-operator]$ [cluster2] deploy_resource /opt/shipyard/scripts/resources/netshoot.yaml
[15:18:50.664] [submariner-operator]$ [cluster2] resource_name=$(basename "$resource_file" ".yaml")
[15:18:50.666] [submariner-operator]$ [cluster2] basename /opt/shipyard/scripts/resources/netshoot.yaml .yaml
[15:18:50.667] [submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
[15:18:50.669] [submariner-operator]$ [cluster2] kubectl apply -f /opt/shipyard/scripts/resources/netshoot.yaml
[15:18:50.670] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 apply -f /opt/shipyard/scripts/resources/netshoot.yaml
deployment.apps/netshoot created
Waiting for netshoot pods to be ready.
[15:18:51.065] [submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m
[15:18:51.067] [submariner-operator]$ [cluster2] kubectl rollout status deploy/netshoot --timeout=5m
[15:18:51.070] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 rollout status deploy/netshoot --timeout=5m
Waiting for deployment "netshoot" rollout to finish: 0 of 1 updated replicas are available...
deployment "netshoot" successfully rolled out
[15:18:53.291] [submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
[15:18:53.293] [submariner-operator]$ [cluster2] with_context cluster1 deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
[15:18:53.295] [submariner-operator]$ [cluster1] deploy_resource /opt/shipyard/scripts/resources/nginx-demo.yaml
[15:18:53.297] [submariner-operator]$ [cluster1] deploy_resource
[15:18:53.300] [submariner-operator]$ [cluster1] resource_name=$(basename "$resource_file" ".yaml")
[15:18:53.302] [submariner-operator]$ [cluster1] basename /opt/shipyard/scripts/resources/nginx-demo.yaml .yaml
[15:18:53.304] [submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
[15:18:53.306] [submariner-operator]$ [cluster1] kubectl apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
[15:18:53.308] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 apply -f /opt/shipyard/scripts/resources/nginx-demo.yaml
deployment.apps/nginx-demo created
service/nginx-demo created
serviceexport.multicluster.x-k8s.io/nginx-demo created
Waiting for nginx-demo pods to be ready.
[15:18:53.636] [submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m
[15:18:53.638] [submariner-operator]$ [cluster1] kubectl rollout status deploy/nginx-demo --timeout=5m
[15:18:53.640] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 rollout status deploy/nginx-demo --timeout=5m
Waiting for deployment "nginx-demo" rollout to finish: 0 of 2 updated replicas are available...
Waiting for deployment "nginx-demo" rollout to finish: 1 of 2 updated replicas are available...
deployment "nginx-demo" successfully rolled out
[15:18:56.094] [submariner-operator]$ [cluster2] netshoot_pod=$(kubectl get pods -l app=netshoot | awk 'FNR == 2 {print $1}')
[15:18:56.096] [submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot
[15:18:56.097] [submariner-operator]$ [cluster2] awk FNR == 2 {print $1}
[15:18:56.097] [submariner-operator]$ [cluster2] kubectl get pods -l app=netshoot
[15:18:56.099] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 get pods -l app=netshoot
[15:18:56.301] [submariner-operator]$ [cluster2] nginx_svc_ip=$(with_context "$target_cluster" get_svc_ip nginx-demo)
[15:18:56.303] [submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo
[15:18:56.304] [submariner-operator]$ [cluster2] with_context cluster1 get_svc_ip nginx-demo
[15:18:56.305] [submariner-operator]$ [cluster1] get_svc_ip nginx-demo
[15:18:56.307] [submariner-operator]$ [cluster1] get_svc_ip
[15:18:56.308] [submariner-operator]$ [cluster1] svc_ip=$(with_retries 30 get_globalip "${svc_name}")
[15:18:56.310] [submariner-operator]$ [cluster1] with_retries 30 get_globalip nginx-demo
[15:18:56.311] [submariner-operator]$ [cluster1] with_retries 30 get_globalip nginx-demo
[15:18:56.313] [submariner-operator]$ [cluster1] iteration=1
[15:18:56.316] [submariner-operator]$ [cluster1] wait 3928
[15:18:56.316] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:18:56.317] [submariner-operator]$ [cluster1] get_globalip
[15:18:56.318] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:18:56.320] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:56.321] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:56.323] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:56.523] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:18:56.525] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:18:56.527] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:18:56.529] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:18:56.729] [submariner-operator]$ [cluster1] sleep 1
[15:18:57.731] [submariner-operator]$ [cluster1] return 1
[15:18:57.736] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:18:57.736] [submariner-operator]$ [cluster1] wait 4006
[15:18:57.738] [submariner-operator]$ [cluster1] get_globalip
[15:18:57.739] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:18:57.741] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:57.743] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:57.744] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:57.941] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:18:57.943] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:18:57.944] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:18:57.945] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:18:58.140] [submariner-operator]$ [cluster1] sleep 1
[15:18:59.143] [submariner-operator]$ [cluster1] return 1
[15:18:59.148] [submariner-operator]$ [cluster1] wait 4085
[15:18:59.148] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:18:59.149] [submariner-operator]$ [cluster1] get_globalip
[15:18:59.151] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:18:59.152] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:59.154] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:59.155] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:18:59.357] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:18:59.358] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:18:59.360] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:18:59.361] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:18:59.557] [submariner-operator]$ [cluster1] sleep 1
[15:19:00.559] [submariner-operator]$ [cluster1] return 1
[15:19:00.564] [submariner-operator]$ [cluster1] wait 4163
[15:19:00.565] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:00.566] [submariner-operator]$ [cluster1] get_globalip
[15:19:00.568] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:00.569] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:00.571] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:00.572] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:00.772] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:00.773] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:00.774] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:00.776] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:00.973] [submariner-operator]$ [cluster1] sleep 1
[15:19:01.975] [submariner-operator]$ [cluster1] return 1
[15:19:01.980] [submariner-operator]$ [cluster1] wait 4241
[15:19:01.980] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:01.982] [submariner-operator]$ [cluster1] get_globalip
[15:19:01.983] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:01.985] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:01.987] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:01.988] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:02.188] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:02.189] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:02.191] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:02.192] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:02.391] [submariner-operator]$ [cluster1] sleep 1
[15:19:03.394] [submariner-operator]$ [cluster1] return 1
[15:19:03.399] [submariner-operator]$ [cluster1] wait 4318
[15:19:03.399] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:03.400] [submariner-operator]$ [cluster1] get_globalip
[15:19:03.402] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:03.403] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:03.405] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:03.406] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:03.603] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:03.605] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:03.606] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:03.607] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:03.803] [submariner-operator]$ [cluster1] sleep 1
[15:19:04.806] [submariner-operator]$ [cluster1] return 1
[15:19:04.811] [submariner-operator]$ [cluster1] wait 4396
[15:19:04.811] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:04.813] [submariner-operator]$ [cluster1] get_globalip
[15:19:04.814] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:04.816] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:04.818] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:04.819] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:05.018] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:05.020] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:05.021] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:05.022] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:05.220] [submariner-operator]$ [cluster1] sleep 1
[15:19:06.226] [submariner-operator]$ [cluster1] return 1
[15:19:06.235] [submariner-operator]$ [cluster1] wait 4474
[15:19:06.235] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:06.237] [submariner-operator]$ [cluster1] get_globalip
[15:19:06.239] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:06.241] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:06.242] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:06.244] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:06.445] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:06.447] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:06.448] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:06.450] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:06.646] [submariner-operator]$ [cluster1] sleep 1
[15:19:07.648] [submariner-operator]$ [cluster1] return 1
[15:19:07.653] [submariner-operator]$ [cluster1] wait 4553
[15:19:07.653] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:07.655] [submariner-operator]$ [cluster1] get_globalip
[15:19:07.657] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:07.659] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:07.660] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:07.662] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:07.860] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:07.862] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:07.863] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:07.865] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:08.060] [submariner-operator]$ [cluster1] sleep 1
[15:19:09.063] [submariner-operator]$ [cluster1] return 1
[15:19:09.068] [submariner-operator]$ [cluster1] wait 4632
[15:19:09.068] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:09.070] [submariner-operator]$ [cluster1] get_globalip
[15:19:09.071] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:09.073] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:09.075] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:09.076] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:09.275] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:09.277] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:09.278] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:09.279] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:09.476] [submariner-operator]$ [cluster1] sleep 1
[15:19:10.478] [submariner-operator]$ [cluster1] return 1
[15:19:10.483] [submariner-operator]$ [cluster1] wait 4711
[15:19:10.483] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:10.485] [submariner-operator]$ [cluster1] get_globalip
[15:19:10.486] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:10.488] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:10.489] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:10.491] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:10.691] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:10.693] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:10.694] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:10.696] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:10.893] [submariner-operator]$ [cluster1] sleep 1
[15:19:11.895] [submariner-operator]$ [cluster1] return 1
[15:19:11.899] [submariner-operator]$ [cluster1] wait 4790
[15:19:11.899] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:11.901] [submariner-operator]$ [cluster1] get_globalip
[15:19:11.902] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:11.904] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:11.905] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:11.906] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:12.102] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:12.103] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:12.105] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:12.106] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:12.306] [submariner-operator]$ [cluster1] sleep 1
[15:19:13.308] [submariner-operator]$ [cluster1] return 1
[15:19:13.313] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:13.313] [submariner-operator]$ [cluster1] wait 4869
[15:19:13.315] [submariner-operator]$ [cluster1] get_globalip
[15:19:13.316] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:13.318] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:13.319] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:13.321] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:13.522] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:13.523] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:13.525] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:13.526] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:13.722] [submariner-operator]$ [cluster1] sleep 1
[15:19:14.724] [submariner-operator]$ [cluster1] return 1
[15:19:14.730] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:14.730] [submariner-operator]$ [cluster1] wait 4948
[15:19:14.731] [submariner-operator]$ [cluster1] get_globalip
[15:19:14.733] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:14.734] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:14.736] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:14.737] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:14.937] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:14.938] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:14.940] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:14.941] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:15.138] [submariner-operator]$ [cluster1] sleep 1
[15:19:16.141] [submariner-operator]$ [cluster1] return 1
[15:19:16.146] [submariner-operator]$ [cluster1] wait 5026
[15:19:16.146] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:16.147] [submariner-operator]$ [cluster1] get_globalip
[15:19:16.149] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:16.150] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:16.152] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:16.153] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:16.350] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:16.351] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:16.353] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:16.354] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:16.549] [submariner-operator]$ [cluster1] sleep 1
[15:19:17.556] [submariner-operator]$ [cluster1] return 1
[15:19:17.563] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:17.563] [submariner-operator]$ [cluster1] wait 5103
[15:19:17.565] [submariner-operator]$ [cluster1] get_globalip
[15:19:17.566] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:17.568] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:17.570] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:17.571] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:17.771] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:17.773] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:17.774] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:17.775] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:17.972] [submariner-operator]$ [cluster1] sleep 1
[15:19:18.975] [submariner-operator]$ [cluster1] return 1
[15:19:18.980] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:18.980] [submariner-operator]$ [cluster1] wait 5181
[15:19:18.981] [submariner-operator]$ [cluster1] get_globalip
[15:19:18.983] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:18.984] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:18.986] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:18.988] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:19.187] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:19.188] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:19.190] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:19.191] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:19.387] [submariner-operator]$ [cluster1] sleep 1
[15:19:20.390] [submariner-operator]$ [cluster1] return 1
[15:19:20.395] [submariner-operator]$ [cluster1] wait 5259
[15:19:20.395] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:20.396] [submariner-operator]$ [cluster1] get_globalip
[15:19:20.398] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:20.400] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:20.401] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:20.403] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:20.601] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:20.603] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:20.604] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:20.606] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:20.802] [submariner-operator]$ [cluster1] sleep 1
[15:19:21.805] [submariner-operator]$ [cluster1] return 1
[15:19:21.809] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:21.809] [submariner-operator]$ [cluster1] wait 5338
[15:19:21.811] [submariner-operator]$ [cluster1] get_globalip
[15:19:21.812] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:21.814] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:21.815] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:21.817] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:22.014] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:22.016] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:22.017] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:22.018] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:22.215] [submariner-operator]$ [cluster1] sleep 1
[15:19:23.217] [submariner-operator]$ [cluster1] return 1
[15:19:23.222] [submariner-operator]$ [cluster1] wait 5416
[15:19:23.223] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:23.224] [submariner-operator]$ [cluster1] get_globalip
[15:19:23.226] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:23.228] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:23.229] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:23.231] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:23.434] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:23.436] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:23.437] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:23.439] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:23.641] [submariner-operator]$ [cluster1] sleep 1
[15:19:24.644] [submariner-operator]$ [cluster1] return 1
[15:19:24.649] [submariner-operator]$ [cluster1] wait 5494
[15:19:24.649] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:24.650] [submariner-operator]$ [cluster1] get_globalip
[15:19:24.652] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:24.654] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:24.655] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:24.657] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:24.856] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:24.857] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:24.859] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:24.860] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:25.057] [submariner-operator]$ [cluster1] sleep 1
[15:19:26.060] [submariner-operator]$ [cluster1] return 1
[15:19:26.066] [submariner-operator]$ [cluster1] wait 5573
[15:19:26.066] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:26.068] [submariner-operator]$ [cluster1] get_globalip
[15:19:26.069] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:26.071] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:26.073] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:26.074] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:26.277] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:26.278] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:26.280] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:26.281] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:26.477] [submariner-operator]$ [cluster1] sleep 1
[15:19:27.481] [submariner-operator]$ [cluster1] return 1
[15:19:27.486] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:27.486] [submariner-operator]$ [cluster1] wait 5651
[15:19:27.487] [submariner-operator]$ [cluster1] get_globalip
[15:19:27.489] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:27.490] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:27.492] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:27.493] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:27.689] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:27.691] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:27.692] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:27.694] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:27.890] [submariner-operator]$ [cluster1] sleep 1
[15:19:28.893] [submariner-operator]$ [cluster1] return 1
[15:19:28.898] [submariner-operator]$ [cluster1] wait 5730
[15:19:28.898] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:28.900] [submariner-operator]$ [cluster1] get_globalip
[15:19:28.901] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:28.903] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:28.905] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:28.906] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:29.105] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:29.107] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:29.108] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:29.110] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:29.308] [submariner-operator]$ [cluster1] sleep 1
[15:19:30.310] [submariner-operator]$ [cluster1] return 1
[15:19:30.315] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:30.315] [submariner-operator]$ [cluster1] wait 5809
[15:19:30.317] [submariner-operator]$ [cluster1] get_globalip
[15:19:30.318] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:30.320] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:30.321] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:30.323] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:30.525] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:30.527] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:30.528] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:30.530] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:30.727] [submariner-operator]$ [cluster1] sleep 1
[15:19:31.730] [submariner-operator]$ [cluster1] return 1
[15:19:31.736] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:31.736] [submariner-operator]$ [cluster1] wait 5888
[15:19:31.738] [submariner-operator]$ [cluster1] get_globalip
[15:19:31.739] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:31.741] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:31.743] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:31.745] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:31.954] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:31.956] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:31.957] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:31.959] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:32.156] [submariner-operator]$ [cluster1] sleep 1
[15:19:33.158] [submariner-operator]$ [cluster1] return 1
[15:19:33.163] [submariner-operator]$ [cluster1] wait 5967
[15:19:33.163] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:33.165] [submariner-operator]$ [cluster1] get_globalip
[15:19:33.166] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:33.168] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:33.169] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:33.171] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:33.369] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:33.371] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:33.372] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:33.373] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:33.571] [submariner-operator]$ [cluster1] sleep 1
[15:19:34.573] [submariner-operator]$ [cluster1] return 1
[15:19:34.578] [submariner-operator]$ [cluster1] wait 6045
[15:19:34.578] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:34.580] [submariner-operator]$ [cluster1] get_globalip
[15:19:34.582] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:34.583] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:34.585] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:34.586] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:34.785] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:34.786] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:34.788] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:34.789] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:34.986] [submariner-operator]$ [cluster1] sleep 1
[15:19:35.988] [submariner-operator]$ [cluster1] return 1
[15:19:35.994] [submariner-operator]$ [cluster1] wait 6122
[15:19:35.994] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:35.996] [submariner-operator]$ [cluster1] get_globalip
[15:19:35.997] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:35.999] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:36.001] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:36.002] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:36.204] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:36.206] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:36.207] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:36.208] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:36.405] [submariner-operator]$ [cluster1] sleep 1
[15:19:37.407] [submariner-operator]$ [cluster1] return 1
[15:19:37.412] [submariner-operator]$ [cluster1] wait 6200
[15:19:37.412] [submariner-operator]$ [cluster1] get_globalip nginx-demo
[15:19:37.414] [submariner-operator]$ [cluster1] get_globalip
[15:19:37.415] [submariner-operator]$ [cluster1] gip=$(kubectl get svc "$svc_name" -o jsonpath='{.metadata.annotations.submariner\.io/globalIp}')
[15:19:37.417] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:37.418] [submariner-operator]$ [cluster1] kubectl get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:37.420] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get svc nginx-demo -o jsonpath={.metadata.annotations.submariner\.io/globalIp}
[15:19:37.619] [submariner-operator]$ [cluster1] gip=$(kubectl get giip "$svc_name" -o jsonpath='{.status.allocatedIP}')
[15:19:37.621] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:37.622] [submariner-operator]$ [cluster1] kubectl get giip nginx-demo -o jsonpath={.status.allocatedIP}
[15:19:37.623] [submariner-operator]$ [cluster1] command kubectl --context=cluster1 get giip nginx-demo -o jsonpath={.status.allocatedIP}
Error from server (NotFound): globalingressips.submariner.io "nginx-demo" not found
[15:19:37.820] [submariner-operator]$ [cluster1] sleep 1
[15:19:38.822] [submariner-operator]$ [cluster1] return 1
[15:19:38.827] [submariner-operator]$ [cluster1] exit 1
[15:19:38.829] [submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:19:38.831] [submariner-operator]$ [cluster2] with_retries 5 test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:19:38.832] [submariner-operator]$ [cluster2] iteration=1
[15:19:38.835] [submariner-operator]$ [cluster2] wait 6293
[15:19:38.835] [submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:19:38.837] [submariner-operator]$ [cluster2] test_connection
Attempting connectivity between clusters - netshoot-57677568b8-nsdt7 --> Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:19:38.838] [submariner-operator]$ [cluster2] wait_time=30
[15:19:38.840] [submariner-operator]$ [cluster2] start_time=$(date +%s)
[15:19:38.841] [submariner-operator]$ [cluster2] date +%s
[15:19:38.843] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:19:38.844] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:19:38.846] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
command terminated with exit code 3
[15:19:39.099] [submariner-operator]$ [cluster2] end_time=$(date +%s)
[15:19:39.100] [submariner-operator]$ [cluster2] date +%s
[15:19:39.102] [submariner-operator]$ [cluster2] execution_time=$(( end_time - start_time))
[15:19:39.104] [submariner-operator]$ [cluster2] remaining_time=$(( wait_time - execution_time))
curl returned too soon. Sleeping for 29 secs
[15:19:39.105] [submariner-operator]$ [cluster2] sleep 29
[15:20:08.112] [submariner-operator]$ [cluster2] return 1
[15:20:08.121] [submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:08.121] [submariner-operator]$ [cluster2] wait 6364
[15:20:08.123] [submariner-operator]$ [cluster2] test_connection
Attempting connectivity between clusters - netshoot-57677568b8-nsdt7 --> Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:08.125] [submariner-operator]$ [cluster2] wait_time=30
[15:20:08.126] [submariner-operator]$ [cluster2] start_time=$(date +%s)
[15:20:08.128] [submariner-operator]$ [cluster2] date +%s
[15:20:08.130] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:08.132] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:08.134] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
command terminated with exit code 3
[15:20:08.383] [submariner-operator]$ [cluster2] end_time=$(date +%s)
[15:20:08.385] [submariner-operator]$ [cluster2] date +%s
[15:20:08.387] [submariner-operator]$ [cluster2] execution_time=$(( end_time - start_time))
[15:20:08.388] [submariner-operator]$ [cluster2] remaining_time=$(( wait_time - execution_time))
curl returned too soon. Sleeping for 30 secs
[15:20:08.390] [submariner-operator]$ [cluster2] sleep 30
[15:20:38.392] [submariner-operator]$ [cluster2] return 1
[15:20:38.397] [submariner-operator]$ [cluster2] wait 6434
[15:20:38.397] [submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:38.398] [submariner-operator]$ [cluster2] test_connection
Attempting connectivity between clusters - netshoot-57677568b8-nsdt7 --> Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:38.400] [submariner-operator]$ [cluster2] wait_time=30
[15:20:38.401] [submariner-operator]$ [cluster2] start_time=$(date +%s)
[15:20:38.403] [submariner-operator]$ [cluster2] date +%s
[15:20:38.405] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:38.406] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:20:38.408] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
command terminated with exit code 3
[15:20:38.652] [submariner-operator]$ [cluster2] end_time=$(date +%s)
[15:20:38.654] [submariner-operator]$ [cluster2] date +%s
[15:20:38.656] [submariner-operator]$ [cluster2] execution_time=$(( end_time - start_time))
[15:20:38.657] [submariner-operator]$ [cluster2] remaining_time=$(( wait_time - execution_time))
curl returned too soon. Sleeping for 30 secs
[15:20:38.659] [submariner-operator]$ [cluster2] sleep 30
[15:21:08.661] [submariner-operator]$ [cluster2] return 1
[15:21:08.666] [submariner-operator]$ [cluster2] wait 6504
[15:21:08.666] [submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:08.668] [submariner-operator]$ [cluster2] test_connection
Attempting connectivity between clusters - netshoot-57677568b8-nsdt7 --> Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:08.669] [submariner-operator]$ [cluster2] wait_time=30
[15:21:08.671] [submariner-operator]$ [cluster2] start_time=$(date +%s)
[15:21:08.672] [submariner-operator]$ [cluster2] date +%s
[15:21:08.675] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:08.676] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:08.678] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
command terminated with exit code 3
[15:21:08.930] [submariner-operator]$ [cluster2] end_time=$(date +%s)
[15:21:08.931] [submariner-operator]$ [cluster2] date +%s
[15:21:08.933] [submariner-operator]$ [cluster2] execution_time=$(( end_time - start_time))
[15:21:08.935] [submariner-operator]$ [cluster2] remaining_time=$(( wait_time - execution_time))
curl returned too soon. Sleeping for 30 secs
[15:21:08.936] [submariner-operator]$ [cluster2] sleep 30
[15:21:38.938] [submariner-operator]$ [cluster2] return 1
[15:21:38.944] [submariner-operator]$ [cluster2] test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:38.944] [submariner-operator]$ [cluster2] wait 6573
[15:21:38.945] [submariner-operator]$ [cluster2] test_connection
Attempting connectivity between clusters - netshoot-57677568b8-nsdt7 --> Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:38.947] [submariner-operator]$ [cluster2] wait_time=30
[15:21:38.948] [submariner-operator]$ [cluster2] start_time=$(date +%s)
[15:21:38.950] [submariner-operator]$ [cluster2] date +%s
[15:21:38.952] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:38.954] [submariner-operator]$ [cluster2] kubectl exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
[15:21:38.955] [submariner-operator]$ [cluster2] command kubectl --context=cluster2 exec netshoot-57677568b8-nsdt7 -- curl --output /dev/null -m 30 --silent --head --fail Max attempts reached, failed to run 'get_globalip nginx-demo'!
command terminated with exit code 3
[15:21:39.211] [submariner-operator]$ [cluster2] end_time=$(date +%s)
[15:21:39.213] [submariner-operator]$ [cluster2] date +%s
[15:21:39.215] [submariner-operator]$ [cluster2] execution_time=$(( end_time - start_time))
[15:21:39.216] [submariner-operator]$ [cluster2] remaining_time=$(( wait_time - execution_time))
curl returned too soon. Sleeping for 29 secs
[15:21:39.218] [submariner-operator]$ [cluster2] sleep 29
[15:22:08.220] [submariner-operator]$ [cluster2] return 1
Max attempts reached, failed to run 'test_connection netshoot-57677568b8-nsdt7 Max attempts reached, failed to run 'get_globalip nginx-demo'!'!
[15:22:08.227] [submariner-operator]$ [cluster2] exit 1
make: *** [/opt/shipyard/Makefile.inc:134: deploy] Error 1
[15:22:08.231] [submariner-operator]$ make --debug=b deploy
make: *** [Makefile.dapper:43: deploy] Error 2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment