Skip to content

Instantly share code, notes, and snippets.

@ajiteshk-G
Created February 11, 2022 13:35
Show Gist options
  • Save ajiteshk-G/102c1f3c87b57c2b8837251b1cc8bad2 to your computer and use it in GitHub Desktop.
Save ajiteshk-G/102c1f3c87b57c2b8837251b1cc8bad2 to your computer and use it in GitHub Desktop.
Debezium GKE Setup
apiVersion: v1
kind: ConfigMap
metadata:
name: debezium
labels:
app: debezium
namespace: debezium
data:
application.properties: |-
debezium.sink.type=pubsub
debezium.sink.pubsub.project.id=acko-poc
debezium.sink.pubsub.ordering.enabled=true
debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector
debezium.source.offset.storage.file.filename=data/offsets.dat
debezium.source.offset.flush.interval.ms=0
debezium.source.database.hostname=localhost #as we are connecting via Cloud Proxy
debezium.source.database.port= <Postgres Port> #generally 5432
debezium.source.database.user= <user created> # in our case - debezium_user
debezium.source.database.password=<password while creating new user> #in our case "admin"
debezium.source.database.dbname=<Database> #OurCase postgres
debezium.source.database.server.name=<Database Server name> #In Our Case debezium-db
debezium.source.table.include.list=public.actor #Table which needs to captured
debezium.source.plugin.name=pgoutput
apiVersion: v1
kind: Service
metadata:
name: debezium
labels:
name: debezium
namespace: debezium
spec:
type: ClusterIP
ports:
- port: 8080
targetPort: 8080
protocol: TCP
name: http
selector:
name: debezium
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: debezium-gke
labels:
name: debezium
namespace: debezium
spec:
replicas: 1
serviceName: debezium
selector:
matchLabels:
name: debezium
template:
metadata:
labels:
name: debezium
version: v1
spec:
serviceAccountName: <Service Name Created while creating Cluster>
nodeSelector:
iam.gke.io/gke-metadata-server-enabled: "true"
containers:
- name: debezium
image: debezium/server:1.7.0.Final
volumeMounts:
- name: debezium-config-volume
mountPath: /debezium/conf
- name: debezium-data-volume
mountPath: /debezium/data
- name: cloud-sql-proxy
image: gcr.io/cloudsql-docker/gce-proxy:1.17
command:
- "/cloud_sql_proxy"
- "-instances=<ConnectionName from CloudSQL Page>=tcp:<Port>"
securityContext:
runAsNonRoot: true
volumes:
- name: debezium-config-volume
configMap:
name: debezium
volumeClaimTemplates:
- metadata:
name: debezium-data-volume
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 10Gb
@ajiteshk-G
Copy link
Author

Seems to be related to your configuration, Please validate

@akash-binwal
Copy link

My PG SQL version is 11, do you think this may be causing the issue ?

@akash-binwal
Copy link

And also i just want to ask one more thing in the debezium-statefulset.yaml file at line no-20, which service account do we need to pass? the one which is using by GKE VM's?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment