Skip to content

Instantly share code, notes, and snippets.

View onefoursix's full-sized avatar

Mark Brooks onefoursix

View GitHub Profile
@onefoursix
onefoursix / get-stage-libs.sh
Last active July 27, 2020 08:17
Downloads a set of SDC stage libs and enterprise stage libs
#!/usr/bin/env bash
# This script will download SDC stage libs and enterprise stage libs
# and write them to a directory named streamsets-libs
SDC_VERSION=3.16.1
BASE_URL=https://archives.streamsets.com/datacollector
SDC_STAGE_LIBS="streamsets-datacollector-aws-lib streamsets-datacollector-basic-lib streamsets-datacollector-bigtable-lib streamsets-datacollector-dataformats-lib streamsets-datacollector-dev-lib streamsets-datacollector-google-cloud-lib streamsets-datacollector-groovy_2_4-lib streamsets-datacollector-jdbc-lib streamsets-datacollector-jms-lib streamsets-datacollector-jython_2_7-lib streamsets-datacollector-stats-lib streamsets-datacollector-windows-lib"
@onefoursix
onefoursix / sdc-with-sa.yaml
Last active July 27, 2020 00:27
SDC Deployment manifest with ServiceAccount for AWS S3 IAM Role
apiVersion: apps/v1
kind: Deployment
metadata:
name: sdc
spec:
selector:
matchLabels:
app: sdc
template:
metadata:
@onefoursix
onefoursix / auth-sdc-with-sa.yaml
Last active July 27, 2020 05:58
Authoring SDC Deployment manifest with ServiceAccount for AWS S3 IAM Role
apiVersion: 1
kind: List
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-sdc
namespace: <your namespace>
spec:
selector:
@onefoursix
onefoursix / example-sdc-deployment.yaml
Created July 27, 2020 06:13
example-sdc-deployment.yaml
apiVersion: 1
kind: List
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-sdc
namespace: ns1
spec:
selector:
@onefoursix
onefoursix / sdc-extras.yaml
Created September 5, 2020 05:32
SDC Deployment Manifest with VolumeMount for sdc-extras
apiVersion: apps/v1
kind: Deployment
metadata:
name: sdc
namespace: ns1
spec:
selector:
matchLabels:
app: sdc
template:
@onefoursix
onefoursix / sdc-id.sh
Last active September 23, 2020 00:22
This script generates a lowercase UUID and stores the value in a Secret named sdc-id within the specified namespace
#!/usr/bin/env bash
## This script generates a lowercase UUID and stores the value
## in a Secret named sdc-id
# Create a lower-cased UUID
SDC_ID=`uuidgen | tr "[:upper:]" "[:lower:]"`
# Store the UUID in a Secret
kubectl create secret generic sdc-id --from-literal=sdc.id=${SDC_ID} -n <your namespace>
@onefoursix
onefoursix / deployment-with-sdc-id.yaml
Last active September 22, 2020 23:45
Transformer Helm Chart deployment manifest with sdc-id Volume Mount from Secret
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "transformer.fullname" . }}
labels:
app: {{ include "transformer.name" . }}
chart: {{ include "transformer.chart" . }}
release: {{ .Release.Name }}
managed-by: {{ .Release.Service }}
spec:
@onefoursix
onefoursix / ingress-nginx-nodeport.yaml
Last active October 28, 2020 03:20
ingress-nginx Ingress Controller configured as a NodePort Service
apiVersion: v1
kind: Namespace
metadata:
name: ingress-nginx
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
---
@onefoursix
onefoursix / nginx-ingress.yaml
Last active October 28, 2020 04:04
nginx-ingress.yaml that routes to nginx NodePort Service
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host:
http:
@onefoursix
onefoursix / sdc-hpa.yaml
Last active November 16, 2020 07:40
StreamSets Data Collector Deployment Manifest with HPA
apiVersion: 1
kind: List
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
name: sdc-hpa
namespace: ns1
labels:
app: sdc-hpa