Skip to content

Instantly share code, notes, and snippets.

View onefoursix's full-sized avatar

Mark Brooks onefoursix

59 followers · 3 following
View GitHub Profile
@onefoursix
onefoursix / auth-sdc-deployment-nfs.yaml
Last active April 17, 2020 01:16
Authoring SDC Deployment YAML with NFS-based stage-libs and resources, Service and Ingress
apiVersion: 1
kind: List
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-sdc
namespace: ns1
labels:
app : auth-sdc
@onefoursix
onefoursix / auth-sdc-deploy-svc-ingress.yaml
Last active June 10, 2020 00:30
Authoring Data Collector YAML with Deployment, Service and Ingress
apiVersion: 1
kind: List
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-sdc
labels:
app : auth-sdc
spec:
@onefoursix
onefoursix / sdc-jdbc-k8s-secret.yaml
Created May 15, 2020 18:32
SDC Deployment YAML with MySQL connection properties loaded from a Kubernetes Secret
apiVersion: apps/v1
kind: Deployment
metadata:
name: sdc-jdbc
namespace: ns1
spec:
replicas: 1
selector:
matchLabels:
app: sdc-jdbc
@onefoursix
onefoursix / sdc-deploy-svc-ingress-nginx.yaml
Last active June 10, 2020 06:44
SDC Deployment ,Service and Ingress using NGINX
apiVersion: 1
kind: List
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-sdc
labels:
app : auth-sdc
spec:
@onefoursix
onefoursix / sdc-hostPath-https.yaml
Created June 11, 2020 06:06
hostPath Volume Mount for SDC deployment with HTTPS
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-sdc
labels:
app : auth-sdc
spec:
replicas: 1
selector:
matchLabels:
@onefoursix
onefoursix / sdc-deploy-with-configmap-and-secrets.yaml
Last active June 14, 2020 20:38
Deployment manifest for SDC with sdc.id, auth-token and keystore.jks mounted from secrets and dpm.properties from a configmap
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-sdc
labels:
app : auth-sdc
spec:
replicas: 1
selector:
matchLabels:
@onefoursix
onefoursix / sdc-microservice-deployment-and-service.yaml
Last active June 26, 2020 03:51
SDC Deployment Manifest with a Service that exposes an SDC Microservice
apiVersion: 1
kind: List
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
name: sdc-microservice
labels:
app : sdc-microservice
spec:
@onefoursix
onefoursix / build.sh
Last active July 2, 2020 00:34
SDC 3.16.1 Docker Image build file to update AWS Java SDK libs (workaround for https://issues.streamsets.com/browse/SDC-10577)
#!/usr/bin/env bash
SDC_VERSION=3.16.1
# Your SDC Docker Image Name
IMAGE_NAME=<your repo>/<your image>:$SDC_VERSION
# Cleanup old runs
rm -rf streamsets-datacollector-aws-lib
@onefoursix
onefoursix / Dockerfile
Created July 2, 2020 00:32
SDC 3.16.1 Dockerfile to update AWS Java SDK libs (workaround for https://issues.streamsets.com/browse/SDC-10577)
ARG SDC_VERSION
ARG SDC_LIBS
FROM streamsets/datacollector:${SDC_VERSION}
RUN "${SDC_DIST}/bin/streamsets" stagelibs -install="${SDC_LIBS}"
COPY streamsets-datacollector-aws-lib ${SDC_DIST}/streamsets-libs/streamsets-datacollector-aws-lib
@onefoursix
onefoursix / get_s3_file.groovy
Last active July 6, 2020 16:07
SDC Groovy Script to get S3 File
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectInputStream;
records = sdc.records
for (record in records) {
S3ObjectInputStream s3is
FileOutputStream fos