This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| POST apm-*-span/_search | |
| { | |
| "size": 0, | |
| "query": { | |
| "bool": { | |
| "must": [ | |
| { | |
| "range": { | |
| "@timestamp": { | |
| "gte": "2023-02-16T13:00:00.540Z" |
ebuildy
/ impl-workflow.yaml
Last active
November 12, 2022 21:21
ETL to elasticsearch with pyspark , argo-workflow , kubernetes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| apiVersion: argoproj.io/v1alpha1 | |
| kind: Workflow | |
| metadata: | |
| generateName: djobi-workflow- | |
| spec: | |
| entrypoint: djobi | |
| serviceAccountName: workflow | |
| ttlStrategy: |
ebuildy
/ simple-pyspark-argo-workflow.yaml
Last active
November 10, 2022 00:09
Play with pyspark from argo workflow
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| apiVersion: argoproj.io/v1alpha1 | |
| kind: Workflow | |
| metadata: | |
| generateName: pyspark- | |
| spec: | |
| entrypoint: pyspark | |
| templates: | |
| - name: pyspark | |
| inputs: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public async deleteKeys(keys: string[]): Promise<any[]> { | |
| function spliceIntoChunks(arr: any[], chunkSize: number) { | |
| const res = []; | |
| while (arr.length > 0) { | |
| const chunk = arr.splice(0, chunkSize); | |
| res.push(chunk); | |
| } | |
| return res; | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| spark.executor.extraJavaOptions: -Dorg.xerial.snappy.use.systemlib=true -Dorg.xerial.snappy.lib.path=/usr/lib/libsnappyjava.so |
ebuildy
/ create_transform.json
Created
April 12, 2022 06:22
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| PUT _transform/test | |
| { | |
| "source": { | |
| "index": [ | |
| "apm-*-span" | |
| ], | |
| "query": { | |
| "bool": { | |
| "filter": [ | |
| { |
ebuildy
/ create aliases
Last active
February 17, 2022 09:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| POST _aliases | |
| { | |
| "actions": [ | |
| { | |
| "remove": { | |
| "index": "consolidation-ctr-v1", | |
| "alias": "consolidation-ctr" | |
| } | |
| }, | |
| { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| --- | |
| - tags: ["always"] | |
| set_fact: | |
| argocd_url: http://localhost:2369 | |
| api_headers: &api_headers | |
| Authorization: "Bearer {{ lookup('env', 'ARGO_TOKEN') }}" |
ebuildy
/ update_labels_bulk.sh
Created
December 28, 2021 09:14
Update kubernetes pod labels, bulk mode
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # get pods to update | |
| kubectl get pods --selector=elastic.org/elasticsearch-cluster=hot --output=jsonpath={.items..metadata.name} | |
ebuildy
/ chart.yaml
Last active
January 21, 2022 12:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| apiVersion: v2 | |
| name: my-elasticsearch | |
| type: application | |
| version: v0.0.1 | |
| dependencies: | |
| - name: elasticsearch | |
| repository: https://helm.datatok.io | |
| version: 7.16.1-4 | |
| - name: networkpolicy | |
| repository: https://ebuildy.github.io/k8s-as-helm/ |