Skip to content

Instantly share code, notes, and snippets.

View a-mpch's full-sized avatar
🏠
Working from home

Maurice Poirrier a-mpch

🏠
Working from home
View GitHub Profile
@a-mpch
a-mpch / pekenExample.c
Created March 17, 2021 00:47
Peken Example
/*
Code example of Peken.io
We assume that you have to setup other variables
*/
#include "Peken.h"
int value;
void setup(){
startYourInternetConnection();
from apache_beam.pvalue import TaggedOutput
class SplitByRoute(beam.DoFn):
OUTPUT_PARROT = "PARROT"
OUTPUT_DOGGO = "DOGGO"
OUTPUT_CAT = "CAT"
OUTPUT_HORSE = "HORSE"
def __init__(self):
beam.DoFn.__init__(self)
@a-mpch
a-mpch / pipeline_start.py
Created September 14, 2020 21:02
Run Pipeline different runners
if options.view_as(StandardOptions).runner in ("DirectRunner", None):
pubsub_id = None
with beam.Pipeline(options=options) as p:
_run_pipeline(p, pubsub_id, file_processing_options, gcp_project)
elif options.view_as(StandardOptions).runner == "DataflowRunner":
pipeline = beam.Pipeline(options=options)
_run_pipeline(pipeline, pubsub_id, file_processing_options, gcp_project)
pipeline.run()
else:
@a-mpch
a-mpch / Makefile
Created September 14, 2020 20:57
Deploy
deploy:
python main.py \
--region $(REGION) \
--job_name deepdrive-pipe \
--subscription $(SUB_INPUT) \
--project $(GCP_PROJECT) \
--output_topic $(OUTPUT_TOPIC) \
--api_endpoint $(API_ENDPOINT) \
--runner=DataflowRunner \
--staging_location=gs://$(BUCKET)/staging \
import pdb
pdb.set_trace()

Keybase proof

I hereby claim:

  • I am mauricepoirrier on github.
  • I am mpoirrier (https://keybase.io/mpoirrier) on keybase.
  • I have a public key whose fingerprint is 862F 1A4E 85E2 6D8C C982 4618 EA57 8AF9 3EEA F225

To claim this, I am signing this object: