LABOUARDY Mohamed mlabouardy
☁️
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| appengine.googleapis.com | |
| bigquery-json.googleapis.com | |
| compute.googleapis.com | |
| cloudfunctions.googleapis.com | |
| container.googleapis.com | |
| cloudresourcemanager.googleapis.com | |
| cloudkms.googleapis.com | |
| dns.googleapis.com | |
| dataflow.googleapis.com | |
| dataproc.googleapis.com |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| func handler(ctx context.Context) error { | |
| client, err := bigquery.NewClient(ctx, os.Getenv("PROJECT_ID")) | |
| if err != nil { | |
| return err | |
| } | |
| err = RemoveDataSet(client) | |
| if err != nil { | |
| return err | |
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| echo "Download BigQuery Credentials" | |
| aws s3 cp s3://$GCP_AUTH_BUCKET/auth.json . | |
| echo "Upload CSV to GCS" | |
| mkdir -p csv | |
| rm tables |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FROM alpine:latest | |
| LABEL Author mlabouardy | |
| ENV SDK_URL https://dl.google.com/dl/cloudsdk/channels/rapid/google-cloud-sdk.zip | |
| ENV CLOUDSDK_PYTHON_SITEPACKAGES=1 | |
| ENV PATH /google-cloud-sdk/bin:$PATH | |
| ENV HOME / | |
| ENV BUILD_DEPS \ | |
| g++ \ | |
| gcc \ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import mysql.connector | |
| import os | |
| import time | |
| from mysql.connector import Error | |
| from google.cloud import bigquery | |
| bigquery_client = bigquery.Client() | |
| def mapToBigQueryDataType(columnType): | |
| if columnType.startswith('int'): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| default_platform(:ios) | |
| platform :ios do | |
| lane :tests do | |
| scan( | |
| scheme: options[:scheme], | |
| clean: true, | |
| skip_detect_devices: true, | |
| build_for_testing: true, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def bucket = 'mobile-artifacts-foxintelligence' | |
| node('mac') { | |
| try { | |
| stage('Checkout') { | |
| checkout scm | |
| notifySlack('STARTED') | |
| } | |
| stage('Install Dependencies') { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def bucket = 'mobile-artifacts-foxintelligence' | |
| node('android') { | |
| try { | |
| stage('Checkout') { | |
| checkout scm | |
| notifySlack('STARTED') | |
| } | |
| stage('Clean & Prepare') { |
mlabouardy
/ create-subscription.sh
Created
January 21, 2019 10:42
Create filter subscription on log group
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| aws logs put-subscription-filter --log-group-name "/aws/lambda/Fibonacci" \ | |
| --filter-name "parse-logs" --filter-pattern '[timestamp=*Z, request_id="*-*", event]' \ | |
| --destination-arn "arn:aws:logs:AWS_REGION:ACCOUNT_ID:destination:kinesis-stream" |
mlabouardy
/ cloudwatch-destination.sh
Created
January 21, 2019 10:40
Create CloudWatch destination to stream logs to Kinesis
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| aws logs put-destination --destination-name kinesis-stream \ | |
| --target-arn "arn:aws:kinesis:AWS_REGION:ACCOUNT_ID:stream/cloudwatch-logs" \ | |
| --role-arn "arn:aws:iam::ACCOUNT_ID:role/CloudWatchStreamToKinesisRole" |