Skip to content

Instantly share code, notes, and snippets.

View taufiqibrahim's full-sized avatar

taufiqibrahim

View GitHub Profile
@taufiqibrahim
taufiqibrahim / data-pipeline-service-graph-using-aws-neptune--graphexp-docker-compose.yml
Created March 12, 2021 17:06
Data Pipeline Service Graph Using AWS Neptune - Graphexp Docker Compose
# https://github.com/bricaud/graphexp/archive/v0.8.0.zip
#
# Copyright IBM Corp All Rights Reserved
#
# SPDX-License-Identifier: Apache-2.0
#
version: "3"
services:
graphexp:
@taufiqibrahim
taufiqibrahim / data-pipeline-service-graph-using-aws-neptune--graphexp-Dockerfile
Created March 12, 2021 17:02
Data Pipeline Service Graph Using AWS Neptune - Graphexp Dockerfile
FROM nginx:stable
LABEL maintainer="[email protected]"
ENV http_proxy $http_proxy
ENV https_proxy $https_proxy
ENV no_proxy $no_proxy
ENV PATH /usr/local/bin:$PATH
ENV GRAPHEXP_SOURCE_URL $GRAPHEXP_SOURCE_URL
@taufiqibrahim
taufiqibrahim / data-pipeline-service-graph-using-aws-neptune--neptune.params.json
Created March 12, 2021 09:10
Data Pipeline Service Graph Using AWS Neptune - Cloudformation Parameters
[
{
"ParameterKey": "Env",
"ParameterValue": "test"
},
{
"ParameterKey": "AppName",
"ParameterValue": "neptune-data-pipeline-graph"
},
{
@taufiqibrahim
taufiqibrahim / data-pipeline-service-graph-using-aws-neptune--neptune-cfn.yaml
Last active March 12, 2021 08:27
Data Pipeline Service Graph Using AWS Neptune - Cloudformation Template
# /*
# * Copyright 2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
# *
# * Permission is hereby granted, free of charge, to any person obtaining a copy of this
# * software and associated documentation files (the "Software"), to deal in the Software
# * without restriction, including without limitation the rights to use, copy, modify,
# * merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
# * permit persons to whom the Software is furnished to do so.
# *
# * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
@taufiqibrahim
taufiqibrahim / serverless_kafka_lambda-2-requirements.txt
Last active December 2, 2020 17:03
serverless_kafka_lambda-2-requirements.txt
avro
confluent-kafka[avro,json]
pendulum
psycopg2-binary
pymysql
sqlalchemy
@taufiqibrahim
taufiqibrahim / serverless_kafka_lambda-1-initialize-serverless-project.sh
Last active December 2, 2020 17:05
serverless_kafka_lambda-1-initialize-serverless-project.sh
# install serverless
npm install -g serverless
# creating serverless project
serverless create \
--template aws-python3 \
--name serverless-etl-sap-tms \
--path serverless-etl-sap-tms
@taufiqibrahim
taufiqibrahim / mytopic-schema.json
Last active May 27, 2020 13:20
mytopic-schema.json
{
"type": "object",
"title": "MyTopic",
"properties": {
"o_id": {"type": ["number", "null"], "multipleOf": 1.0},
"o_sap_ts": {"type": ["string", "null"]},
"o_so_key": {"type": ["string", "null"]},
"o_ordertypecode": {"type": ["string", "null"]},
"o_salesorderno": {"type": ["string", "null"]},
"o_salesofficecode": {"type": ["string", "null"]},
SELECT * FROM my_table;
@taufiqibrahim
taufiqibrahim / test.sql
Created May 16, 2020 14:20
Test Medium
SELECT * FROM table_name
@taufiqibrahim
taufiqibrahim / order_billing_aggregate_rdbms.sql
Last active May 16, 2020 14:07
Apache Flink: SQL Client Walkthrough
SELECT
do_no,
SUM(amount) AS amount_agg,
SUM(selling_amount) AS selling_amount_agg,
SUM(discount_amount) AS discount_amount_agg
FROM order_billing
GROUP BY
do_no