Skip to content

Instantly share code, notes, and snippets.

View rizplate's full-sized avatar

Whats in the name rizplate

View GitHub Profile
@rizplate
rizplate / spark_aws_lambda.py
Created March 27, 2018 00:56 — forked from tomron/spark_aws_lambda.py
Example of python code to submit spark process as an emr step to AWS emr cluster in AWS lambda function
import sys
import time
import boto3
def lambda_handler(event, context):
conn = boto3.client("emr")
# chooses the first cluster which is Running or Waiting
# possibly can also choose by name or already have the cluster id
clusters = conn.list_clusters()
@rizplate
rizplate / boto3_emr_create_cluster_with_wordcount_step.py
Created March 27, 2018 00:58 — forked from ruanbekker/boto3_emr_create_cluster_with_wordcount_step.py
Create EMR Cluster with a Wordcount Job as a Step in Boto3
import boto3
client = boto3.client(
'emr',
region_name='eu-west-1'
)
cmd = "hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount file:///etc/services /output"
emrcluster = client.run_job_flow(
from sqlalchemy import create_engine
engine = create_engine('sqlite:///:memory:', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy import Column, Integer, String, Float
class User(Base):
__tablename__ = 'users'
@rizplate
rizplate / deploy_with_ebcli3_on_circleci.md
Created April 20, 2018 02:41 — forked from RobertoSchneiders/deploy_with_ebcli3_on_circleci.md
Settings to deploy to AWS Elastic Beanstalk on CircleCi (EB Cli 3)

This is how I configured the deploy of my rails apps to AWS Elastic Beanstalk through CircleCI.

Configure Environments Variables

On Project Settings > Environment Variables add this keys:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
    The aws user must have the right permissions. This can be hard, maybe, this can help you.

Create a bash script to create the eb config file

@rizplate
rizplate / app.py
Created April 23, 2018 20:19 — forked from mattbennett/app.py
Nameko celery integration
from flask import Flask, request
from nameko.standalone.rpc import ServiceRpcProxy
app = Flask(__name__)
@app.route('/')
def task_list():
return """
<html>
@rizplate
rizplate / infra-secret-management-overview.md
Created May 3, 2018 19:57 — forked from maxvt/infra-secret-management-overview.md
Infrastructure Secret Management Software Overview

Currently, there is an explosion of tools that aim to manage secrets for automated, cloud native infrastructure management. Daniel Somerfield did some work classifying the various approaches, but (as far as I know) no one has made a recent effort to summarize the various tools.

This is an attempt to give a quick overview of what can be found out there. The list is alphabetical. There will be tools that are missing, and some of the facts might be wrong--I welcome your corrections. For the purpose, I can be reached via @maxvt on Twitter, or just leave me a comment here.

There is a companion feature matrix of various tools. Comments are welcome in the same manner.

@rizplate
rizplate / README.md
Created May 13, 2018 00:18
AltcoinGenerator

Altcoin Generator

Easiest way to create your own cryptocurrency.

What does this script do?

This script is an experiment to generate new cryptocurrencies (altcoins) based on litecoin. It will help you creating a git repository with minimal required changes to start your new coin and blockchain.

What do I have to do?

@rizplate
rizplate / README.md
Last active May 13, 2018 00:18
AltcoinGenerator

Altcoin Generator

Easiest way to create your own cryptocurrency.

What does this script do?

This script is an experiment to generate new cryptocurrencies (altcoins) based on litecoin. It will help you creating a git repository with minimal required changes to start your new coin and blockchain.

What do I have to do?

Awesome Blockchain

Curated list of awesome bitcoin blockchain services:

Assets

  • Coinspark - Upgrade your bitcoin with messaging and assets
  • Coinprism - Use the Bitcoin blockchain with any kind of asset
  • Colu.co - Creating, Storing and Managing Digital Assets On Top of the Bitcoin Blockchain
  • Chronicled - Collect and trade 100% authentic products
@rizplate
rizplate / odo_csv_upload.py
Created May 17, 2018 22:29 — forked from bobquest33/odo_csv_upload.py
100 Scripts in 30 Days challenge: Script 2 & 3 — Loading a 3 GB csv file into DB using Pandas & Odo
Statsitics:
pandas_csv_upoad.py
Timing: 01:59:36.97 (1 Hour, 59 Mins, 36.97 Sec)
odo_csv_upload.py
Timing: 00:30:12.96 (30 Mins, 12.96 Sec)