Skip to content

Instantly share code, notes, and snippets.

View rizplate's full-sized avatar

Whats in the name rizplate

View GitHub Profile
@rizplate
rizplate / deploy_with_ebcli3_on_circleci.md
Created April 20, 2018 02:41 — forked from RobertoSchneiders/deploy_with_ebcli3_on_circleci.md
Settings to deploy to AWS Elastic Beanstalk on CircleCi (EB Cli 3)

This is how I configured the deploy of my rails apps to AWS Elastic Beanstalk through CircleCI.

Configure Environments Variables

On Project Settings > Environment Variables add this keys:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY
    The aws user must have the right permissions. This can be hard, maybe, this can help you.

Create a bash script to create the eb config file

from sqlalchemy import create_engine
engine = create_engine('sqlite:///:memory:', echo=True)
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from sqlalchemy import Column, Integer, String, Float
class User(Base):
__tablename__ = 'users'
@rizplate
rizplate / boto3_emr_create_cluster_with_wordcount_step.py
Created March 27, 2018 00:58 — forked from ruanbekker/boto3_emr_create_cluster_with_wordcount_step.py
Create EMR Cluster with a Wordcount Job as a Step in Boto3
import boto3
client = boto3.client(
'emr',
region_name='eu-west-1'
)
cmd = "hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar wordcount file:///etc/services /output"
emrcluster = client.run_job_flow(
@rizplate
rizplate / spark_aws_lambda.py
Created March 27, 2018 00:56 — forked from tomron/spark_aws_lambda.py
Example of python code to submit spark process as an emr step to AWS emr cluster in AWS lambda function
import sys
import time
import boto3
def lambda_handler(event, context):
conn = boto3.client("emr")
# chooses the first cluster which is Running or Waiting
# possibly can also choose by name or already have the cluster id
clusters = conn.list_clusters()