Skip to content

Instantly share code, notes, and snippets.

View cvargas-xbrein's full-sized avatar
🎯

Cristian Vargas cvargas-xbrein

🎯
View GitHub Profile
@llimllib
llimllib / jenks2.py
Last active April 3, 2023 19:26
The jenks algorithm in python. Since replaced with: https://github.com/llimllib/jenks-python which has tests and stuff
import json
from pprint import pprint as pp
def jenks_matrices_init(data, n_classes):
#fill the matrices with data+1 arrays of n_classes 0s
lower_class_limits = []
variance_combinations = []
for i in xrange(0, len(data)+1):
temp1 = []
temp2 = []
@nicor88
nicor88 / bootstrap_jupyter.sh
Created April 20, 2017 10:23
Bootstrap action to install Conda and Jupyter on EMR
#!/usr/bin/env bash
set -x -e
JUPYTER_PASSWORD=${1:-"myJupyterPassword"}
NOTEBOOK_DIR=${2:-"s3://myS3Bucket/notebooks/"}
# home backup
if [ ! -d /mnt/home_backup ]; then
sudo mkdir /mnt/home_backup
sudo cp -a /home/* /mnt/home_backup
@cosmincatalin
cosmincatalin / install-jupyter.sh
Last active April 17, 2023 14:23
AWS EMR bootstraps to install Jupyter (R, SparkR, Python 2, Python 3, PySpark)
#!/bin/bash
MINICONDA_VERSION="4.3.21"
PANDAS_VERSION="0.20.3"
SCIKIT_VERSION="0.19.0"
while [[ $# > 1 ]]; do
key="$1"
case $key in
@hervenivon
hervenivon / awazon-linux-gdal-installation.sh
Last active April 15, 2022 11:48
Install GEOS, PROJ4 & GDAL on amazon linux
export PYTHON_VERSION=3.4.3
export PYTHON_SHORT_VERSION=3.4
export GEOS_VERSION=3.6.2
export GDAL_VERSION=2.2.2
export PROJ4_VERSION=4.9.3
sudo yum-config-manager --enable epel
sudo yum install gdal-python
sudo yum -y install make automake gcc gcc-c++ libcurl-devel proj-devel geos-devel
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
from pyspark.sql.functions import *
from awsglue.dynamicframe import DynamicFrame
@slowkow
slowkow / install-units.md
Last active September 8, 2024 02:54
Install the 'units' R package on Partners

Summary

I had a difficult time installing the units R package on the Partners ERIS servers.

I hope this post helps you to figure out how to work around the errors.

Instructions

@tilakpatidar
tilakpatidar / pyspark_jdbc_df_count.md
Last active May 10, 2022 13:34
Gist to perform count() on jdbc sources without re-reading the df

Postgres snippet

create database test_db;

create table t_random as select s, md5(random()::text) from generate_Series(1,5000) s;

Pyspark snippet

In [1]: df=spark.read.jdbc(url="jdbc:postgresql://localhost:5432/test_db", table="t_random", properties={"driver": "org.postgresql.Driver"}).repartition(10)
@wachunei
wachunei / ✨ Ñoñerías para Fintual.md
Last active January 19, 2022 23:02
Una recopilación de ñoñerías chicas que he hecho como usuario de Fintual

✨ Ñoñerías para Fintual

Una recopilación de ñoñerías chicas que he hecho como usuario de Fintual.

Importante: no formo parte del equipo de fintual, todo lo que está acá no es responsabilidad de ellos. Todo lo que está acá puede fallar si el sitio o API de fintual cambia.


Contenido