Skip to content

Instantly share code, notes, and snippets.

@mieitza
mieitza / mongodb_2_pandas.py
Created January 3, 2018 15:05 — forked from jmquintana79/mongodb_2_pandas.py
Functions to connect and read mongodb data to pandas df
import pandas as pd
from pymongo import MongoClient
# set connection with mongodb
def _connect_mongo(host, port, username, password, db):
""" A util for making a connection to mongo """
if username and password:
mongo_uri = 'mongodb://%s:%s@%s:%s/%s' % (username, password, host, port, db)
conn = MongoClient(mongo_uri)
@mieitza
mieitza / pyspark_to_elasticsearch.py
Created January 3, 2018 15:04 — forked from adrianva/pyspark_to_elasticsearch.py
Save RDD and/or DataFrame from Spark to Elasticsearch
# Elastic configs
es_read_conf = {
"es.nodes" : "localhost",
"es.port" : "9200",
"es.resource" : "twitter/tweet"
}
es_write_conf = {
"es.nodes" : "localhost",
"es.port" : "9200",
@mieitza
mieitza / pyspark_to_elasticsearch.py
Created January 3, 2018 15:04 — forked from adrianva/pyspark_to_elasticsearch.py
Save RDD and/or DataFrame from Spark to Elasticsearch
# Elastic configs
es_read_conf = {
"es.nodes" : "localhost",
"es.port" : "9200",
"es.resource" : "twitter/tweet"
}
es_write_conf = {
"es.nodes" : "localhost",
"es.port" : "9200",
@mieitza
mieitza / cmd_flag_parser.go
Created November 1, 2015 19:33 — forked from andreagrandi/cmd_flag_parser.go
Parse command line flags in Go
package main
import (
"flag"
"fmt"
)
var hostName = flag.String("host", "localhost", "Hostname or IP you want to run this service on")
var portNumber = flag.Int("port", 8080, "Port you want this service to listen on (default 8080)")
@mieitza
mieitza / substitute_string
Created November 1, 2015 19:32 — forked from andreagrandi/substitute_string
Substitute a placeholder like {{hostname}} that is inside a file, with the output of the hostname command.
sed -i "s/{{hostname}}/`hostname`/g" /etc/supervisor/conf.d/celeryd.conf
import commands
import time
import sys
from socket import socket
import argparse
import os
import pymongo
from pymongo import Connection
import yaml