Skip to content

Instantly share code, notes, and snippets.

@sany2k8
sany2k8 / install virtualenv ubuntu 16.04.md
Created September 26, 2019 07:31 — forked from frfahim/install virtualenv ubuntu 16.04.md
How to install virtual environment on ubuntu 16.04

How to install virtualenv:

Install pip first

sudo apt-get install python3-pip

Then install virtualenv using pip3

sudo pip3 install virtualenv 
@sany2k8
sany2k8 / how-to-copy-aws-rds-to-local.md
Created September 3, 2019 07:22 — forked from syafiqfaiz/how-to-copy-aws-rds-to-local.md
How to copy production database on AWS RDS(postgresql) to local development database.
  1. Change your database RDS instance security group to allow your machine to access it.
    • Add your ip to the security group to acces the instance via Postgres.
  2. Make a copy of the database using pg_dump
    • $ pg_dump -h <public dns> -U <my username> -f <name of dump file .sql> <name of my database>
    • you will be asked for postgressql password.
    • a dump file(.sql) will be created
  3. Restore that dump file to your local database.
    • but you might need to drop the database and create it first
    • $ psql -U <postgresql username> -d <database name> -f <dump file that you want to restore>
  • the database is restored
@sany2k8
sany2k8 / s3.sh
Created July 4, 2019 12:06 — forked from chrismdp/s3.sh
Uploading to S3 in 18 lines of Shell (used to upload builds for http://soltrader.net)
# You don't need Fog in Ruby or some other library to upload to S3 -- shell works perfectly fine
# This is how I upload my new Sol Trader builds (http://soltrader.net)
# Based on a modified script from here: http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash
S3KEY="my aws key"
S3SECRET="my aws secret" # pass these in
function putS3
{
path=$1
@sany2k8
sany2k8 / FullExample.php
Created March 11, 2019 13:00 — forked from IamSmith/FullExample.php
Fully working example
<?php
require_once("PHPExcel/PHPExcel.php");
$phpExcel = new PHPExcel();
$styleArray = array(
'font' => array(
'bold' => true,
)
);
@sany2k8
sany2k8 / access_postgresql_with_docker.md
Created March 11, 2019 10:10 — forked from MauricioMoraes/access_postgresql_with_docker.md
Allow Docker Container Access to Host's Postgres Database on linux (ubuntu)

You have to do 2 things in order to allow your container to access your host's postgresql database

  1. Make your postgresql listen to an external ip address
  2. Let this client ip (your docker container) access your postgresql database with a given user

Obs: By "Host" here I mean "the server where docker is running on".

Make your postgresql listen to an external ip address

Find your postgresql.conf (in case you don't know where it is)

$ sudo find / -type f -name postgresql.conf

@sany2k8
sany2k8 / gist:bad3927a7c37c51d518903719190e9d2
Created February 12, 2019 02:14 — forked from drorata/gist:146ce50807d16fd4a6aa
Minimal Working example of Elasticsearch scrolling using Python client
# Initialize the scroll
page = es.search(
index = 'yourIndex',
doc_type = 'yourType',
scroll = '2m',
search_type = 'scan',
size = 1000,
body = {
# Your query's body
})
@sany2k8
sany2k8 / csv_to_elastic_search_bulk_insert.py
Created January 31, 2019 17:14 — forked from clemsos/csv_to_elastic_search_bulk_insert.py
Elastic Search : index large csv files with Python Pandas
from pyelasticsearch import ElasticSearch
import pandas as pd
from time import time
root_path="/home/clemsos/Dev/mitras/"
raw_data_path=root_path+"data/"
csv_filename="week10.csv"
t0=time()
@sany2k8
sany2k8 / import_csv_to_db.py
Created December 30, 2018 14:56 — forked from weaming/import_csv_to_db.py
import_csv_to_db
# coding: utf8
"""
Goal: import csv to database.
pip install pandas, numpy
"""
import pandas
import numpy as np
@sany2k8
sany2k8 / useful_pandas_snippets.py
Last active December 29, 2018 16:32 — forked from fomightez/useful_pandas_snippets.py
Useful Pandas Snippets
# -*- coding: utf-8 -*-
# import pandas and numpy package
import numpy as np
import pandas as pd
# List unique values in a DataFrame column
df['Column Name'].unique()
# To extract a specific column (subset the dataframe), you can use [ ] (brackets) or attribute notation.