Skip to content

Instantly share code, notes, and snippets.

@kovid-rathee
kovid-rathee / count_vectorizer_pandas.py
Created May 27, 2017 12:09
How to vectorize sentences using a Pandas and sklearn's CountVectorizer
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
vectorizer = CountVectorizer()
corpus = [ 'This is a sentence',
'Another sentence is here',
'Wait for another sentence',
'The sentence is coming',
'The sentence has come'
]
@kovid-rathee
kovid-rathee / apache_role.yml
Created February 15, 2017 19:19
Basic Ansible Role to install and configure Apache HTTP Server
---
- name: install apache
yum: name=httpd state=present
- name: insert firewalld rule for httpd
firewalld: port={{httpd_port}}/tcp permanent=true state=enabled immediate=yes
- name: start and enable apache
service: name=httpd state=started enabled=yes
@kovid-rathee
kovid-rathee / mysql_redshift.py
Last active February 11, 2017 16:00
Python Script to Export Data into a File from MySQL on the MySQL machine and Upload it to S3 and Copy it to Amazon Redshift. Simple as that. Plus, email notifications. This is just a skeleton.
#!/usr/bin/env python
import os
import sys
import shlex
import boto3
import shutil
import logging
import urllib2
import botocore
@kovid-rathee
kovid-rathee / mysql_user_backup.py
Last active February 28, 2017 08:01
Backup MySQL User Table using Python
#!/usr/bin/env python
import os
import subprocess
from datetime import datetime
# Directory where the backup will be stored -- this will be changed on production
directory = "/usr/local/backups/mysql_user/"
# Generating a script to fetch grants for all users
grants = "select concat('show grants for ''',user,'''@''',host,''';') from mysql.user where user <> '';"
@kovid-rathee
kovid-rathee / closure_table.md
Created February 5, 2017 09:33
Persistent tree structure using closure table in MySQL

Using closure tables to manage hierarchical relations in MySQL

Create DB tables

Create a table to represent tree nodes.

CREATE TABLE `tree_node` (
    `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
    `data_body` text,

node_deleted datetime DEFAULT NULL,

@kovid-rathee
kovid-rathee / dump.sh
Created February 5, 2017 09:24 — forked from andsens/dump.sh
Backup all MySQL databases into separate files
#!/bin/sh
## backup each mysql db into a different file, rather than one big file
## as with --all-databases. This will make restores easier.
## To backup a single database simply add the db name as a parameter (or multiple dbs)
## Putting the script in /var/backups/mysql seems sensible... on a debian machine that is
## Create the user and directories
# mkdir -p /var/backups/mysql/databases
# useradd --home-dir /var/backups/mysql --gid backup --no-create-home mysql-backup
## Remember to make the script executable, and unreadable by others
@kovid-rathee
kovid-rathee / README.md
Created February 5, 2017 09:23 — forked from oodavid/README.md
Restore MySQL from Amazon S3

Restore MySQL from Amazon S3

This is a hands-on way to pull down a set of MySQL dumps from Amazon S3 and restore your database with it

Sister Document - Backup MySQL to Amazon S3 - read that first

1 - Set your MySQL password and S3 bucket, make a temp dir, get a list of snapshots

# Set our variables

export mysqlpass="ROOTPASSWORD"

@kovid-rathee
kovid-rathee / mysql-rename-db.sh
Created February 5, 2017 09:21 — forked from tadas-s/mysql-rename-db.sh
MySQL database rename script
#!/bin/bash
# Disclaimer - make backups, use at your own risk.
#
# Based on this comment: http://stackoverflow.com/a/13944924/843067
# Views and stored procedures have to be done separately.
OLDDB="old_db_name"
NEWDB="new_db_name"
MYSQL="mysql -u root -pyour_password "
@kovid-rathee
kovid-rathee / fetch_organize_instalooter.py
Created February 4, 2017 07:41
Python Script to fetch and organize Instagram photos and videos chronologically using instaLooter
import os
import sys
import subprocess
import instaLooter
basepath = "/Users/kovid.rathee/Desktop/instaLooter/"
accounts = ['rupikaur_','fursty','tropicalratchet','dylankato','asyrafacha','tomashavel','runawayueli','dreamingandwandering']
for account in accounts:
dir = basepath + account
print(os.path.isdir(dir))
if not os.path.exists(dir):
@kovid-rathee
kovid-rathee / rds2redshift.sh
Created February 3, 2017 21:15 — forked from erincerys/rds2redshift.sh
Loads a MySQL data dump into a Redshift table. Useful for AWS RDS instances. Dependent on external script to stream MySQL data to file, and postgres psql command line client.
PYTHON_PATH=$(which python)
PSQL_PATH=$(which psql)
MYSQL_SCRIPT='mysql2file.py'
MYSQL_SERVER=
MYSQL_PORT=3306
MYSQL_DATABASE=
MYSQL_USER=
MYSQL_PASSWORD=