Skip to content

Instantly share code, notes, and snippets.

@kovid-rathee
kovid-rathee / mysql_user_backup.py
Last active February 28, 2017 08:01
Backup MySQL User Table using Python
#!/usr/bin/env python
import os
import subprocess
from datetime import datetime
# Directory where the backup will be stored -- this will be changed on production
directory = "/usr/local/backups/mysql_user/"
# Generating a script to fetch grants for all users
grants = "select concat('show grants for ''',user,'''@''',host,''';') from mysql.user where user <> '';"
@kovid-rathee
kovid-rathee / mysql_redshift.py
Last active February 11, 2017 16:00
Python Script to Export Data into a File from MySQL on the MySQL machine and Upload it to S3 and Copy it to Amazon Redshift. Simple as that. Plus, email notifications. This is just a skeleton.
#!/usr/bin/env python
import os
import sys
import shlex
import boto3
import shutil
import logging
import urllib2
import botocore
@kovid-rathee
kovid-rathee / apache_role.yml
Created February 15, 2017 19:19
Basic Ansible Role to install and configure Apache HTTP Server
---
- name: install apache
yum: name=httpd state=present
- name: insert firewalld rule for httpd
firewalld: port={{httpd_port}}/tcp permanent=true state=enabled immediate=yes
- name: start and enable apache
service: name=httpd state=started enabled=yes
@kovid-rathee
kovid-rathee / count_vectorizer_pandas.py
Created May 27, 2017 12:09
How to vectorize sentences using a Pandas and sklearn's CountVectorizer
import pandas as pd
from sklearn.feature_extraction.text import CountVectorizer
vectorizer = CountVectorizer()
corpus = [ 'This is a sentence',
'Another sentence is here',
'Wait for another sentence',
'The sentence is coming',
'The sentence has come'
]
@kovid-rathee
kovid-rathee / gist:7aa4b1595a5e35d510748b5d04458cbc
Created May 30, 2017 14:13 — forked from bryhal/gist:4129042
MYSQL: Generate Calendar Table
DROP TABLE IF EXISTS time_dimension;
CREATE TABLE time_dimension (
id INTEGER PRIMARY KEY, -- year*10000+month*100+day
db_date DATE NOT NULL,
year INTEGER NOT NULL,
month INTEGER NOT NULL, -- 1 to 12
day INTEGER NOT NULL, -- 1 to 31
quarter INTEGER NOT NULL, -- 1 to 4
week INTEGER NOT NULL, -- 1 to 52/53
day_name VARCHAR(9) NOT NULL, -- 'Monday', 'Tuesday'...
@kovid-rathee
kovid-rathee / get_started_python_on_mac.txt
Created June 2, 2017 09:28
How to get started with Python on a Mac machine.
1. Install pip on your system.
-- curl https://bootstrap.pypa.io/ez_setup.py -o - | sudo python
-- sudo easy_install pip
2. Check python version
-- python --version
3. Install PyCharm
-- https://www.jetbrains.com/pycharm/
-- Create a project in PyCharm -- Select an environment if you have multiple Python installations
4. Install Libraries
-- pip install MySQL-python (for MySQL only)
@kovid-rathee
kovid-rathee / distance.sql
Created August 8, 2017 03:33 — forked from Usse/distance.sql
MySQL calculate distance between two latitude/longitude coordinates
CREATE FUNCTION `lat_lng_distance` (lat1 FLOAT, lng1 FLOAT, lat2 FLOAT, lng2 FLOAT)
RETURNS FLOAT
DETERMINISTIC
BEGIN
RETURN 6371 * 2 * ASIN(SQRT(
POWER(SIN((lat1 - abs(lat2)) * pi()/180 / 2),
2) + COS(lat1 * pi()/180 ) * COS(abs(lat2) *
pi()/180) * POWER(SIN((lng1 - lng2) *
pi()/180 / 2), 2) ));
END
@kovid-rathee
kovid-rathee / vpn.md
Created November 5, 2017 09:48 — forked from joepie91/vpn.md
Don't use VPN services.

Don't use VPN services.

No, seriously, don't. You're probably reading this because you've asked what VPN service to use, and this is the answer.

Note: The content in this post does not apply to using VPN for their intended purpose; that is, as a virtual private (internal) network. It only applies to using it as a glorified proxy, which is what every third-party "VPN provider" does.

(A Russian translation of this article can be found here, contributed by Timur Demin.)

Why not?

@kovid-rathee
kovid-rathee / postgres-cheatsheet.md
Created January 5, 2018 06:25 — forked from Kartones/postgres-cheatsheet.md
PostgreSQL command line cheatsheet

PSQL

Magic words:

psql -U postgres

Some interesting flags (to see all, use -h):

  • -E: will describe the underlaying queries of the \ commands (cool for learning!)
  • -l: psql will list all databases and then exit (useful if the user you connect with doesn't has a default database, like at AWS RDS)
@kovid-rathee
kovid-rathee / mysql_kill_long_processes.sql
Created May 10, 2018 04:15
Kill Long Processes in MySQL to prevent Slave Lag and reduce read/write locks.
drop procedure if exists your_schema.kill_long_processes;
delimiter ;;
create procedure your_schema.kill_long_processes (p_long_query_time int(10), p_list_of_users varchar(255), p_kill_write_queries_too tinyint(1))
begin
if find_in_set ('admin', p_list_of_users)
or find_in_set ('root', p_list_of_users)
or find_in_set ('event_scheduler', p_list_of_users)
or find_in_set('system user', p_list_of_users) then
select 'You are not authorized to perform this action!';
else