Skip to content

Instantly share code, notes, and snippets.

@phonglh79
phonglh79 / gist:e1a63db04b421cddb2f1bf7a9759c6ff
Created June 23, 2017 03:39 — forked from bobuss/gist:6515517
Tuning the Linux Kernel for many tcp connections

(from http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1)

Save yourself some time and tune the kernel tcp settings before testing with lots of connections, or your test will fail and you’ll see lots of Out of socket memory messages (and if you are masquerading, nf_conntrack: table full, dropping packet.)

Here are the sysctl settings I ended up with - YMMV, but these will probably do:

$ cat /etc/sysctl.conf
# General gigabit tuning:
net.core.rmem_max = 16777216
@phonglh79
phonglh79 / Bcache Tutorial
Created September 4, 2017 07:37 — forked from TheAnonymous/Bcache Tutorial
Making a SSD working as a Cache for a HDD with bcache.
There are 3 Methods for using a SSD as cache.
dm-cache, bcache,enhance-io all three should be avaliable in Kernel Version 3.10.
Performance differenceses are discussed here => http://lkml.indiana.edu/hypermail/linux/kernel/1306.1/01246.html
In this tutorial I will use bcache since dm-cache didn't worked for me.
At the moment there is only an rc5 candidate for kernel 3.10.0
How to compile it is explained below. If you have a newer kernel than 3.10 and it is bcache enabled you dont need to compile it by yourself. In this case just jump over the kernel-compile-exlanation. If your kernel is compiled with bcache support you can test with "modprobe bcache". If that doesnt throw an error it should work :)
I will link to the tutorials I used. If you found better ones please paste a link in the comments
How to compile a kernel by yourself is explained at:
=> http://bodhizazen.net/Tutorials/kernel
@phonglh79
phonglh79 / cors-nginx.conf
Created September 6, 2017 03:39 — forked from alexjs/cors-nginx.conf
Slightly tighter CORS config for nginx
#
# Slightly tighter CORS config for nginx
#
# A modification of https://gist.github.com/1064640/ to include a white-list of URLs
#
# Despite the W3C guidance suggesting that a list of origins can be passed as part of
# Access-Control-Allow-Origin headers, several browsers (well, at least Firefox)
# don't seem to play nicely with this.
#
@phonglh79
phonglh79 / cors-nginx.conf
Created September 6, 2017 03:39 — forked from michiel/cors-nginx.conf
Wide-open CORS config for nginx
#
# Wide-open CORS config for nginx
#
location / {
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
#
@phonglh79
phonglh79 / nginx.conf
Created September 7, 2017 04:46 — forked from plentz/nginx.conf
Best nginx configuration for improved security(and performance). Complete blog post here http://tautt.com/best-nginx-configuration-for-security/
# to generate your dhparam.pem file, run in the terminal
openssl dhparam -out /etc/nginx/ssl/dhparam.pem 2048
@phonglh79
phonglh79 / nginx.conf
Created September 22, 2017 08:13 — forked from kmjones1979/nginx.conf
Example NGINX configuration to route based on country code using GeoIP
# load dynamic modules
load_module /etc/nginx/modules/ngx_http_geoip_module.so;
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log info;
pid /var/run/nginx.pid;
events { worker_connections 1024; }
http {
@phonglh79
phonglh79 / Setup GeoIP with NginX & PHP
Created September 22, 2017 08:40 — forked from sureshsaggar/Setup GeoIP with NginX & PHP
Setup GeoIP with NginX & PHP
On my Ubuntu machine I located the GeoIP.dat file. If not available then download/intall the same.
root@localhost:~# locate GeoIP.dat
/usr/share/GeoIP/GeoIP.dat
Open Nginx configuration (/etc/nginx/nginx.conf) and specify <geoip_country> <path to GeoIP.dat>
line under the "http" block. Example block:
http {
# SS - meant to find country code from the client IP

The tests

fio full write:

fio --name=writefile --size=100G --filesize=100G --filename=/dev/sdX --bs=1M --nrfiles=1 --direct=1 --sync=0 --randrepeat=0 --rw=write --refill_buffers --end_fsync=1 --iodepth=200 --ioengine=libaio

fio random read:

fio --time_based --name=benchmark --size=100G --runtime=30 --filename=/dev/sdX --ioengine=libaio --randrepeat=0 --iodepth=128 --direct=1 --invalidate=1 --verify=0 --verify_fatal=0 --numjobs=4 --rw=randread --blocksize=4k --group_reporting
yum -y update
yum -y install net-tools
echo kernel.shmmax=209715200 >> /etc/sysctl.conf
yum -y install libaio numactl sysfsutils unzip wget
cd /opt/
wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz"
tar xzf jdk-8u112-linux-x64.tar.gz
cd /opt/jdk1.8.0_112/
alternatives --install /usr/bin/java java /opt/jdk1.8.0_112/bin/java 2
alternatives --config java
@phonglh79
phonglh79 / mongodb-s3-backup.sh
Created January 11, 2018 03:19 — forked from eladnava/mongodb-s3-backup.sh
Automatically backup a MongoDB database to S3 using mongodump, tar, and awscli (Ubuntu 14.04 LTS)
#!/bin/sh
# Make sure to:
# 1) Name this file `backup.sh` and place it in /home/ubuntu
# 2) Run sudo apt-get install awscli to install the AWSCLI
# 3) Run aws configure (enter s3-authorized IAM user and specify region)
# 4) Fill in DB host + name
# 5) Create S3 bucket for the backups and fill it in below (set a lifecycle rule to expire files older than X days in the bucket)
# 6) Run chmod +x backup.sh
# 7) Test it out via ./backup.sh