Skip to content

Instantly share code, notes, and snippets.

View icy's full-sized avatar
🦖
Available

Ky-Anh Huynh icy

🦖
Available
View GitHub Profile
#!/usr/bin/awk
BEGIN {
for (i = 0; i <= 255; i++) {
ord[sprintf("%c", i)] = i
}
}
function escape(str, c, len, res) {
len = length(str)
@unbracketed
unbracketed / export_repo_issues_to_csv.py
Last active August 3, 2023 18:13
Export Issues from Github repo to CSV (API v3)
"""
Exports Issues from a specified repository to a CSV file
Uses basic authentication (Github username + password) to retrieve Issues
from a repository that username has access to. Supports Github API v3.
"""
import csv
import requests
@duydo
duydo / elasticsearch_best_practices.txt
Last active June 20, 2024 09:59
Elasticsearch - Index best practices from Shay Banon
If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example:
- Use create in the index API (assuming you can).
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval).
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap.
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000).
- Increase the memory allocated to elasticsearch node. By default its 1g.
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine.
- Increase the number of machines you have so
@filterfish
filterfish / pipe-client.rb
Created December 2, 2011 23:55
EventMachine Pipe Clinet/Server
#!/usr/bin/env ruby
require 'pathname'
uri = ARGV[0] || (puts "usage: #{$0} <uri>"; exit 1)
Pathname.new('pipe').open("w") do |fd|
fd.puts(uri)
end
@didip
didip / supervisord-example.conf
Created January 30, 2011 05:10
Example configuration file for supervisord.conf
[unix_http_server]
file=/tmp/supervisor.sock ; path to your socket file
[supervisord]
logfile=/var/log/supervisord/supervisord.log ; supervisord log file
logfile_maxbytes=50MB ; maximum size of logfile before rotation
logfile_backups=10 ; number of backed up logfiles
loglevel=error ; info, debug, warn, trace
pidfile=/var/run/supervisord.pid ; pidfile location
nodaemon=false ; run supervisord as a daemon