Skip to content

Instantly share code, notes, and snippets.

View liwh's full-sized avatar
🎯
Focusing

robie lee liwh

🎯
Focusing
View GitHub Profile
@liwh
liwh / avazu_ftrl_concurrent.go
Created June 11, 2020 02:11 — forked from jack281291/avazu_ftrl_concurrent.go
Kaggle Avazu Challenge: FTRL-Proximal with L1 & L2 implemented in Go (Concurrent/Multi-threaded)
// Based on tinrtgu's Python script here:
// https://www.kaggle.com/c/avazu-ctr-prediction/forums/t/10927/beat-the-benchmark-with-less-than-1mb-of-memory
package main
import (
"encoding/csv"
"os"
"strconv"
"hash/fnv"
"math"
@liwh
liwh / elasticsearch_best_practices.txt
Created March 21, 2018 03:07 — forked from duydo/elasticsearch_best_practices.txt
Elasticsearch - Index best practices from Shay Banon
If you want, I can try and help with pointers as to how to improve the indexing speed you get. Its quite easy to really increase it by using some simple guidelines, for example:
- Use create in the index API (assuming you can).
- Relax the real time aspect from 1 second to something a bit higher (index.engine.robin.refresh_interval).
- Increase the indexing buffer size (indices.memory.index_buffer_size), it defaults to the value 10% which is 10% of the heap.
- Increase the number of dirty operations that trigger automatic flush (so the translog won't get really big, even though its FS based) by setting index.translog.flush_threshold (defaults to 5000).
- Increase the memory allocated to elasticsearch node. By default its 1g.
- Start with a lower replica count (even 0), and then once the bulk loading is done, increate it to the value you want it to be using the update_settings API. This will improve things as possibly less shards will be allocated to each machine.
- Increase the number of machines you have so
[core]
# The home folder for airflow, default is ~/airflow
airflow_home = /Users/p1nox/airflow
# The folder where your airflow pipelines live, most likely a
# subfolder in a code repository
dags_folder = /Users/p1nox/airflow/dags
# The folder where airflow should store its log files. This location
base_log_folder = /Users/p1nox/airflow/logs
- certain endpoints are always blocked
if nginx_uri == "/_access_token" or nginx_uri == "/_me" then
ngx.exit(403)
end
-- import requirements
local cjson = require "cjson"
-- setup some app-level vars
local app_id = "APP_ID"

Routes

小心地使用 Match(Rails 3 已实现)

Rails 3 提供了 match 方法供我们自定义 routes,然而我们要小心使用它以避免“跨站脚本攻击”(XSS Attack)。比如像这样的 routes:

注:(r3 代表 Rails 3,r4 代表 Rails 4)

# routes.rb

Fibur

Fibur is a library that allows concurrency during Ruby I/O operations without needing to make use of callback systems. Traditionally in Ruby, to achieve concurrency during blocking I/O operations, programmers would make use of Fibers and callbacks. Fibur eliminates the need for wrapping your I/O calls with Fibers and a callback. It allows you to write your blocking I/O calls the way you normally would, and still have concurrent execution during those I/O calls.

Example

Say you have a method that fetches data from a network resource:

@liwh
liwh / gist:2858648
Created June 2, 2012 14:32 — forked from snatchev/gist:1316470
resque worker devise not eager loading
❷ > QUEUE=* rake resque:work --trace
** Invoke resque:work (first_time)
** Invoke resque:preload (first_time)
** Invoke resque:setup (first_time)
** Execute resque:setup
** Execute resque:preload
rake aborted!
No such file to load -- devise/confirmations_controller
/Users/stefan/.rvm/gems/ruby-1.9.2-p290@my-rails-project/gems/activesupport-3.1.1/lib/active_support/dependencies.rb:306:in `rescue in depend_on'
/Users/stefan/.rvm/gems/ruby-1.9.2-p290@my-rails-project/gems/activesupport-3.1.1/lib/active_support/dependencies.rb:301:in `depend_on'
@liwh
liwh / sequel + fiveruns-memcache-client
Created April 20, 2012 08:21 — forked from EppO/sequel + fiveruns-memcache-client
trying to cache sequel models in memcached
$ sequel postgres://mylogin:mypasswd@localhost/bddtest
Your database is stored in DB...
>> require 'memcache'
=> true
>> MemCache
=> MemCache
>> MemCache::VERSION
=> "1.5.0.1"
>> CACHE = MemCache.new("localhost")
=> MemCache: 1 servers, 1 buckets, ns: nil, ro: false
@liwh
liwh / gh-like.css
Created October 25, 2011 08:28 — forked from robsimmons/gh-like.css
GitHub-esque syntax highlight CSS
/*
Some simple Github-like styles, with syntax highlighting CSS via Pygments.
*/
body{
font-family: helvetica, arial, freesans, clean, sans-serif;
color: #333;
background-color: #fff;
border: none;
line-height: 1.5;
margin: 2em 3em;
@liwh
liwh / rails_3_1_beta_1_changes.md
Created May 6, 2011 05:27 — forked from ryanb/rails_3_1_rc4_changes.md
The Changelogs for Rails 3.1 Beta 1

Railties 3.1 Beta 1

  • The -j option of the application generator accepts an arbitrary string. If passed "foo", the gem "foo-rails" is added to the Gemfile, and the application JavaScript manifest requires "foo" and "foo_ujs". As of this writing "prototype-rails" and "jquery-rails" exist and provide those files via the asset pipeline. Default is "jquery". [fxn]

  • jQuery is no longer vendored, it is provided from now on by the jquery-rails gem. [fxn]

  • Prototype and Scriptaculous are no longer vendored, they are provided from now on by the prototype-rails gem. [fxn]

  • The scaffold controller will now produce SCSS file if Sass is available [Prem Sichanugrist]