This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(ns foami.core | |
"FOreign Asynchronous Mechanism Interop" | |
(:require [clojure.core.async :as async])) | |
(defn put! | |
"Takes a `ch`, a `msg`, a single arg function that when passed `true` enables backpressure | |
and when passed `false` disables it, and a no-arg function which, when invoked, closes the | |
upstream source." | |
[ch msg backpressure! close!] | |
(let [status (atom :sending] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(ns foami.core | |
"FOreign Asynchronous Mechanism Interop" | |
(:require [clojure.core.async :as async])) | |
(def ^:private abandon (doto (async/chan) async/close!)) | |
(def ^:private pending-writes (async/chan)) | |
(defn put! | |
"Tries to put a val into chan, returns either: true if put succeeded, false if chan is |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
; there are some obvious micro optimizations, I left them out for clarity (see the other file for an optimized version) | |
; the relative ordering of read and writes with volatile and plain array should be thread-safe (if not, point it out) | |
; @wagjo asked "Have you found use for such concept? Must be pretty slow compared to unchunked one" | |
; The idea came out of a discussion on transducers so not used for real, yet. | |
; Once you optimize it (remove the boxing induced by the volatile, switch to unchecked math) there should not be much | |
; of an overhead. | |
; When you have one big composed reducing fn (eg several mapping stages) then each item is going to be processed in | |
; each stage, each stage of the mapping may evict from the data cache stuff used by previous stages. So you have cache | |
; misses for each item. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
upstream api_example_net { | |
server api.example.net:4000; | |
keepalive 600; | |
} | |
proxy_cache_path /var/cache/nginx/tag levels=1:2 keys_zone=tag:10m inactive=1d max_size=10g; | |
server { | |
listen 80; | |
server_name api.example.net; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(comment ; Fun with transducers, v2 | |
;; Still haven't found a brief + approachable overview of Clojure 1.7's new | |
;; transducers in the particular way I would have preferred myself - so here goes: | |
;;;; Definitions | |
;; Looking at the `reduce` docstring, we can define a 'reducing-fn' as: | |
(fn reducing-fn ([]) ([accumulation next-input])) -> new-accumulation | |
;; (The `[]` arity is actually optional; it's only used when calling | |
;; `reduce` w/o an init-accumulator). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(ns weighted-rand | |
(:import clojure.lang.PersistentQueue)) | |
(defprotocol Rand | |
(nextr [_ rng])) | |
;; Vose's alias method | |
;; http://www.keithschwarz.com/darts-dice-coins/ | |
(deftype Vose [n ^ints alias ^doubles prob] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
(require '[clojure.core.async :as a]) | |
(def xform (comp (map inc) | |
(filter even?) | |
(dedupe) | |
(flatmap range) | |
(partition-all 3) | |
(partition-by #(< (apply + %) 7)) | |
(flatmap flatten) | |
(random-sample 1.0) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RATE_LIMIT_SCRIPT = r''' | |
local now = tonumber(ARGV[1]) | |
local required = tonumber(ARGV[2]) | |
local rate = tonumber(ARGV[3]) | |
local per_secs = tonumber(ARGV[4]) | |
local do_subtract = tonumber(ARGV[5]) == 1 | |
local full_at = tonumber(redis.call('GET', KEYS[1])) or 0 | |
local score, result | |
if full_at < now then | |
score = rate |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
''' | |
rate_limit.py | |
Written May 7-8, 2014 by Josiah Carlson | |
Released under the MIT license. | |
Offers a simple interface for offering rate limiting on a per second, minute, | |
hour, and day basis. Useful for offering varying rate limits depending on user | |
behavior. Sliding window based limits offer per minute, hour, and day limits |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// more config options at http://hc.apache.org/httpcomponents-asyncclient-4.0.x/httpasyncclient/examples/org/apache/http/examples/nio/client/AsyncClientConfiguration.java | |
public RequestRunner provideRequestRunner( ComponentContext context, URI pageUri, Iterable<URI> assetUris ) | |
throws IOException | |
{ | |
try | |
{ | |
IOReactorConfig ioReactorConfig = IOReactorConfig.custom() | |
.setIoThreadCount( Runtime.getRuntime().availableProcessors() ) | |
.setConnectTimeout( 30_000 ) |