Skip to content

Instantly share code, notes, and snippets.

View aluxian's full-sized avatar

Alexandru Rosianu aluxian

View GitHub Profile
@chanks
chanks / gist:7585810
Last active January 10, 2025 03:03
Turning PostgreSQL into a queue serving 10,000 jobs per second

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

@rab
rab / .gitconfig
Last active May 14, 2024 07:05
A good starting point for ~/.gitconfig
# -*- Conf -*-
[color]
branch = auto
diff = auto
status = auto
showbranch = auto
ui = true
# color.branch
# A boolean to enable/disable color in the output of git-branch(1). May be set to always, false (or
@sos4nt
sos4nt / xterm-256color-italic.terminfo
Created July 27, 2012 12:13
A xterm-256color based TERMINFO that adds the escape sequences for italic
# A xterm-256color based TERMINFO that adds the escape sequences for italic.
#
# Install:
#
# tic xterm-256color-italic.terminfo
#
# Usage:
#
# export TERM=xterm-256color-italic
#
@simme
simme / Install_tmux
Created October 19, 2011 07:55
Install and configure tmux on Mac OS X
# First install tmux
brew install tmux
# For mouse support (for switching panes and windows)
# Only needed if you are using Terminal.app (iTerm has mouse support)
Install http://www.culater.net/software/SIMBL/SIMBL.php
Then install https://bitheap.org/mouseterm/
# More on mouse support http://floriancrouzat.net/2010/07/run-tmux-with-mouse-support-in-mac-os-x-terminal-app/
@benatkin
benatkin / Global.sublime-settings
Created July 20, 2011 04:26
excluding node_modules from Sublime Text 2
// Place user-specific overrides in this file, to ensure they're preserved
// when upgrading
{
"folder_exclude_patterns": [".svn", ".git", ".hg", "CVS", "node_modules"]
}
@jorgeortiz85
jorgeortiz85 / PrivateMethodCaller.scala
Created April 7, 2011 15:41
Calling private methods in Scala
// Usage:
// p(instance)('privateMethod)(arg1, arg2, arg3)
class PrivateMethodCaller(x: AnyRef, methodName: String) {
def apply(_args: Any*): Any = {
val args = _args.map(_.asInstanceOf[AnyRef])
def _parents: Stream[Class[_]] = Stream(x.getClass) #::: _parents.map(_.getSuperclass)
val parents = _parents.takeWhile(_ != null).toList
val methods = parents.flatMap(_.getDeclaredMethods)
val method = methods.find(_.getName == methodName).getOrElse(throw new IllegalArgumentException("Method " + methodName + " not found"))
object GeoHash{
val base32 = "0123456789bcdefghjkmnpqrstuvwxyz"
def encode( longtitude:Double, latitude:Double, range:Double) = {
def asBit( d:Double,max:Double,min:Double ) : List[Boolean] = {
val mid = ( max + min ) / 2
var res = ( d >= mid )
val (m,n) = if( res ) (max,mid) else (mid,min)
if( Math.abs( m - n ) <= range )
res :: Nil