rponte

fair, fine grained, and deadlock free locking

first, a spin lock

you know what a spinlock is, right?

here's a spinlock:

rponte

-- (This code was run in PostgreSQL 9.6.1)
-- Demonstration of how serializable isolation for PostgreSQL, which detects possible
-- interference between concurrent transactions, can produce false positives

-- in psql, create the following table

CREATE TABLE users(
  id SERIAL NOT NULL PRIMARY KEY,
  username VARCHAR NOT NULL
);

rponte

Race conditions, locking e bancos de dados relacionais

Pretendo apresentar um código com uma lógica de negócio simples que funciona bem na perspectiva do negócio mas que quebra miseravelmente em ambientes minimamente concorrentes. Para isso, vou demonstrar através de testes de integração como identificar o Race Condition no código e principalmente como resolvê-lo através de mecanismos de locking e Isolation Level do seu banco de dados;

O que vamos discutir

1. Por que estudamos esse assunto mais a fundo?
   • 1.1. Bootcamp Seniors
   • 1.2. Handora: Curso CRUD e Atualizações Concorrentes com JPA/Hibernate - by Jordi Silva

• 1.3. A importância do treino: o que eu levei alguns bons anos para aprender o Jordi levou 6 meses

rponte

import org.slf4j.LoggerFactory
import org.springframework.jdbc.core.JdbcTemplate
import org.springframework.stereotype.Component
import org.springframework.transaction.annotation.Propagation
import org.springframework.transaction.annotation.Transactional
import java.time.Duration

interface LockManager {
    fun <T> tryWithLock(key: Long, timeout: Duration, function: () -> T): T
}

rponte

title
Elements of Messaging and Eventing Platforms

This document provides a brief overview of the essential elements of a messaging and eventing platform and how they relate to each other.

Message and Event Broker Categories

rponte

Turning PostgreSQL into a queue serving 10,000 jobs per second

RDBMS-based job queues have been criticized recently for being unable to handle heavy loads. And they deserve it, to some extent, because the queries used to safely lock a job have been pretty hairy. SELECT FOR UPDATE followed by an UPDATE works fine at first, but then you add more workers, and each is trying to SELECT FOR UPDATE the same row (and maybe throwing NOWAIT in there, then catching the errors and retrying), and things slow down.

On top of that, they have to actually update the row to mark it as locked, so the rest of your workers are sitting there waiting while one of them propagates its lock to disk (and the disks of however many servers you're replicating to). QueueClassic got some mileage out of the novel idea of randomly picking a row near the front of the queue to lock, but I can't still seem to get more than an an extra few hundred jobs per second out of it under heavy load.

So, many developers have started going straight t

rponte

-- Let's say you have a table full of work:

CREATE TABLE tasks (
       id UUID PRIMARY KEY NOT NULL DEFAULT gen_random_uuid(),
       status TEXT NOT NULL DEFAULT 'pending',
       payload JSON NOT NULL, -- or just have meaningful columns!
       created_at TIMESTAMP NOT NULL DEFAULT NOW()
);

rponte

package cz.monetplus.mnsp.tools.misc;

import org.apache.commons.codec.DecoderException;
import org.apache.commons.codec.binary.Hex;
import org.apache.commons.lang.StringUtils;

/**
 * Tools for encoding a decoding pinblock
 * 
 * @author Tomas Jacko <tomas.jacko [at] monetplus.cz>

rponte

Things I believe

This is a collection of the things I believe about software development. I have worked for years building backend and data processing systems, so read the below within that context.

Agree? Disagree? Feel free to let me know at @JanStette. See also my blog at www.janvsmachine.net.

Fundamentals

Keep it simple, stupid. You ain't gonna need it.

rponte

#!/bin/bash -eu
if [ -z "${LIB_ROBUST_BASH_SH:-}" ]; then
  LIB_ROBUST_BASH_SH=included

  function error {
    echo "Error:  ${1:-}"
  }

  # Check to see that we have a required binary on the path
  function require_binary {