Skip to content

Instantly share code, notes, and snippets.

View sorentwo's full-sized avatar
🏡

Parker Selbert sorentwo

🏡
View GitHub Profile
@sorentwo
sorentwo / queue_breaker.ex
Last active August 2, 2022 14:33
Oban Queue Circuit Breaker
defmodule QueueBreaker do
@moduledoc """
Automatic queue pausing/resuming based on accumulated errors.
## Example
Pause after 1000 errors:
QueueBreaker.attach("risky-queue", 1000)
"""
@sorentwo
sorentwo / start_oban_web.livemd
Created July 27, 2022 16:38
Single Page Oban Web

Single Page Oban Web

Application.put_env(:sample, Sample.Repo, database: "oban_dev")

Application.put_env(:phoenix, :json_library, Jason)

Application.put_env(:sample, Sample.Endpoint,
  http: [ip: {127, 0, 0, 1}, port: 5001],
  server: true,
@sorentwo
sorentwo / hooks.md
Created July 13, 2022 20:16
Oban Pro Hooks Documentation

Worker Hooks

Worker hooks are called after a job finishes executing. They can be defined as callback functions on the worker, or in a separate module for reuse across workers.

Hooks are called synchronously, from within the job's process with safety applied. Any exceptions or crashes are caught and logged, they won't cause the job to fail or the queue to crash.

Hooks do not modify the job or execution results. Think of them as a convenient alternative

@sorentwo
sorentwo / throughput.exs
Created June 3, 2022 20:42
A benchmark for measuring the throughput of a single Oban queue
defmodule Metrics do
@moduledoc false
def attach(event) do
counter = :counters.new(4, [])
:persistent_term.put(event, counter)
for idx <- 1..4, do: :counters.put(counter, idx, 0)
@sorentwo
sorentwo / recording_output_with_pro.livemd
Created February 21, 2022 14:15
Demonstration of using Oban job output with Pro's Relay and Workflow

Recording Output with Oban Pro

Setup

To get started, install Oban, Pro, and our dependencies. Note that the oban_pro install requires a working Pro license.

Mix.install([
@sorentwo
sorentwo / batch_test_example.ex
Created February 11, 2022 18:29
Demonstrate how to integration test batch callbacks
def start_supervised_oban!(opts) do
opts =
opts
|> Keyword.put_new(:name, Oban)
|> Keyword.put_new(:repo, MyApp.Repo)
|> Keyword.put_new(:poll_interval, :timer.minutes(10))
|> Keyword.put_new(:shutdown_grace_period, 25)
pid = start_supervised!({Oban, opts}, id: opts[:name])
@sorentwo
sorentwo / partitioned_rate_limit_demo.exs
Created August 29, 2021 16:52
Tenant based partitioned rate limiting in Oban Pro
defmodule MyApp.TenantWorker do
@moduledoc false
use Oban.Worker
@impl Worker
def perform(%Job{conf: conf, args: %{"tenant" => tenant, "id" => id}}) do
time =
Time.utc_now()
|> Time.truncate(:second)
@sorentwo
sorentwo / relay_testing.exs
Created February 9, 2021 22:37
Oban Pro Relay Plugin Mini-Demo
alias Oban.Pro.Plugins.Relay
Oban.Pro.Repo.start_link()
Oban.start_link(
repo: Oban.Pro.Repo,
queues: [alpha: 2],
plugins: [Oban.Pro.Plugins.Relay]
)
@sorentwo
sorentwo / chunk_bench.exs
Created February 7, 2021 16:16
Benchmark comparing a basic Worker to the Chunk Worker in Pro
Oban.Pro.Repo.start_link()
Oban.start_link(
repo: Oban.Pro.Repo,
queues: [alpha: 20],
plugins: [Oban.Pro.Plugins.ChunkManager]
)
:persistent_term.put(:chunk_bench_pid, self())
@sorentwo
sorentwo / workflow.md
Created November 5, 2020 20:38
Workflow Worker

Workflow Worker

🌟 This worker is available through Oban.Pro

Workflow workers compose together with arbitrary dependencies between jobs, allowing sequential, fan-out and fan-in execution workflows. Workflows are fault tolerant, can be homogeneous or heterogeneous, and scale horizontally across all available nodes. Workflows are unique to Oban—only possible due to historic job retention.