Skip to content

Instantly share code, notes, and snippets.

View sorentwo's full-sized avatar
🏡

Parker Selbert sorentwo

🏡
View GitHub Profile
@sorentwo
sorentwo / start_oban_web.livemd
Created July 27, 2022 16:38
Single Page Oban Web

Single Page Oban Web

Application.put_env(:sample, Sample.Repo, database: "oban_dev")

Application.put_env(:phoenix, :json_library, Jason)

Application.put_env(:sample, Sample.Endpoint,
  http: [ip: {127, 0, 0, 1}, port: 5001],
  server: true,
@sorentwo
sorentwo / hooks.md
Created July 13, 2022 20:16
Oban Pro Hooks Documentation

Worker Hooks

Worker hooks are called after a job finishes executing. They can be defined as callback functions on the worker, or in a separate module for reuse across workers.

Hooks are called synchronously, from within the job's process with safety applied. Any exceptions or crashes are caught and logged, they won't cause the job to fail or the queue to crash.

Hooks do not modify the job or execution results. Think of them as a convenient alternative

@sorentwo
sorentwo / throughput.exs
Created June 3, 2022 20:42
A benchmark for measuring the throughput of a single Oban queue
defmodule Metrics do
@moduledoc false
def attach(event) do
counter = :counters.new(4, [])
:persistent_term.put(event, counter)
for idx <- 1..4, do: :counters.put(counter, idx, 0)
@sorentwo
sorentwo / recording_output_with_pro.livemd
Created February 21, 2022 14:15
Demonstration of using Oban job output with Pro's Relay and Workflow

Recording Output with Oban Pro

Setup

To get started, install Oban, Pro, and our dependencies. Note that the oban_pro install requires a working Pro license.

Mix.install([
@sorentwo
sorentwo / batch_test_example.ex
Created February 11, 2022 18:29
Demonstrate how to integration test batch callbacks
def start_supervised_oban!(opts) do
opts =
opts
|> Keyword.put_new(:name, Oban)
|> Keyword.put_new(:repo, MyApp.Repo)
|> Keyword.put_new(:poll_interval, :timer.minutes(10))
|> Keyword.put_new(:shutdown_grace_period, 25)
pid = start_supervised!({Oban, opts}, id: opts[:name])
@sorentwo
sorentwo / partitioned_rate_limit_demo.exs
Created August 29, 2021 16:52
Tenant based partitioned rate limiting in Oban Pro
defmodule MyApp.TenantWorker do
@moduledoc false
use Oban.Worker
@impl Worker
def perform(%Job{conf: conf, args: %{"tenant" => tenant, "id" => id}}) do
time =
Time.utc_now()
|> Time.truncate(:second)
@sorentwo
sorentwo / relay_testing.exs
Created February 9, 2021 22:37
Oban Pro Relay Plugin Mini-Demo
alias Oban.Pro.Plugins.Relay
Oban.Pro.Repo.start_link()
Oban.start_link(
repo: Oban.Pro.Repo,
queues: [alpha: 2],
plugins: [Oban.Pro.Plugins.Relay]
)
@sorentwo
sorentwo / chunk_bench.exs
Created February 7, 2021 16:16
Benchmark comparing a basic Worker to the Chunk Worker in Pro
Oban.Pro.Repo.start_link()
Oban.start_link(
repo: Oban.Pro.Repo,
queues: [alpha: 20],
plugins: [Oban.Pro.Plugins.ChunkManager]
)
:persistent_term.put(:chunk_bench_pid, self())
@sorentwo
sorentwo / workflow.md
Created November 5, 2020 20:38
Workflow Worker

Workflow Worker

🌟 This worker is available through Oban.Pro

Workflow workers compose together with arbitrary dependencies between jobs, allowing sequential, fan-out and fan-in execution workflows. Workflows are fault tolerant, can be homogeneous or heterogeneous, and scale horizontally across all available nodes. Workflows are unique to Oban—only possible due to historic job retention.

@sorentwo
sorentwo / description.md
Created February 18, 2020 21:54
Oban Bi-Directional Workflow

This models a generic "workflow" module that can resolve dependencies, fan-in and fan-out in a declarative way.

Some general notes about the approach:

  • A ref is used as a key to glue jobs together (any unique argument would do)
  • Args are passed through to each worker to customize worker behavior or output (i.e. to disable quoting)
  • Each job is guaranteed to only run once per-hour (the unique period can be changed to anything)
  • Starting any job in the flow will resolve the full dependency graph (this example models a fully connected graph)
  • Dependencies are checked in parallel, while dependencies execute the depenent will wait for a maximum of 60s (could be any sane value)