Skip to content

Instantly share code, notes, and snippets.

@thepaul
thepaul / yieldscanner.py
Created January 12, 2016 00:57
Similar to re.Scanner but works as a coroutine, yielding tokens as found
import re
class NotAToken(Exception):
pass
class YieldScanner(re.Scanner):
def scan(self, inputstr):
match = self.scanner.scanner(inputstr).match
pos = 0
while True:
git for-each-ref --format '%(refname) %(authoremail) %(committerdate:raw)' refs/remotes/origin/ \
| sort -k2,3 \
| awk '$3 < (systime()-(86400*30)) {
gsub(/refs\/remotes\//, "", $1);
printf "%-30s %-26s %s\n", $1, $2, strftime("%Y-%m-%d %H:%M", $3)
}'

Keybase proof

I hereby claim:

  • I am thepaul on github.
  • I am thepaul (https://keybase.io/thepaul) on keybase.
  • I have a public key ASDVjs42Wo8PNSFWHDKiZmIsGEemdqZ4En1SFeO0uzkhswo

To claim this, I am signing this object:

@thepaul
thepaul / pg_bytes.sql
Created April 12, 2018 15:23
postgresql PL/PGSQL functions for encoding 64-bit numbers as 8-byte strings and vice versa
-- transform 8 bytes into an int64
CREATE FUNCTION bytes_to_int64(s BYTEA, OUT result INT8)
AS $$
BEGIN
SELECT bit_or(get_byte(s, ind)::INT8 << ((7 - ind) * 8))
INTO result
FROM generate_series(0, 7) AS ind;
END;
$$ LANGUAGE 'plpgsql';
@thepaul
thepaul / git-get-remote-head.sh
Created December 18, 2020 17:22
Get the name of the remote HEAD (the default branch on that remote)
#!/bin/bash -e
REMOTE=${1:-origin}
ref=$(git rev-parse --symbolic-full-name "$REMOTE/HEAD" 2>/dev/null) || {
git remote set-head "$REMOTE" -a >/dev/null
ref=$(git rev-parse --symbolic-full-name "$REMOTE/HEAD")
}
echo "${ref#refs/remotes/$REMOTE/}"
// Copyright (C) 2021 Storj, Inc.
// See LICENSE for copying information.
package main
import (
"context"
"crypto/tls"
"fmt"
"log"
@thepaul
thepaul / tuning reputation further.ipynb
Created February 4, 2022 19:43
Tuning reputation calculation parameters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@thepaul
thepaul / data_loss_vs_dq_chance_sim.py
Created February 4, 2022 19:50
Simulating chance of disqualification for a given % of data loss
#!/usr/bin/env python3
#
# This is meant to act as a double-check of math used elsewhere to calculate the chance
# of disqualification given a level of data loss on a storage node. Instead of calculating
# the chance algorithmically, this performs a simulation using the specified parameters
# and reports how many nodes under the given circumstances would actually be disqualified.
import argparse
import csv
import math
Data Lost Nodes Lambda Init-alpha Init-beta Succ-adj Fail-adj DQ thresh Weight Audits % DQ'd Mean audits to DQ Stddev audits to DQ
0.01 3000 0.95 1 0 1 -1 0.8 1 10000 4.27 97 819
0.02 3000 0.95 1 0 1 -1 0.8 1 10000 11.03 944 2276
0.03 3000 0.95 1 0 1 -1 0.8 1 10000 27.73 2563 3248
0.04 3000 0.95 1 0 1 -1 0.8 1 10000 61.62 3098 3139
0.05 3000 0.95 1 0 1 -1 0.8 1 10000 93.24 2328 2565
0.06 3000 0.95 1 0 1 -1 0.8 1 10000 99.77 1192 1544
0.07 3000 0.95 1 0 1 -1 0.8 1 10000 100.00 571 802
0.08 3000 0.95 1 0 1 -1 0.8 1 10000 100.00 311 457
0.09 3000 0.95 1 0 1 -1 0.8 1 10000 100.00 189 281
data loss lambda dq thresh grace rounds dq fraction mean rounds to dq rounds
0.01 0.950 0.90 30 0.193 473.36 1000
0.02 0.950 0.90 30 0.724 399.98 1000
0.03 0.950 0.90 30 0.975 271.07 1000
0.04 0.950 0.90 30 0.998 153.60 1000
0.05 0.950 0.90 30 1.000 99.79 1000
0.06 0.950 0.90 30 1.000 74.29 1000
0.01 0.980 0.90 30 0.001 50.00 1000
0.02 0.980 0.90 30 0.032 145.78 1000
0.03 0.980 0.90 30 0.202 291.65 1000