Skip to content

Instantly share code, notes, and snippets.

View catid's full-sized avatar

Chris Taylor catid

View GitHub Profile
@catid
catid / bad_ecc_64.cpp
Created July 26, 2018 06:00
This is a bad erasure code where it has to send some number of extra bits for each 32-bit word of input data. But it does work, which is neat.
/*
// Convolutional encoder:
s = a * x + b * y;
t = c * x + d * y;
The problem with this one is that the "ezc" number of low bit zeros in e is also a number of additional bits that must be sent with each recovery word.
Specifically in the decoder this line:
> uint64_t yr = ei * t >> ezc;
So to tolerate ezc = 4 zero bits, we need to make t 4 bits longer.
@catid
catid / fast_field_65.cpp
Created July 24, 2018 16:21
Fast 64-bit almost-finite field
#include <stdint.h>
/*
65-bit multiplication of two 64-bit values a, b
with low bits set to 1 (implicitly):
(2a + 1)(2b + 1)
= 4ab + 2a + 2b + 1
Stuffing it back into a 64-bit value:
uint64_t f64(uint64_t x, uint64_t y) {
return y * (2 - y * x);
}
static uint64_t inverse64(uint64_t x) {
uint64_t y = x;
y = f64(x, y);
y = f64(x, y);
y = f64(x, y);
y = f64(x, y);
q = x >> 32
r = x - (q << 32)
i = 0
while (q > 0) {
t = (q << 4) - q
q = t >> 32
r_i = t - (q << 32)
if (++i % 2 == 1)
r -= r_i
Spinal Codes (totally metal name)
Paper: http://nms.csail.mit.edu/papers/fp049-perry.pdf
This is a bitwise error correction code that can correct bit errors anywhere in the message.
It is not MDS and the strength of the code includes the processing time allowed for decoding.
More discussion at the end.
Simple version of the algorithm:
@catid
catid / int_compress.cpp
Created July 2, 2018 00:12
Lossy integer compression 32 => 16 bits, 16 => 8 bits
//------------------------------------------------------------------------------
// Intrinsics
/// Returns first position with set bit (LSB = 0)
/// Precondition: x != 0
TONK_FORCE_INLINE unsigned NonzeroLowestBitIndex(uint32_t x)
{
#ifdef _MSC_VER
unsigned long index;
// Note: Ignoring result because x != 0
{d-Connection} [127.0.0.1:1] Connection::OnTimeSync(ReceivedBPS=105088, LossRate=0)
{d-Connection} [127.0.0.1:1] Connection::OnTimeSync(ReceivedBPS=129088, LossRate=0)
{d-Connection} [127.0.0.1:1] Connection::OnTimeSync(ReceivedBPS=129088, LossRate=0)
{d-Connection} [127.0.0.1:1] Connection::OnTimeSync(ReceivedBPS=113088, LossRate=0.125)
{d-Connection} [127.0.0.1:1] Connection::OnTimeSync(ReceivedBPS=113088, LossRate=0.125)
{d-Connection} [127.0.0.1:1] Connection::OnTimeSync(ReceivedBPS=98944, LossRate=0.230469)
Got router queue filling up simulation working
My crappy current CC increases the send rate to 129KB/S until the queue fills up (filling 1 MB buffer) and data starts dropping off the back
Then CC detects loss and sets send rate lower
@catid
catid / gist:1e8e993e1581e92d96515a07c5ce4804
Last active June 19, 2018 04:23
Extra huf_compress heuristic to speed up already-compressed or random data
/* Scan input and build symbol stats */
{ CHECK_V_F(largest, FSE_count_wksp (table->count, &maxSymbolValue, (const BYTE*)src, srcSize, table->count) );
if (largest == srcSize) { *ostart = ((const BYTE*)src)[0]; return 1; } /* single symbol, rle */
if (largest <= (srcSize >> 7) + 1) {
return 0; /* heuristic : probably not compressible enough */
}
/* Added this more accurate heuristic to handle high entropy data properly -catid */
U64 total = 0;
U64 sum_clogc = 0;
#pragma once
#include "xxxxxxxxxProtocol.h"
#include "xxxxxxxxxTools.h"
#define ZSTD_STATIC_LINKING_ONLY /* Enable advanced API */
#include "thirdparty/zstd/zstd.h" // Zstd
#include "thirdparty/zstd/zstd_errors.h"
namespace xxx {
@catid
catid / RoughStrCompare
Created May 27, 2018 03:51
RoughStrCompare
/**
RoughStrCompare()
This does a rough ASCII case-insensitive string comparison. Mainly for fun.
All of the a-z and A-Z ASCII codes are between 0x41 - 0x5A and 0x61 - 0x7A.
The idea here is to mask out 0x20 to make these equivalent (case-insensitive)
but the problem with this idea is that it aliases these characters:
@ with ` [ with { \ with | ] with } ^ with ~
*/