Skip to content

Instantly share code, notes, and snippets.

View brycied00d's full-sized avatar
👍
"Hasn't fucked up recently" engineer.

Bryce Chidester brycied00d

👍
"Hasn't fucked up recently" engineer.
  • Tigard, OR
View GitHub Profile
@brycied00d
brycied00d / fluentd-buffer-cleaner.rb
Created July 9, 2020 23:20
fluentd-buffer-cleaner.rb - Validate each buffered record against some basic principles, dumping good and "bad"/"corrupt" records to separate buffers. (This is v2 of fluentd-buffer-check.rb)
# Quick and dirty script to split a fluentd buffer chunk into "good" and "bad" buffers.
# Uses the validation logic from fluentd-buffer-check.rb (https://gist.github.com/brycied00d/70c53104d780835d2d48610b7a783c8c)
# Compatible with fluentd 1.9.0. (Much of the buffer/metadata code taken from fluentd source.)
# Copyright 2020 Bryce Chidester <[email protected]>
require 'msgpack'
require 'time'
def flush_new_buffer(state, chunk_id, data, prefix = 'NEW_buffer', suffix = 'log')
path = "#{prefix}.#{state}#{chunk_id}.#{suffix}"
@brycied00d
brycied00d / fluentd-buffer-check.rb
Last active March 15, 2020 02:30
fluentd-buffer-check.rb - Validate each buffered record against some basic principles, identifying "bad"/"corrupt" records within buffer files.
#!/usr/bin/env ruby
# Simple script to "validate" all records in a fluentd buffer
# Sometimes, something goes wonky in my environment and weird data ends up in the
# buffer file, and then causes certain output plugins to choke. Ideally, I can
# catch this earlier as a fluentd filter, but in the meantime I'm left with buffer
# files with lots of good data and bad data and I need a tool to confirm what that
# buffer contains.
# Version 2 of this script will write the good records to a "good file" for reinclusion
# back into fluentd, and a "bad file" recording the unprocessable entities. Additionally,
@brycied00d
brycied00d / fluentd-buffer-dump.rb
Created March 10, 2020 15:43
fluentd-buffer-dump.rb - Dump the contents of a fluentd buffer to stdout for inspection.
#!/usr/bin/env ruby
# Quick and dirty script to print the contents of a fluentd buffer chunk
# Useful when investigating a "corrupt" chunk, typically containing malformed
# record data. Recrd data is printed to stdout; progress/information is printed
# to stderr.
# Compatible with fluentd 1.9.0. (Much of the buffer/metadata code taken from fluentd source.)
# Copyright 2020 Bryce Chidester <[email protected]>
require 'msgpack'
@brycied00d
brycied00d / backup_zeroshell.sh
Created February 10, 2020 18:47
Script to fetch/download a backup from a Zeroshell system
#!/bin/sh
# This is a quick, simple script to download a profile backup from a Zeroshell
# instance. It has been tested with release 3.9.3
# No warranty or guarantee of any kind is made, nor any support offered.
# Copyright Bryce Chidester <[email protected]>
# Configuration: Replace <hostname>, <username>, and <password> as appropriate.
endpoint='https://<hostname>/cgi-bin/kerbynet'
@brycied00d
brycied00d / fluentd-buffer-split.rb
Last active March 10, 2020 15:46
fluentd-buffer-split.rb - Split a single large fluentd buffer chunk (of type "file") into smaller pieces that can be manually reintroduced gradually in order to reduce load on processing pipelines.
# Quick and dirty script to split a fluentd buffer chunk into smaller chunks.
# Compatible with fluentd 1.9.0. (Much of the buffer/metadata code taken from fluentd source.)
# Copyright 2020 Bryce Chidester <[email protected]>
require 'msgpack'
require 'time'
NEW_CHUNK_SIZE = 100_000
def flush_new_buffer(state, chunk_id, data)
@brycied00d
brycied00d / backup-restic.sample.conf
Last active August 25, 2023 03:44
backup-restic.sh - My generap-purpose script to backup a system, balancing simplicity with configurability.
# This is an example file, with example values.
#export RESTIC_PASSWORD=apassword
#export RESTIC_REPOSITORY=b2:bucket:path
#export B2_ACCOUNT_ID=12345
#export B2_ACCOUNT_KEY=abcdefg
#restic_backup /etc
#restic_backup /root
#restic_backup /var
@brycied00d
brycied00d / ttrss-migrator.rb
Created December 28, 2016 20:03
TT-RSS Database Migrator, Migration from MySQL to Postgres
#!/usr/bin/env ruby
=begin
Migrate TT-RSS data from MySQL to Postgres
Author: Bryce Chidester <[email protected]>
Provided as-is, no warranty. Make backups!
This script may work in the reverse direction... but probably not. And seeing as
TT-RSS doesn't officially support any other database backend at this time, there's
really no point in migrating to or from any other database.
@brycied00d
brycied00d / update_monit.sh
Last active February 11, 2016 19:12
Quick script to update Monit wih the binary release, since Debian's packages lag so far behind.
#!/bin/sh
# Script for Debian, which lags majorly behind on Monit releases.
# This will download the given version and overwrite Debian's binary and man-page,
# install a symlink to monitrc (Debian's package expects it in /etc/monit/monitrc,
# while the distributed binary expects /etc/monitrc), and installs the distributed
# example configuration at /etc/monit/monitrc.pkgdist
#
# Bryce Chidester <[email protected]>
# Run as root
@brycied00d
brycied00d / update-gparted.sh
Created October 18, 2015 01:32
Quick script to update my GParted netboot.
#!/bin/sh
GPVER=$1
if [ -z ${GPVER} ]; then
# Match the download link to the LiveCD download on the project homepage.
GPVER=$(curl -qs http://gparted.sourceforge.net/ | sed -n -r -e 's/.*iso">Live (.*)<\/a>/\1/p')
echo "Fetched GParted version: ${GPVER}"
else
echo "Using GParted version: ${GPVER}"
fi
@brycied00d
brycied00d / update-clonezilla.sh
Created October 18, 2015 01:30
Quick script to update my Clonezilla netboot.
#!/bin/sh
for CZBRANCH in stable alternative; do
# Try to use whatever version was given on the command line on both stable and alternative branches
echo "Processing branch ${CZBRANCH}..."
CZVER=$1
if [ -z ${CZVER} ]; then
# Match the <input> field on the download selector form
CZVER=$(curl -qs http://clonezilla.org/downloads/download.php?branch=${CZBRANCH} | sed -n -r -e 's/.*<input.*name=.version.*value=.(.*).>/\1/p')
echo "Fetched Clonezilla ${CZBRANCH} version ${CZVER}"