Joshua Carp jmcarp

oximeter perf harness

A/B the oximeter collector under synthetic load. Topology per measured run, all on one illumos box:

ch-dev (ClickHouse)  <--inserts--  oximeter standalone collector (the arm)
                                          ^ HTTP pull
                     N x oximeter-load-test producers -----------------------+

Optimizing badge performance

Before proposing possible solutions, here are a few questions I would ask if this were more than an exercise:

The badge endpoint uses the pyramid app, the sql backend (authorization, block list), and the elastic backend (search). Which commponent(s) take up the most time or are most challenging to scale?
- For the purpose of the exercise, I'll assume the slowest/most expensive comopnent is elastic.
What are the current and anticipated costs of the badge endpoint under the status quo? Would the cost of additional complexity (e.g. learning a new database, web framework, or programming paradigm) outweigh the savings from buying more servers?
- I'll assume that cloud infrastructure is cheap and developer time expensive, and lean toward keeping things simple rather than maximizing performance.
What's the hypothesized value-add of the badge feature, and how can we test that hypothesis?
- Does the badge make users more likely to view or create annotations?
Does A/B tes

	#!/usr/bin/env bash
	# THROWAWAY — measure per-producer cardinality: print "<id> <kind> <address> <port> <samples>"
	# (tab-separated) per producer, where <samples> is the sample count from the producer's last
	# successful collection. Straight from the oximeter collector HTTP API. Needs curl + jq;
	# omdb only for URL discovery.
	#
	# URL discovery (in order):
	# 1. explicit: ./oximeter_producer_cardinality.sh 'http://[fd00:...]:12223'
	# 2. env: OXIMETER_URL='http://[fd00:...]:12223' ./oximeter_producer_cardinality.sh
	# 3. fallback: scrape `omdb oximeter list-producers` stderr for the collector URL.

	#!/usr/bin/env bash
	#
	# Empirical test: which settings actually make ALTER TABLE ... MODIFY TTL
	# return without waiting for the materialization mutation, on the v23.8 fork
	# of ClickHouse used by oximeter.
	#
	# Predictions from reading work/src/src/Storages/StorageMergeTree.cpp:
	# - defaults: ALTER blocks until mutation finishes
	# - SETTINGS mutations_sync = 0: identical to defaults (no-op)
	# - SETTINGS alter_sync = 0: identical to defaults (no-op in v23.8)

	Disk is haunted:

	```
	ubuntu@monitoring:~/monitoring$ ls /data/
	ls: cannot open directory '/data/': Input/output error
	```

	Disk is very haunted:

	```

	#!/usr/bin/env python3
	"""
	Parse illumos zonestat output and convert to VictoriaMetrics JSON format.

	Usage:
	python parse_zonestat.py <input_file> [output_file]

	If output_file is not specified, outputs to stdout.
	"""

	name,titles,roles,locations,email
	Whistleblower Hotline,hotline@rvaschools.net,Faculty & Staff,,hotline@rvaschools.net
	Kayla Aaron,Teacher,Faculty & Staff,Ginter Park Elementary,kaaron@rvaschools.net
	Tania Abayomi,,Faculty & Staff,Linwood Holton Elementary,tabayomi@rvaschools.net
	Hannah Abbey,Mental Health Contractor,Faculty & Staff,Huguenot High School,habbey.contractor@rvaschools.net
	Kiera Abdelkader,Coach,Faculty & Staff,"Boushall Middle School, River City Middle School",kriddick@rvaschools.net
	Mohammed Fahmy Abdelrazele,HS English Teacher,Faculty & Staff,Rich Academy formerly CCP,mabdelraze.contractor@rvaschools.net
	Juvenal Abrego meneses,Principal Iii,Faculty & Staff,Cardinal Elementary School,jabrego@rvaschools.net
	Elena Acevedo,Teacher,Faculty & Staff,George W. Carver Elementary,eacevedo@rvaschools.net
	Mary Ackerly,Teacher 200 Days - Henry Marsh,Faculty & Staff,,mackerly@rvaschools.net

	import json

	import fiona
	import requests


	def get_map(base_url, parcel_id, out_path, bbox_scale_factor=1.25):
	parcel_id_clean = parcel_id.replace("-", "")

	state = {

	#!/bin/bash

	set -euo pipefail

	rm -f parcels.db

	curl -O https://widget.charlottesville.org/gis/zip_download/parcel_area.zip
	unzip parcel_area.zip

	layer=$(ogrinfo parcel_area_*.shp \| grep '1: ' \| awk '{print $2}')

	#!/usr/bin/env python
	# -- coding: utf-8 --

	import csv
	import datetime
	import collections

	import pytest