Joseph Sebastian alkmst-xyz

Make Ubuntu packages 90% faster by rebuilding them

TL;DR

You can take the same source code package that Ubuntu uses to build jq, compile it again, and realize 90% better performance.

Setting

I use jq for processing GeoJSON files and other open data offered in JSON format. Today I am working with a 500MB GeoJSON file that contains the Alameda County Assessor's parcel map. I want to run a query that prints the city for every parcel worth more than a threshold amount. The program is

FastAPI and Uvicorn Logging

When running FastAPI app, all the logs in console are from Uvicorn and they do not have timestamp and other useful information. As Uvicorn applies python logging module, we can override Uvicorn logging formatter by applying a new logging configuration.

Meanwhile, it's able to unify the your endpoints logging with the Uvicorn logging by configuring all of them in the config file log_conf.yaml.

Before overriding:

uvicorn main:app --reload

Python - hashing JSON data structures

A function hash_json(data), accepting a structure loaded from json.load() and computing a hash.

Example

$ ./hashjson.py
Hash a.json: 8212462b8e9ce805cac2f0758127c5cfd7710baf
Hash b.json: 8212462b8e9ce805cac2f0758127c5cfd7710baf

In the last years I've been asked multiple times about the comparison between raylib and SDL libraries. Unfortunately, my experience with SDL was quite limited so I couldn't provide a good comparison. In the last two years I've learned about SDL and used it to teach at University so I feel that now I can provide a good comparison between both.

Hope it helps future users to better understand this two libraries internals and functionality.

	# -- coding: utf-8 --
	"""Example combination of FastAPI and Pydantic with aiosql and aiosqlite.

	This module demonstrates the minimum viable integration approach for putting
	together a few components:

	- FastAPI. This provides a very high-performance and type-driving approach to
	building APIs in Python
	- Pydantic. A powerful data validation library.
	- aiosql. Inspired by annosql and originally Clojure's yeSql, a way to programatically,

	import asyncio
	import psycopg2

	# dbname should be the same for the notifying process
	conn = psycopg2.connect(host="localhost", dbname="example", user="example", password="example")
	conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)

	cursor = conn.cursor()
	cursor.execute(f"LISTEN match_updates;")

	-- returns a child object with single parent as nested JSON object
	--
	-- {
	-- "id": 155,
	-- "name": "Stephen King",
	-- "publisher": {
	-- "id": 13,
	-- "name": "Paladin Books"
	-- }
	-- }

	# We set an environment variable in this phase so it gets picked up by pip, but we don't want to bake secrets into our container image
	FROM python:3.6-alpine AS builder

	ARG INDEX_URL
	ENV PIP_EXTRA_INDEX_URL=$INDEX_URL

	COPY requirements.txt .

	RUN pip install -U pip \
	&& pip install --user -r requirements.txt

	# Determines which branch(es) will cause a CI build to be started
	trigger:
	- master

	# Stages precede strategy, in other words each stage can contain a strategy (or multiple or none)
	# Full run-cylce described here: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/runs?view=azure-devops
	stages:
	- stage: Prepare
	jobs:
	- job: