Skip to content

Instantly share code, notes, and snippets.

View bobelev's full-sized avatar

Andrey bobelev

View GitHub Profile
@mpkocher
mpkocher / example.py
Last active January 26, 2025 21:39
Pydantic Settings. Example of configure a yaml/json file at runtime.
"""
Different workarounds for setting a configuration file path at runtime.
https://github.com/pydantic/pydantic-settings/issues/259
There's an explicit step of validating the file path to make the errors more obvious.
Otherwise, a cryptic pydantic error will be raised.
"""
import argparse
@nkhitrov
nkhitrov / structlog_fastapi.py
Created March 16, 2023 00:04
Structlog FastAPI example
"""
Structlog example configuration with FastAPI.
Features:
- async bound logger
- contextvars to log request-id and other meta data
- custom format for default logging loggers and structlog loggers
"""
import asyncio
import logging
@nymous
nymous / README.md
Last active April 14, 2025 13:44
Logging setup for FastAPI, Uvicorn and Structlog (with Datadog integration)

Logging setup for FastAPI

This logging setup configures Structlog to output pretty logs in development, and JSON log lines in production.

Then, you can use Structlog loggers or standard logging loggers, and they both will be processed by the Structlog pipeline (see the hello() endpoint for reference). That way any log generated by your dependencies will also be processed and enriched, even if they know nothing about Structlog!

Requests are assigned a correlation ID with the asgi-correlation-id middleware (either captured from incoming request or generated on the fly). All logs are linked to the correlation ID, and to the Datadog trace/span if instrumented. This data "global to the request" is stored in context vars, and automatically added to all logs produced during the request thanks to Structlog. You can add to these "global local variables" at any point in an endpoint with `structlog.contextvars.bind_contextvars(custom

@atyachin
atyachin / Android_Emulator_AWS_EC2_ARM64_2022.txt
Last active April 17, 2025 08:11
Running headless android emulator on AWS EC2 Ubuntu instance (ARM64 / aarch64) - 2022
Android Emulator (ARM64) on EC2 - 2022
---------------------------------------
1. Launch EC2 ARM based Instance (a1.metal / a1.2xlarge): (16 Gb RAM, 32Gb Disk), Ubuntu Server 22.04 LTS (HVM) ARM x64
2. sudo apt update && sudo apt upgrade
3. sudo apt install default-jdk python3-pip repo python-is-python3 unzip libpcre2-dev adb
4. wget https://dl.google.com/android/repository/commandlinetools-linux-8512546_latest.zip
5. unzip commandlinetools-linux-8512546_latest.zip -d android-sdk
6. sudo mv android-sdk /opt/
7. mkdir /opt/android-sdk/cmdline-tools/latest
8. mv /opt/android-sdk/cmdline-tools/* /opt/android-sdk/cmdline-tools/latest (ignore the error)
@slok
slok / pprof.md
Last active March 29, 2025 10:36
Go pprof cheat sheet

Enable profiling

Default http server

import (
    _ "net/http/pprof"
    "net/http"
)

Proxmox VE Installation on Hetzner Server via Rescue System

Follow these steps to install Proxmox VE on a Hetzner server via the Rescue System. The Rescue System is a Linux-based environment that can be booted into to perform system recovery tasks. We'll be using it to install Proxmox VE.

In order to complete the process, it is indeed necessary to first boot into the Rescue System and then connect to it via SSH. This will allow you to run the commands for installing Proxmox VE. Here are the steps:

Starting the Rescue System

  1. Log into the Hetzner Robot.
  2. Under "Main Functions; Server" select the desired server and then open the tab "Rescue".
@den-crane
den-crane / Clickhouse-fast-not-exists
Last active May 29, 2024 23:22
Clickhouse fast not exists
ClickHouse server version 18.14.12 revision 54409.
create table data(K Int64, V String) engine=MergeTree order by K;
insert into data select number, toString(number) from numbers(100,100000000);
optimize table data final;
create table buffer(K Int64, V String) engine=Memory;
insert into buffer select number, toString(number) from numbers(0,1000);
@marcan
marcan / linux.sh
Last active July 21, 2024 14:00
Linux kernel initialization, translated to bash
#!/boot/bzImage
# Linux kernel userspace initialization code, translated to bash
# (Minus floppy disk handling, because seriously, it's 2017.)
# Not 100% accurate, but gives you a good idea of how kernel init works
# GPLv2, Copyright 2017 Hector Martin <[email protected]>
# Based on Linux 4.10-rc2.
# Note: pretend chroot is a builtin and affects the current process
# Note: kernel actually uses major/minor device numbers instead of device name
@epiphani
epiphani / CDHTez.md
Last active February 14, 2024 08:03
Getting Tez enabled on CDH5.4+

So Hive in CDH is horribly, painfully slow. Cloudera ships Hive 1.1, which is actually moderately modern. It is, however, very badly configured out of the box and patched with custom code from Cloudera. With a bit of effort, we managed to improve hive performance considerably. We really shouldn't have to do this, but Cloudera is actively working against supporting a performant Hive.

First, building Tez was fairly straightforward. Using the instructions at https://github.com/apache/tez/blob/master/docs/src/site/markdown/install.md, the only change was to use the version string "2.6.0" for the build. I believe that was the default. Don't use the CDH string, it won't work.

At the bottom of the installation instructions, there's mention of the fact that to use the local hadoop jars (rather than those packaged with tez) you must unpack the jars in HDFS rather than using the tarball. In this case, unpack the tez-minimal tarball and upload the contents to /apps/tez-0.7.0 (or whatever you prefer). Don't fo

@nolanlawson
nolanlawson / protips.js
Last active November 19, 2024 02:40
Promise protips - stuff I wish I had known when I started with Promises
// Promise.all is good for executing many promises at once
Promise.all([
promise1,
promise2
]);
// Promise.resolve is good for wrapping synchronous code
Promise.resolve().then(function () {
if (somethingIsNotRight()) {
throw new Error("I will be rejected asynchronously!");