Skip to content

Instantly share code, notes, and snippets.

@antonputra
antonputra / gist:533fdd507f797cc30b082eed2c4f6fb4
Created August 23, 2024 20:22
Golang Dockerfile with upx
FROM golang:1.23.0-bookworm AS build
ARG upx_version=4.2.4
RUN apt-get update && apt-get install -y --no-install-recommends xz-utils && \
curl -Ls https://github.com/upx/upx/releases/download/v${upx_version}/upx-${upx_version}-amd64_linux.tar.xz -o - | tar xvJf - -C /tmp && \
cp /tmp/upx-${upx_version}-amd64_linux/upx /usr/local/bin/ && \
chmod +x /usr/local/bin/upx && \
apt-get remove -y xz-utils && \
rm -rf /var/lib/apt/lists/*
@NickCrews
NickCrews / coalesce_parquet.py
Last active January 10, 2024 03:48
Coalesce parquet files
"""coalesce_parquets.py
gist of how to coalesce small row groups into larger row groups.
Solves the problem described in https://issues.apache.org/jira/browse/PARQUET-1115
"""
from __future__ import annotations
from pathlib import Path
from typing import Callable, Iterable, TypeVar
@anandkunal
anandkunal / README.md
Last active May 30, 2024 15:36
AWS SigV4 & SES Walkthrough in Go

AWS SigV4 & SES Walkthrough in Go

A few years ago, I helped a non-profit organization build and deploy a series of web applications and micro-services on top of AWS. One of the services was responsible for sending out email notifications to people via AWS SES.

Back then, I wrote a really simple program in Go that made direct calls to the AWS API. Instead of using an official library, I read the API specification, learned how to authenticate requests, and implemented a fairly trivial SDK. The code was succinct, readable, and worked perfectly for 3 years without any issues.

That is until this past week…

AWS recently updated the specification, we’re now at Signature Version 4 (SigV4), for how API requests must be formed and signed by clients. In this walkthrough, I’ll share the full Go source code (~130 LOC) that I put together to make valid authenticated calls to AWS for SES. You don’t need to be knowledgeable about Go to grok code in this post. In fact, code in this post will translate pretty cleanly to

from fastapi import FastAPI, Body
from pydantic import BaseModel
app = FastAPI()
class Number(BaseModel):
value: int
@sanchezzzhak
sanchezzzhak / clickhouse-get-tables-size.sql
Created January 18, 2018 13:43
clickhouse get tables size
SELECT table,
formatReadableSize(sum(bytes)) as size,
min(min_date) as min_date,
max(max_date) as max_date
FROM system.parts
WHERE active
GROUP BY table
@knibals
knibals / redirectExample.go
Created August 16, 2017 09:18 — forked from jaymecd/redirectExample.go
How to redirect HTTP to HTTPS with a golang webserver.
package main
import (
"net/http"
)
func redirect(w http.ResponseWriter, req *http.Request) {
http.Redirect(w, req,
"https://" + req.Host + req.URL.String(),
http.StatusMovedPermanently)
}
@gboudreau
gboudreau / AuthyToOtherAuthenticator.md
Last active November 15, 2024 12:45 — forked from Ingramz/AuthyToOtherAuthenticator.md
Export TOTP tokens from Authy

Exporting your 2FA tokens from Authy to transfer them into another 2FA application

IMPORTANT - Update regarding deprecation of Authy desktop apps

Past August 2024, Authy stopped supported the desktop version of their apps:
See Authy is shutting down its desktop app | The 2FA app Authy will only be available on Android and iOS starting in August for details.

And indeed, after a while, Authy changed something in their backend which now prevents the old desktop app from logging in. If you are already logged in, then you are in luck, and you can follow the instructions below to export to tokens.

If you are not logged in anymore, but can find a backup of the necessary files, then restore those files, and re-install Authy 2.2.3 following the instructions below, and it should work as expected.

@ravibhure
ravibhure / git_rebase.md
Last active November 9, 2024 05:19
Git rebase from remote fork repo

In your local clone of your forked repository, you can add the original GitHub repository as a "remote". ("Remotes" are like nicknames for the URLs of repositories - origin is one, for example.) Then you can fetch all the branches from that upstream repository, and rebase your work to continue working on the upstream version. In terms of commands that might look like:

Add the remote, call it "upstream":

git remote add upstream https://github.com/whoever/whatever.git

Fetch all the branches of that remote into remote-tracking branches, such as upstream/master:

git fetch upstream

@zmagg
zmagg / word_games.py
Last active December 21, 2015 02:38
hacker school mini improvised contest for the fastest solution to find what words are the longest anagrams of each other (single words, please, no phrases) in the sowpods scrabble dictionary.
f = open("sowpods.txt", "r")
canonized_words = {}
word1 = ""
word2 = ""
max_len = 0
for line in f:
if len(line) > max_len:
sorted_word = ''.join(sorted(line))
@heewa
heewa / diff_data.py
Created March 5, 2013 17:52
For when you're comparing two large, deeply hierarchical things that are very slightly different.
def diff_data(d1, d2, d1_name='d1', d2_name='d2'):
"""Return fields in one but not the other, or None if they're the same.
"""
if d1 == d2:
return None
elif type(d1) != type(d2):
return {d1_name: d1, d2_name: d2}
elif isinstance(d1, dict):
diff = {}
for key, v1 in d1.iteritems():