At some point, you might find yourself in need to transfer a file (i.e. a dump of the database, or a backup tarball) quickly over a WAN link. Unfortunately, regular copy speeds are impaired by TCP protocol and latency and speed of light - and link bandwidth has very limited effect on the transfer speeds. However, there is a workaround: one can use UDP-based file transfer (UDT). Here is a proof-of-concept experiment that demonstrates the transfer speed improvement.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
# | |
# Ad-hoc monitoring for time_squeeze changes in /proc/net/softnet_stat . | |
# | |
# usage: ./softnetstat.py <refresh_interval_seconds> | |
# | |
# see https://blog.packagecloud.io/eng/2016/06/22/monitoring-tuning-linux-networking-stack-receiving-data/ | |
# | |
from time import sleep | |
from datetime import datetime |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# see https://github.com/iovisor/bpftrace/blob/master/docs/tutorial_one_liners.md | |
# histogram for udp_recvmsg timing | |
sudo bpftrace -e 'kprobe:udp_recvmsg { @start[tid] = nsecs; } kretprobe:udp_recvmsg /@start[tid]/ { @ns[comm] = hist(nsecs - @start[tid]); delete(@start[tid]); }' | |
# histogram for retval of udp_recvmsg (which is message size in bytes) | |
sudo bpftrace -e 'kretprobe:udp_recvmsg { @bytes[comm] = hist(retval); }' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cat ~/.kube/config | grep client-certificate | sed -e ‘s/ client-certificate-data: //’ | base64 -d | openssl x509 -in - -noout -text |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SSH_ENV="$HOME/.ssh/agent-environment" | |
function start_agent { | |
echo "Initialising new SSH agent..." | |
/usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}" | |
echo succeeded | |
chmod 600 "${SSH_ENV}" | |
. "${SSH_ENV}" > /dev/null | |
/usr/bin/ssh-add; | |
} |
- iTerm2
- iStat menus
- VS code
- ObjeciveSee: BlockBlock, KnockKnock
- Homebrew: ansible, telegraf, bash5, curl, wget, sqlite, python3, go,
- Commons: Chrome, Chromium, FF, Kindle, VLC
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
go install -v github.com/projectdiscovery/dnsx/cmd/dnsx@latest | |
go install -v github.com/projectdiscovery/subfinder/v2/cmd/subfinder@latest | |
brew install ipinfo-cli | |
./go/bin/subfinder -d google.com | ./go/bin/dnsx -resp-only | ipinfo summarize |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://stackoverflow.com/questions/614795/simulate-delayed-and-dropped-packets-on-linux | |
https://wiki.linuxfoundation.org/networking/netem | |
```bash | |
TARGET1='8.8.8.8' | |
DEV='eth0' | |
tc qdisc delete dev $DEV root | |
tc qdisc add dev $DEV root handle 1: prio |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -sha256 -days 365 -nodes |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
TARGET_FILE_NUM=$1 | |
CURRENT_FILE_NUM=`ls -l | wc -l` | |
FILES_TO_GENERATE=$((TARGET_FILE_NUM - CURRENT_FILE_NUM)) | |
if [[ $FILES_TO_GENERATE -lt 0 ]]; then | |
echo "We already have $CURRENT_FILE_NUM files, no need to generate more" | |
exit | |
fi |