tsukanov-as

• What do Etcd, Consul, and Zookeeper do?
  • Service Registration:
    • Host, port number, and sometimes authentication credentials, protocols, versions numbers, and/or environment details.
  • Service Discovery:
    • Ability for client application to query the central registry to learn of service location.
  • Consistent and durable general-purpose K/V store across distributed system.
    • Some solutions support this better than others.
    • Based on Paxos or some derivative (i.e. Raft) algorithm to quickly converge to a consistent state.
• Centralized locking can be based on this K/V store.

tsukanov-as

#netcat proxy to a different backed and serve requests on port80
mkfifo fifo_pipe
nc -lk -p 80 < fifo_pipe | nc 192.168.1.10 3306 >fifo_pipe 


#socat doing the same with connection verbosity
socat -d -d TCP-LISTEN:80,fork TCP:192.168.1.10:3306

tsukanov-as

1. Download docker-compose.yml to dir named sentry
2. Change SENTRY_SECRET_KEY to random 32 char string
3. Run docker-compose up -d
4. Run docker-compose exec sentry sentry upgrade to setup database and create admin user
5. (Optional) Run docker-compose exec sentry pip install sentry-slack if you want slack plugin, it can be done later
6. Run docker-compose restart sentry
7. Sentry is now running on public port 9000

• https://docs.docker.com/compose/compose-file/#env_file

tsukanov-as

import os
import argparse
import hashlib
import hf_transfer # py -m pip install hf-transfer

CHUNK_SIZE = 10_485_760

parser = argparse.ArgumentParser(description='HuggingFace fast model downloader')
parser.add_argument('--url', type=str, help='file url')
args = parser.parse_args()

tsukanov-as

import numpy as np
from numpy import pi
# import matplotlib.pyplot as plt

N = 400
theta = np.sqrt(np.random.rand(N))*2*pi # np.linspace(0,2*pi,100)

r_a = 2*theta + pi
data_a = np.array([np.cos(theta)*r_a, np.sin(theta)*r_a]).T
x_a = data_a + np.random.randn(N,2)

tsukanov-as

"""
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
BSD License
"""
import numpy as np

# data I/O
data = open('input.txt', 'r').read() # should be simple plain text file
chars = list(set(data))
data_size, vocab_size = len(data), len(chars)

tsukanov-as

Here's a list of mildly interesting things about the C language that I learned mostly by consuming Clang's ASTs. Although surprises are getting sparser, I might continue to update this document over time.

There are many more mildly interesting features of C++, but the language is literally known for being weird, whereas C is usually considered smaller and simpler, so this is (almost) only about C.

1. Combined type and variable/field declaration, inside a struct scope [https://godbolt.org/g/Rh94Go]

struct foo {
   struct bar {
 int x;

tsukanov-as

• example: https://medium.com/swlh/build-and-use-go-packages-as-c-libraries-889eb0c19838

https://medium.com/@walkert/fun-building-shared-libraries-in-go-639500a6a669

https://www.darkcoding.net/software/building-shared-libraries-in-go-part-2/

https://karthikkaranth.me/blog/calling-c-code-from-go/

tsukanov-as

Latency Comparison Numbers (~2012)
----------------------------------
L1 cache reference                           0.5 ns
Branch mispredict                            5   ns
L2 cache reference                           7   ns                      14x L1 cache
Mutex lock/unlock                           25   ns
Main memory reference                      100   ns                      20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy             3,000   ns        3 us
Send 1K bytes over 1 Gbps network       10,000   ns       10 us
Read 4K randomly from SSD*             150,000   ns      150 us          ~1GB/sec SSD

tsukanov-as

Mike's x86-64 Assembly (ASM) Notes

Assembling Binary Machine Code

Operating Modes:

These determine the assumed/default size of instruction operands, and restricts which opcodes are available, and how they are used.

Modern operating systems, booted inside Real mode,