Skip to content

Instantly share code, notes, and snippets.

@OhadRubin
OhadRubin / docker how much space
Last active April 5, 2024 17:35
docker how much space
sudo ncdu /
sudo docker container ls -a --format '{{.ID}}:{{.Names}}' | xargs -I {} sh -c 'echo "Container {} ($(echo {} | cut -d: -f2)): $(sudo docker inspect --format "{{.GraphDriver.Data.MergedDir}}" $(echo {} | cut -d: -f1) | sed "s/\/merged$//") (Size: $(sudo du -sh $(sudo docker inspect --format "{{.GraphDriver.Data.MergedDir}}" $(echo {} | cut -d: -f1) | sed "s/\/merged$//") | cut -f1))"'
sudo docker image ls --format '{{.ID}}:{{.Repository}}:{{.Tag}}' | xargs -I {} sh -c 'echo "Image {} ($(echo {} | cut -d: -f2):$(echo {} | cut -d: -f3)): $(sudo docker inspect --format "{{.GraphDriver.Data.UpperDir}}" $(echo {} | cut -d: -f1) | sed "s/\/diff$//") (Size: $(sudo du -sh $(sudo docker inspect --format "{{.GraphDriver.Data.UpperDir}}" $(echo {} | cut -d: -f1) | sed "s/\/diff$//") | cut -f1))"'
@OhadRubin
OhadRubin / inc_read_pickle.py
Created June 2, 2024 08:56
incrementally read a pickle file and show progress
import pickle
import numpy as np
from tqdm import tqdm
import os
# Create a large numpy array and save it to a pickle file
# a = np.random.random((1000, 1000))
a = np.random.random((40000,10000))
with open('large_array.pkl', 'wb') as f:
pickle.dump(a, f)
@OhadRubin
OhadRubin / inverse_square_root_schedule.py
Created July 22, 2024 15:39
the inverse_square_root_schedule from PaliGemma and Scaling Vision Transformers
import numpy as np
import jax
import matplotlib.pyplot as plt
import jax.numpy as jnp
def inverse_square_root_schedule(
max_lr: float,
warmup_steps: int,
):
def schedule(count):
@OhadRubin
OhadRubin / download_repo_folder.py
Created August 25, 2024 10:50
How to pull specific directory with git
import git # !pip install GitPython
import os
def download_repo_folder(repo_path, url, dirs_to_checkout, branch="main"):
# Initialize the repo (make sure the path is where you want to clone)
os.makedirs(repo_path, exist_ok=True)
repo = git.Repo.init(repo_path)
@OhadRubin
OhadRubin / btree.py
Created September 25, 2024 10:20
B-tree in python
class BTreeNode:
def __init__(self, t, leaf=False):
"""
Initialize a B-tree node.
Args:
t (int): Minimum degree of B-tree node.
leaf (bool, optional): True if node is a leaf. Defaults to False.
"""
self.t = t # Minimum degree (defines the range for number of keys)
@OhadRubin
OhadRubin / add_line_numbers.py
Created October 1, 2024 09:35
add_line_numbers.py
def add_line_numbers(input_file, output_file):
with open(input_file) as f:
text = f.read()
output_text = "\n".join([f"{i} {line}" for i, line in list(enumerate(text.split("\n")))])
with open(output_file, "w") as f:
f.write(output_text)
Understand the Task: Grasp the main objective, goals, requirements, constraints, and expected output.
- Minimal Changes: If an existing prompt is provided, improve it only if it's simple. For complex prompts, enhance clarity and add missing elements without altering the original structure.
- Reasoning Before Conclusions: Encourage reasoning steps before any conclusions are reached. ATTENTION! If the user provides examples where the reasoning happens afterward, REVERSE the order! NEVER START EXAMPLES WITH CONCLUSIONS!
- Reasoning Order: Call out reasoning portions of the prompt and conclusion parts (specific fields by name). For each, determine the ORDER in which this is done, and whether it needs to be reversed.
- Conclusion, classifications, or results should ALWAYS appear last.
- Examples: Include high-quality examples if helpful, using placeholders [in brackets] for complex elements.
- What kinds of examples may need to be included, how many, and whether they are complex enough to benefit from p
@OhadRubin
OhadRubin / tol_verbalizers.md
Created November 29, 2024 10:42
Thinking out loud Verbalizers

Hmm... Well, let me see... I'm wondering if... This makes me think of... Building on that... Wait a minute... Going back to what I said about... Something's not quite right... Hold that thought... Here's where it gets interesting...

@OhadRubin
OhadRubin / create_enc_script.sh
Created December 27, 2024 09:30
create_enc_script.sh
#!/bin/bash
# create_enc_script.sh - Creates an encrypted script containing API keys and uploads to GitHub Gist
# Usage: bash create_enc_script.sh --path_to_dot_env_file .my_tmp --password your_password --github_token $GITHUB_ACCESS_TOKEN
# Function to show usage
show_usage() {
echo "Usage: $0 --path_to_dot_env_file <path> --password <password> --github_token <token>"
exit 1
}
@OhadRubin
OhadRubin / animate_svg.py
Created December 31, 2024 08:32
animate_svg.py
# pip install svgelements
import io
from svgelements import SVG, Path
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
import numpy as np
def read_svg(svg_file_path):
try:
with io.open(svg_file_path, "r", encoding="utf-8") as f: