Dominik Weckmüller do-me
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # clone folder from s3 to local recursively | |
| aws s3 cp s3://some_folder ./some_folder/ --recursive | |
| # get recursive size of folder on s3, human readable | |
| aws s3 ls s3://some_folder/ \ | |
| --recursive --summarize | | |
| awk ' | |
| /Total Objects/ {print} | |
| /Total Size/ { | |
| size=$3 |
do-me
/ docx_xlsx_to_md.sh
Created
May 26, 2026 08:16
Docx and xlsx to markdown with inline comments from reviewers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # One-shot run — uvx fetches the repo into an isolated env, runs the CLI, then cleans up | |
| uvx --from git+https://github.com/do-me/docx-comments-to-text docx-comments-to-text path/to/file.docx | |
| uvx --from git+https://github.com/do-me/docx-comments-to-text docx-comments-to-text path/to/file.xlsx --sheet "Sheet1" -o out.md |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ⏺ # 1. Pick a destination | |
| DEST=~/data/osm_boundaries.parquet | |
| # 2. Download with resume support (the server can stall mid-stream) | |
| curl -SL -C - --max-time 2400 \ |
do-me
/ duckdb_one_liners.sh
Last active
April 9, 2026 13:37
Useful Parquet oneliners, view and manipulate with DuckDB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # view 10 lines | |
| uvx duckdb -c "FROM 'results.parquet' LIMIT 10" | |
| # add 2 new columns | |
| uvx duckdb -c "COPY (SELECT *, NULL::INT AS dominik_label, NULL::VARCHAR AS dominik_comments FROM 'results.parquet') TO 'results.parquet'" | |
| # remove 2 columns | |
| uvx duckdb -c "COPY (SELECT * EXCLUDE (dominik_label, dominik_comments) FROM 'results.parquet') TO 'results.parquet'" | |
| # sort by 2 columns |
do-me
/ download.py
Created
February 19, 2026 17:08
Download all EO metadata from WEkEO platform via API
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # /// script | |
| # dependencies = [ | |
| # "requests", | |
| # "pandas", | |
| # "pyarrow", | |
| # "tqdm", | |
| # ] | |
| # /// | |
| import requests |
do-me
/ delete_files_in_hf_dataset.py
Created
February 16, 2026 16:59
Delete wrongly prefixed files in a hf dataset
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from huggingface_hub import HfApi, CommitOperationDelete | |
| # Configure your repo details | |
| repo_id = "user/reponame" | |
| token = "your token" | |
| api = HfApi(token=token) | |
| # 1. List files specifically in the target folder | |
| target_folder = "files/2026" |
do-me
/ dir_size.sh
Created
February 16, 2026 15:58
Get size of dir, number of files and average file size
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| read s c < <(find images -type f -printf '%s\n' 2>/dev/null | awk '{t+=$1} END{print t, NR}'); printf "Total size: %s\nFiles: %d\nAverage: %s\n" "$(numfmt --to=iec --suffix=B $s)" "$c" "$(numfmt --to=iec --suffix=B $((c? s/c : 0)))" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| tmux kill-server | |
| tmux new -s mysession | |
| exit | |
| tmux kill-session -t mysession |
do-me
/ app.py
Last active
February 15, 2026 19:49
— forked from Maxxen/app.py
DuckDB Vector Tile Serve w/ Flask + MapLibre. Loads geoparquet file with spatial sampling and uv tooling. Run with uv run app.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # /// script | |
| # dependencies = [ | |
| # "duckdb", | |
| # "flask" | |
| # ] | |
| # /// | |
| import duckdb | |
| import flask | |
| import gzip |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from huggingface_hub import HfApi, CommitOperationDelete, RepoFile | |
| # Configure your repo details | |
| repo_id = "user/repo" | |
| token = "your token" # Ensure your token has 'write' permissions | |
| api = HfApi(token=token) | |
| # 1. List files in the repo (non-recursive) | |
| files = api.list_repo_tree(repo_id, repo_type="dataset") |
NewerOlder