Skip to content

Instantly share code, notes, and snippets.

View ssghost's full-sized avatar
🏠
Working from home

Stardust Song ssghost

🏠
Working from home
View GitHub Profile
git clone https://github.com/cpacker/MemGPT.git
conda create -n memgpt python=3.10
conda activate memgpt
cd memgpt
pip install -r requirements.txt
export OPENAI_API_KEY="YOUR_API_KEY"
# download .txt docs from: https://huggingface.co/datasets/MemGPT/example-sec-filings/tree/main
# place in "memgpt/personas/examples/preload_archival/*.txt"
python3 main.py --archival_storage_files_compute_embeddings="memgpt/personas/examples/preload_archival/*.txt" --persona=memgpt_doc --human=basic
@ssghost
ssghost / llama2-mac-gpu.sh
Created July 20, 2023 00:58 — forked from adrienbrault/llama2-mac-gpu.sh
Run Llama-2-13B-chat locally on your M1/M2 Mac with GPU inference. Uses 10GB RAM
# Clone llama.cpp
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
# Build it
LLAMA_METAL=1 make
# Download model
export MODEL=llama-2-13b-chat.ggmlv3.q4_0.bin
wget "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/${MODEL}"

GitHub Search Syntax for Finding API Keys/Secrets/Tokens

As a security professional, it is important to conduct a thorough reconnaissance. With the increasing use of APIs nowadays, it has become paramount to keep access tokens and other API-related secrets secure in order to prevent leaks. However, despite technological advances, human error remains a factor, and many developers still unknowingly hardcode their API secrets into source code and commit them to public repositories. GitHub, being a widely popular platform for public code repositories, may inadvertently host such leaked secrets. To help identify these vulnerabilities, I have created a comprehensive search list using powerful search syntax that enables the search of thousands of leaked keys and secrets in a single search.

Search Syntax:

(path:*.{File_extension1} OR path:*.{File_extension-N}) AND ({Keyname1} OR {Keyname-N}) AND (({Signature/pattern1} OR {Signature/pattern-N}) AND ({PlatformTag1} OR {PlatformTag-N}))

Examples:

**1.

@ssghost
ssghost / delhist.sh
Last active June 11, 2023 03:44 — forked from maurelian/gist:859d8bbce79ef8facdd9eb8493f9b029
A zsh function to delete n lines of .zsh_history
delhist() {
local num_lines="$1"
local file="$HOME/.zsh_history"
if [[ ! "$num_lines" =~ ^[0-9]+$ ]]; then
echo "Invalid number of lines: $num_lines"
return 1
fi
local total_lines=$(wc -l < "$file")
@ssghost
ssghost / brew-upall.sh
Last active June 7, 2023 05:11 — forked from mikker/gist:631106
Update all packages installed with homebrew
brew update && brew upgrade && brew cleanup
#pip install git+https://github.com/huggingface/transformers.git
import datetime
import sys
from transformers import pipeline
from transformers.pipelines.audio_utils import ffmpeg_microphone_live
pipe = pipeline("automatic-speech-recognition", model="openai/whisper-base", device=0)
sampling_rate = pipe.feature_extractor.sampling_rate
@ssghost
ssghost / tpu_topology_env_vars.py
Created November 7, 2022 11:15 — forked from skye/tpu_topology_env_vars.py
You can use these environment variables to run a Python process on a subset of the TPU cores on a Cloud TPU VM. This allows running multiple TPU processes at the same time, since only one process can access a given TPU chip at a time. Note that in JAX, 1 TPU core = 1 TpuDevice as reported by `jax.devices()`, and each TPU chip has 2 cores.
# 4x 1 chip (2 cores) per process:
os.environ["TPU_CHIPS_PER_HOST_BOUNDS"] = "1,1,1"
os.environ["TPU_HOST_BOUNDS"] = "1,1,1"
# Different per process:
os.environ["TPU_VISIBLE_DEVICES"] = "0" # "1", "2", "3"
# Pick a unique port per process
os.environ["TPU_MESH_CONTROLLER_ADDRESS"] = "localhost:8476"
os.environ["TPU_MESH_CONTROLLER_PORT"] = "8476"
# 1-liner for bash: TPU_CHIPS_PER_HOST_BOUNDS=1,1,1 TPU_HOST_BOUNDS=1,1,1 TPU_VISIBLE_DEVICES=0 TPU_MESH_CONTROLLER_ADDRESS=localhost:8476 TPU_MESH_CONTROLLER_PORT=8476

AGX coherency, caching, and TLBs

These are just some notes on my current understanding of the subtleties of the AGX memory model and the TLB/caching issues I'm seeing.

GPU side

MMU (UAT)

The AGX MMU has 64 context slots (63 usable for user contexts). The page table base addresses are stored in a page (Apple calls it "TTBAT") in main memory. There are user and kernel halves (low and high), but we can ignore the high half from the GPU perspective since it is only used in very specific cases. The permission bits are funny and the same page tables are shared (and interpreted differently) by the ASC, but they are basically ARM page tables. Each context slot has an ASID (how many bits?), like a TTBR in ARM.

// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;
library Iterators {
// Function types:
// https://docs.soliditylang.org/en/latest/types.html#function-types
function map(uint256[] memory input, function (uint256) internal pure returns (uint256) f)
internal
pure
@ssghost
ssghost / cli.js
Created September 13, 2022 22:15 — forked from mattdesl/cli.js
colour palette from text prompt using Stable Diffusion https://twitter.com/mattdesl/status/1569457645182152705
/**
* General-purpose NodeJS CLI/API wrapping the Stable-Diffusion python scripts.
*
* Note that this uses an older fork of stable-diffusion
* with the 'txt2img.py' script, and that script was modified to
* support the --outfile command.
*/
var { spawn, exec } = require("child_process");
var path = require("path");