andrewssobral

# Requires:
# pip install pyobjc-framework-Metal
import numpy as np
import Metal

# Get the default GPU device
device = Metal.MTLCreateSystemDefaultDevice()

# Make a command queue to encode command buffers to
command_queue = device.newCommandQueue()

andrewssobral

#!/bin/bash

# set -x  # Enable debug mode
set -e  # Exit immediately if a command exits with a non-zero status.

echo "Script started" >&2

# Function to get absolute path (works on macOS and Linux)
get_abs_path() {
    local path="$1"

andrewssobral

andrewssobral

# This is a modified version of TRL's `SFTTrainer` example (https://github.com/huggingface/trl/blob/main/examples/scripts/sft_trainer.py), 
# adapted to run with DeepSpeed ZeRO-3 and Mistral-7B-V1.0. The settings below were run on 1 node of 8 x A100 (80GB) GPUs.
#
# Usage:
#   - Install the latest transformers & accelerate versions: `pip install -U transformers accelerate`
#   - Install deepspeed: `pip install deepspeed==0.9.5`
#   - Install TRL from main: pip install git+https://github.com/huggingface/trl.git
#   - Clone the repo: git clone github.com/huggingface/trl.git
#   - Copy this Gist into trl/examples/scripts
#   - Run from root of trl repo with: accelerate launch --config_file=examples/accelerate_configs/deepspeed_zero3.yaml --gradient_accumulation_steps 8 examples/scripts/sft_trainer.py  

andrewssobral

docker run \
--rm \
-it \
-e ONNXRUNTIME_REPO=https://github.com/microsoft/onnxruntime \
-e ONNXRUNTIME_COMMIT=v1.15.1 \
-e BUILD_CONFIG=Release \
-e CMAKE_VERSION=3.26.4 \
-e CPU_ARCHITECTURE=$(uname -m) \
-e CUDA_ARCHITECTURES="70;75;80;86" \
-v /usr/lib/aarch64-linux-gnu/tegra:/usr/lib/aarch64-linux-gnu/tegra:ro \

andrewssobral

# Clone llama.cpp
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp

# Build it
LLAMA_METAL=1 make

# Download model
export MODEL=llama-2-13b-chat.ggmlv3.q4_0.bin
wget "https://huggingface.co/TheBloke/Llama-2-13B-chat-GGML/resolve/main/${MODEL}"

andrewssobral

tmux Cheat Sheet

Table of Contents

• General Usage
• Shortcuts
• Commands
• Scripting tmux
• Configuring tmux

andrewssobral

import openai
import asyncio
from typing import Any

async def dispatch_openai_requests(
    messages_list: list[list[dict[str,Any]]],
    model: str,
    temperature: float,
    max_tokens: int,
    top_p: float,

andrewssobral

X11 forwarding on macOS and docker

A quick guide on how to setup X11 forwarding on macOS when using docker containers requiring a DISPLAY. Works on both Intel and M1 macs!

This guide was tested on:

• macOS Catalina 10.15.4
• docker desktop 2.2.0.5 (43884) - stable release
• XQuartz 2.7.11 (xorg-server 1.18.4)
• Macbook Pro (Intel)

andrewssobral

1. Copy m3u8 link

2. Run command

echo "Enter m3u8 link:";read link;echo "Enter output filename:";read filename;ffmpeg -i "$link" -bsf:a aac_adtstoasc -vcodec copy -c copy -crf 50 $filename.mp4