Harsha K N harshkn

source	https://huggingfacetb-smol-training-playbook.hf.space/
title	The Smol Training Playbook: The Secrets to Building World-Class LLMs
type	Technical Blog Post
organization	Hugging Face
cleaned_by	Crawl4AI

The Smol Training Playbook:

Pre-reqs:

Have Python 3 installed. On macOS, this could be installed from homebrew or even via standard Python 3.6 downloaded installer from https://www.python.org/download. On Linux, just use your package manager.
On macOS:
- Install XCode from the Mac App Store (or install the XCode command line utils).
- Have homebrew installed
On Linux:

	# requires sentence_transformers>=3.2.0
	from sentence_transformers import SentenceTransformer, export_optimized_onnx_model, export_dynamic_quantized_onnx_model

	# The model to export to ONNX (+ optimize, quantize), OpenVINO
	model_id = "mixedbread-ai/mxbai-embed-large-v1"
	# Where to save the exported models locally
	output_dir = model_id.replace("/", "-")

	onnx_model = SentenceTransformer(model_id, backend="onnx", model_kwargs={"export": True})
	onnx_model.save_pretrained(output_dir)

	#!/bin/bash

	# Update and upgrade Homebrew
	echo "Updating Homebrew..."
	brew update
	brew upgrade


	# Install nvm (Node Version Manager)
	echo "Installing nvm..."

	# settings we’re about to change
	osascript -e 'tell application "System Preferences" to quit'
	# Ask for the administrator password upfront
	sudo -v

	# Keep-alive: update existing `sudo` time stamp until `.macos` has finished
	while true; do
	sudo -n true
	sleep 60
	kill -0 "$$" \|\| exit

	# Electromagnetic simulation at varying timescales - simulation code to make this animation:
	# https://twitter.com/bencbartlett/status/1369396941730312193
	#
	# Animation idea was inspired by u/cenit997's (Twitter: @__cenit) Reddit post:
	# https://www.reddit.com/r/Python/comments/hxmhai/a_simulation_of_how_an_incoherent_light_source/


	#---
	using LinearAlgebra, Base.Threads, BenchmarkTools, LoopVectorization, CSV, DataFrames, Printf, Plots

	library(tidyverse)
	library(brickr)

	display_colors()

	random_tree <- function(xloc, yloc, height, radius){
	startz <- round(height * (2/5))

	#Build a 3x3 trunk
	trunk_coords <- expand.grid( x = -1:1, y = -1:1,

	"""
	Simple example of manually performing "automatic" differentiation
	"""
	import numpy as np
	from numpy import exp, sin, cos


	def f(x, with_grad=False):
	# Need to cache intermediates from forward pass (might not use all of them).
	a = exp(x)

	import pyaudio
	import numpy as np
	import time
	import wave
	import matplotlib.pyplot as plt


	# open stream
	FORMAT = pyaudio.paInt16
	CHANNELS = 1

	#!/bin/sh

	command="${*}"
	printf "Initialized REPL for `%s`\n" "$command"
	printf "%s> " "$command"
	read -r input
	while [ "$input" != "" ];
	do
	eval "$command $input"
	printf "%s> " "$command"