razhangwei

Here's a concise cheatsheet for JSON Schema:

Basic Structure

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "https://example.com/schema.json",
  "title": "Schema Title",
  "description": "Schema description",

razhangwei

Basic Structure

Hydra uses YAML-like syntax for configuration files, typically with a .yaml extension.

key: value
nested:
  key: value
list:
  - item1

razhangwei

Key Features

• PyTorch implementations of popular LLMs (Llama, Gemma, Mistral, Phi, Qwen)
• Training recipes for full fine-tuning, LoRA, QLoRA, DPO, PPO, QAT, knowledge distillation
• Memory efficiency and performance improvements
• YAML configs for easy recipe configuration
• Support for various dataset formats and prompt templates

Fine-tuning Recipes

razhangwei

Prompt Engineering

Prompt Forge

• Allow you to quickly test system/user prompt along with self-defined examples
• prompt template with placeholder variables.
• Evaluators

Inference

razhangwei

# Create an interactive bar chart with two bars per date
fig = go.Figure()
fig.add_trace(go.Bar(
    x=daily_formula_intake.index,
    y=daily_formula_intake.values,
    name='Formula'
))
fig.add_trace(go.Bar(
    x=daily_expressed_intake.index,
    y=daily_expressed_intake.values,

razhangwei

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

class LanguageModel(nn.Module):
  def __init__(self):
      super().__init__()
      # Define transformer layers, embeddings, etc.

razhangwei

# Pseudocode for estimating advantage function in RLHF using PPO

import numpy as np
import torch
import torch.nn.functional as F

def compute_ppo_loss(policy, old_policy, token_sequences, advantages, returns, clip_epsilon=0.2):
  """
  Compute the PPO loss for language model policy update
  

razhangwei

• quite powerful as it simplies the writing the system prompt

• often need an extra step of post processing, which can be done by LLM or traditional programs.

• structured output vs json mode:

  • structured output: 100% gurantee schema; schema doesn't consume tokens

• json: scheme usually goes to system prompts, costing tokens, doesn't guarantee json or scheme correctness.

razhangwei

pipx install llm
llm install llm-gemini 
llm -m gemini-1.5-flash-latest

razhangwei

## sudo visudo -f /etc/sudoers.d/custom_config


# /etc/sudoers.d/custom_config

# Extend the sudo ticket lifetime to 2 hours (7200 seconds)
Defaults timestamp_timeout=7200

# Allow your user to run the script without a password
wei ALL=(ALL) NOPASSWD: /Users/wei/start_nano_transcriber.sh