torchtune cheatsheet

torchtune.md

Key Features

• PyTorch implementations of popular LLMs (Llama, Gemma, Mistral, Phi, Qwen)
• Training recipes for full fine-tuning, LoRA, QLoRA, DPO, PPO, QAT, knowledge distillation
• Memory efficiency and performance improvements
• YAML configs for easy recipe configuration
• Support for various dataset formats and prompt templates

Fine-tuning Recipes

Training	Fine-tuning Method
Distributed (1-8 GPUs)	Full, LoRA
Single Device / Low Memory (1 GPU)	Full, LoRA + QLoRA
Single Device (1 GPU)	DPO, RLHF with PPO

CLI Commands

• List available configs: tune ls
• Copy a config: tune cp <source> <destination>
• Download a model: tune download <model_name> --output-dir <dir> --hf-token <token>
• Run a recipe: tune run <recipe_name> --config <config_file>
• Validate a config: tune validate <config_file>

Modifying Configs

1. Command-line overrides:

tune run lora_finetune_single_device \
  --config llama2/7B_lora_single_device \
  batch_size=8 \
  enable_activation_checkpointing=True \
  max_steps_per_epoch=128

2. Local copy modification:

tune cp llama3_1/8B_full ./my_custom_config.yaml
tune run full_finetune_distributed --config ./my_custom_config.yaml

Integrations

• Comet ML: Add CometLogger to your config for experiment tracking
• EleutherAI's Eval Harness: Use the eleuther_eval recipe for model evaluation

Supported Models

• Llama 2 (7B, 13B, 70B)
• Llama 3.1 (8B, 70B, 405B)
• Llama 3.2 (1B, 3B, 11B Vision)
• Mistral (7B)
• Gemma (2B, 7B)
• Phi (1.5B, 2.7B)
• Qwen (1.8B, 7B, 14B, 72B)

Best Practices

• Use single-device configs for memory-constrained setups
• Leverage memory optimizations like activation checkpointing
• Experiment with different fine-tuning methods based on your hardware and dataset size
• Utilize the CLI for easy config management and recipe execution

Remember to check the official documentation for detailed information on specific features and advanced usage[1][2][4][5].

Citations: [1] https://pytorch.org/torchtune/ [2] https://github.com/pytorch/torchtune/blob/main/README.md [3] https://www.comet.com/docs/v2/integrations/third-party-tools/torchtune/ [4] https://pytorch.org/torchtune/stable/tutorials/e2e_flow.html [5] https://github.com/pytorch/torchtune/