Skip to content

Instantly share code, notes, and snippets.

View lewtun's full-sized avatar
🤠

lewtun

🤠
View GitHub Profile
@lewtun
lewtun / config.yaml
Created April 16, 2026 08:59
4B SFT long context
# Model arguments
model_name_or_path: Qwen/Qwen3-4B-Instruct-2507
model_revision: main
attn_implementation: kernels-community/vllm-flash-attn3
# Data training arguments
dataset_mixture:
datasets:
- id: hf-carbon/seqqa-sft-v1_gemma-4-31B-it
config: graded_correct
@lewtun
lewtun / trackio-db-bug.md
Last active April 11, 2026 10:44
Trackio debug

Primary root cause

  • In Spaces, Trackio forces SQLite into DELETE journal mode on every connection, not just once at DB creation. See trl- internal/lib/python3.11/site-packages/trackio/sqlite_storage.py:43 and trl-internal/lib/python3.11/site-packages/trackio/ sqlite_storage.py:101.
  • Write paths do take a per-project file lock, for example in init_db and bulk_log. See trl-internal/lib/python3.11/site- packages/trackio/sqlite_storage.py:143 and trl-internal/lib/python3.11/site-packages/trackio/sqlite_storage.py:623.
  • Read/UI paths do not take that lock. get_alerts and get_logs open DB connections directly, and those connections still run _configure_sqlite_pragmas(). See trl-internal/lib/python3.11/site-packages/trackio/sqlite_storage.py:809 and trl-internal/ lib/python3.11/site-packages/trackio/sqlite_storage.py:967. Your container traces are failing in that pragma path before the
@lewtun
lewtun / mcp-bench-sample-prompt.txt
Created September 6, 2025 10:21
MCP-Bench Sample Prompt
<|im_start|>system
You are a strategic multi-tool AI agent planner for Round 1. Plan parallel execution of independent tools for this round.<|im_end|>
<|im_start|>user
You are a strategic decision-making expert for a multi-tool AI agent using the provided tools to perform the task.
TASK: "Hey, I’m working on this new dashboard that pulls search results from three different services—one for AI stuff, one for code hosting, and one for edge networking—and I’m scratching my head over how each handles pagination. Some APIs might use a page/page_size setup, others a cursor or next_cursor, and I’m not even sure if all of them support paging in their search calls or if I have to switch to their “list” routes instead.
Could you dig into each service’s search endpoints and tell me:
• whether it pages at all or not
• if it does, what style it uses (page numbers, cursors, etc.)
# pip install emoji
import argparse
from datasets import load_dataset
import emoji
def remove_emoji(text: str) -> str:
return emoji.replace_emoji(text, replace='').strip()
def format_messages(x):
emojis_found = False
@lewtun
lewtun / chat_template.jinja
Last active July 8, 2025 16:01
SmolLM3 chat template
{# ───── defaults ───── #}
{%- if enable_thinking is not defined -%}
{%- set enable_thinking = true -%}
{%- endif -%}
{# ───── reasoning mode ───── #}
{%- if enable_thinking -%}
{%- set reasoning_mode = "/think" -%}
{%- else -%}
{%- set reasoning_mode = "/no_think" -%}
@lewtun
lewtun / grpo_benchmark_v0.py
Created March 3, 2025 14:04
GRPO benchmarking
from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig, set_seed
import time
import torch
set_seed(0)
device = "cuda"
model = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen2.5-1.5B",
attn_implementation="flash_attention_2",
torch_dtype="bfloat16"
@lewtun
lewtun / grpo_vllm.py
Last active January 29, 2025 10:51
GRPO with vLLM demo
from datasets import load_dataset
from trl import GRPOConfig, GRPOTrainer
import random
"""Usage (on 8 x H100s):
pip install vllm==0.7.0 --extra-index-url https://download.pytorch.org/whl/cu121
pip install -e '.[dev]'
# DDP
accelerate launch --config_file examples/accelerate_configs/multi_gpu.yaml --num_processes 7 scratch/grpo_demo.py
@lewtun
lewtun / extract_boxed.py
Last active January 9, 2025 12:19
Simple parser to extract the \boxed{answer} parts from LLM completions
from typing import Optional
def extract_boxed_solution(text: str) -> Optional[str]:
"""
Extracts the content of the last `\boxed{}` in a given LaTeX-style text.
Args:
text (str): The input string containing LaTeX-style content.
Returns:
@lewtun
lewtun / sft_llama.py
Last active November 9, 2024 02:33
SFT Llama 3.1 8B - full training vs LoRA vs QLoRA
# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
@lewtun
lewtun / dpo_winrate.py
Created September 24, 2024 20:45
DPO with WinRateCallback
# flake8: noqa
# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software