| """ | |
| An Minimal Implementation of IMM (Inductive Moment Matching) | |
| """ | |
| import math | |
| import torch | |
| import torch.nn.functional as F | |
| def compute_mmd_loss_fully_vectorized( |
| ## Note | |
| ## if use vllm in same gpu, remember to set a low gpu_memory_utilization to avoid OOM | |
| ## For larger model please consider to use multi-GPU or CPU offloading | |
| ## AnySchedule: https://github.com/KohakuBlueleaf/AnySchedule | |
| ## LyCORIS: https://github.com/KohakuBlueleaf/LyCORIS | |
| ## Following code can perform reasonable training on Llama-3.2-1B-Instruct model with GSM8K dataset | |
| ## With noticable improvement on each reward function | |
| from itertools import chain | |
| import re |
Gradio's reverse proxy service sucks. The WebUI program integrates ngrok to fix this issue which works well but Paperspace locks your account for using any tunneling proxy service such as (who knows how many they explicitly ban).
There is literally no easy solution for this and you must to host your own server that Paperspace doesn't know about. What I do is run a rathole server on a $4 DigitalOcean VPS.