KohakuBlueleaf

"""
An Minimal Implementation of IMM (Inductive Moment Matching)
"""

import math
import torch
import torch.nn.functional as F


def compute_mmd_loss_fully_vectorized(

KohakuBlueleaf

## Note
## if use vllm in same gpu, remember to set a low gpu_memory_utilization to avoid OOM
## For larger model please consider to use multi-GPU or CPU offloading
## AnySchedule: https://github.com/KohakuBlueleaf/AnySchedule
## LyCORIS: https://github.com/KohakuBlueleaf/LyCORIS
## Following code can perform reasonable training on Llama-3.2-1B-Instruct model with GSM8K dataset
## With noticable improvement on each reward function
from itertools import chain
import re

KohakuBlueleaf

Sd-WebUI standalone script

just throw the script into the root folder of sd-webui. and then modify the content to what you want to run. And then run it.

If you want to use some extension, just use my implementation as example (to modify the args for extensions)

KohakuBlueleaf

import torch
import torch.nn as nn
import torch.nn.functional as F


class TimestepEmbedding(nn.Module):
    def __init__(self, n_time=1024, n_emb=320, n_out=1280) -> None:
        super().__init__()
        self.emb = nn.Embedding(n_time, n_emb)

KohakuBlueleaf

import asyncio
import cv2
import json
import traceback
from concurrent.futures import ProcessPoolExecutor
from pathlib import Path

import numpy as np
from httpx import AsyncClient, HTTPError, Timeout
from tqdm import tqdm

KohakuBlueleaf

import torch
import torch.nn as nn
import torch.nn.functional as F

from einops import rearrange


def parallel_retention(
    q, k, v,            # bsz, heads, seq_len, dim
    decay_mask = None   # heads, seq_len, seq_len

KohakuBlueleaf

import os, sys

from time import time_ns
from json import load
from io import BytesIO
import math

from PIL import Image

import asyncio

KohakuBlueleaf

import torch
import numpy as np
import timeit

#could be "cpu" "cuda" "opencl" "mps"...
DEVICE = "mps"
ROUNDS = 500
NUM = (2048, 2048)

# Input for benchmarking

KohakuBlueleaf

Benchmark for Xmrig

This is a python script that run a customized stratum server to specific the algorigthm combinations of 6 CN algos(used in ghostrider) And it calculate the Expected Average Hashrate automatically(All the weights are as same as cpuminer-gr's)

How to run

You need to install python3.8↑ with pip first! And download the script to xmrig's path

linux

KohakuBlueleaf

{"lastUpload":"2021-04-03T07:50:27.292Z","extensionVersion":"v3.4.3"}