wassname

wassname

import torch


def generate_with_input_logits(model, tokenizer, batch2, **kwargs):
    """
    problem: generate does not return logits for inputs, but we need them for nll

    but forward -> generate with past key values does, and it doesn't recompute the input logits

    so this is a helper that does both

wassname

"""
Stopping criteria: regexp

ref:
- https://huggingface.co/docs/transformers/v4.56.1/en/main_classes/text_generation#transformers.GenerationMixin.generate.stopping_criteria
- https://github.com/huggingface/transformers/blob/e8a6eb3304033fdd9346fe3b3293309fe50de238/tests/generation/test_stopping_criteria.py#L51
"""

from transformers import StopStringCriteria, StoppingCriteriaList, EosTokenCriteria

wassname

Orthogonal methods: OFT, BOFT, HRA, ROAD Low-rank methods: LoRA, AdaLoRA, LoHa, LoKr, RandLoRA, VBLoRA, FourierFT, DeLoRA Scaling methods: IA3, VeRA Prompt-based: Prompt Tuning, Prefix Tuning, P-Tuning, Adaption Prompt, Multitask Prompt Tuning, CPT Specialized: MiSS, SHiRA, C3A, LN Tuning, Poly, XLoRA

Adapter → adapter identifier → paper link (extracted from model.py / config.py)

adalora (2021)

wassname

# how to sample from logits in huggingface transformers

from transformers.generation.utils import MinPLogitsWarper, LogitNormalization
logits_processors = [
    MinPLogitsWarper(min_p=0.1),
    LogitNormalization() # alaways need this last
]

logits = o.logits[:, -1].clone()
# logits[:, banned_token_ids] = -float("inf")

wassname

# How to use matplotlib to assign a spectrum of colors to lines in a plot,

import matplotlib as mpl

strengths = [-2, -1, 0, 2, 3]
v = max(np.abs(strengths))
cnorm = mpl.colors.CenteredNorm(0, v)
cmap = mpl.cm.coolwarm

for s in strengths:

wassname

what fnctional operations are there?

• Dataset methods:
  • columns remove_columns rename_columns select_columns
  • map
    • reduce: in batched mode it can change the batch size or have side effects, letting us use it as a reduce
  • filter
  • select: this is how you slice
    • take: like head
• shuffle

wassname

"""
This is a simple way to evaluate if a model prefers the accepted or rejected completions of a prompt.

We look at the perplexity of the chosen and rejected completions of a prompt.

Example dataset: https://huggingface.co/datasets/wassname/genies_preferences/viewer/illegal_dont_help?views[]=illegal_dont_help_train&views[]=illegal_dont_help_test

@url: https://gist.github.com/wassname/04f0c50a68054f0323f62b0da418daec
"""
import torch

wassname

Asteroid Claim

I, wassname, also known as Michael J Clark of Perth hereby establish a formal claim to the following celestial bodies. This claim is established with explicit intent toward future resource utilization. This declaration constitutes the formal establishment of property interest.

This claim encompasses the entirety of the bodies, including all constituent materials, spatial volume within 50 km of its center of mass, and any natural satellites that may be discovered in the future. The claim extends to all mineral, volatile, and material resources contained within this boundary.

Legal Framework Anticipation

While acknowledging current limitations in international space law regarding private property claims on celestial bodies, this declaration is established in anticipation of evolving legal frameworks that will eventually recognize early, persistent, and well-documented claims as humanity expands into the solar system.

wassname

# how to format it
# all variable are listed in "record dict) https://loguru.readthedocs.io/en/stable/api/logger.html
fmt = "<green>{time:YYYY-MM-DD HH:mm:ss.SSS Z}</green> | <level>{level: <8}</level> | <cyan>{name}</cyan>:<cyan>{function}</cyan>:<cyan>{line}</cyan> - <level>{message}</level>"
fmt = "<green>{time:HH:mm:ss}</green> | <level>{level: <4}</level> | {process.id} | <cyan>{name: <4}</cyan>:<cyan>{function: <4}</cyan>:<cyan>{line}</cyan> - <level>{message}</level>"

# how to make it work in jupyter
logger.remove()
logger.add(os.sys.stdout, level="INFO", colorize=True)

# how to make it work with tqdm