NickyDark1

NickyDark1 / train_adapters_transformers.py

Created August 30, 2023 03:06 — forked from younesbelkada/train_adapters_transformers.py

Train adapters using transformers integration of PEFT

	from datasets import load_dataset
	import torch
	from peft import LoraConfig, prepare_model_for_int8_training
	from trl import SFTTrainer
	from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, AutoTokenizer, TrainingArguments

	dataset_name = "timdettmers/openassistant-guanaco"
	dataset = load_dataset(dataset_name, split="train")

	model_name = "facebook/opt-350m"