Adrian Carballo adriancbo

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

	from IPython import display as ipd
	from audiocraft.models import musicgen
	from audiocraft.utils.notebook import display_audio
	import torch
	import torchaudio
	import os

	# Define the output directory
	output_dir = r'C:\Users\USER\OneDrive\Desktop\musicgen2'

	conda create --name myenv python=3.10.11
	conda activate myenv
	conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

	pip install xformers
	pip install audiocraft
	pip install IPython