hajowieland · September 8, 2023 23:14
diff --git a/llama2-70b-chat.yaml b/llama2-70b-chat.yaml
 backend: llama
 context_size: 4096
 f16: true
 gpu_layers: 43
 ngqa: 8
 mmap: true
 name: llama2-13b-chat
 parameters:
  model: llama-2-70b-chat.Q4_0.gguf
  temperature: 0.2
  top_k: 80
  top_p: 0.7
 system_prompt: |
  You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe.  Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
  If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
 template:
  chat_message: llama2-chat-message
	backend: llama
	context_size: 4096
	f16: true
	gpu_layers: 43
	ngqa: 8
	mmap: true
	name: llama2-13b-chat
	parameters:
	model: llama-2-70b-chat.Q4_0.gguf
	temperature: 0.2
	top_k: 80
	top_p: 0.7
	system_prompt: \|
	You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
	If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
	template:
	chat_message: llama2-chat-message