0xJohnnyGault

#!/usr/bin/env bash
set -Eeuo pipefail

# Experiment to see how using foundry for deploys would work

if [ "${BASH_VERSINFO:-0}" -lt 4 ]; then
 echo "script requires bash version >= 4"
 exit 1
fi

chand1012

// LICENSE: https://chand1012.mit-license.org

const fetchWithTimeout = (url: string, options: RequestInit, timeout: number = 10000): Promise<any> => {
  return new Promise(async (resolve, reject) => {
    const controller = new AbortController();
    const signal = controller.signal;
    const timer = setTimeout(() => {
      controller.abort();
      reject(new Error("Request timed out"));
    }, timeout);

pdtgct

The LLaMA model weights may be converted from Huggingface PyTorch format back to GGML in two steps:

1. download from decapoda-research/llama-7b-hf and save as pytorch .pth
2. use the ggerganov/llama.cpp script, convert-pth-to-ggml.py to convert from pytorch .pth to GGML

This process will result in ggml model with float16 (fp16) precision.

Prerequisite