moyix

Using Linear Algebra to Convert a Large Code Model

Background

The SalesForce CodeGen models are a family of large language models trained on a large amount of natural language data and then fine-tuned on specialized datasets of code. Models of size 350M, 2B, 6B, and 16B parameters are provided in three flavors:

• nl, the base model trained on The Pile, a large natural language dataset compiled by EleutherAI
• multi, which is fine-tuned from the nl model on a dataset of code in multiple languages, scraped from GitHub, and
• mono, which is fine-tuned from the multi model on Python code only.

moyix

#!/usr/bin/env python

import argparse
import torch
from transformers import GPTJForCausalLM, GPTJConfig
# Note: these need the git version of Transformers as of 7/22/2022
from transformers import CodeGenTokenizer, CodeGenForCausalLM
from transformers import CODEGEN_PRETRAINED_MODEL_ARCHIVE_LIST

parser = argparse.ArgumentParser('Convert SalesForce CodeGen model to GPT-J')

ih2502mk

Lecture 1: Introduction to Research — [📝Lecture Notebooks] [▶️Video]
Lecture 2: Introduction to Python — [📝Lecture Notebooks] [▶️Video]
Lecture 3: Introduction to NumPy — [📝Lecture Notebooks] [▶️Video]
Lecture 4: Introduction to pandas — [📝Lecture Notebooks] [▶️Video]
Lecture 5: Plotting Data — [📝Lecture Notebooks] [[▶️Vide

endolith

"""
Train a neural network to implement the discrete Fourier transform
"""
import matplotlib.pyplot as plt
import numpy as np
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential

N = 32
batch = 10000

hfreire

# create a vault
aws glacier create-vault --account-id - --vault-name my-vault-name

# check that the vault was created successfully
aws glacier describe-vault --account-id - --vault-name my-vault-name

# upload a file to the vault
aws glacier upload-archive --vault-name my-vault-name --account-id - --archive-description "my-archive-description" --body my-text-file.txt

# request an inventory of the vault

aumouvantsillage

sudo apt install git cmake clang-3.9 opencl-headers ocl-icd-dev ocl-icd-opencl-dev

sudo ln -s /usr/bin/clang{-3.9,}

git clone https://github.com/doe300/VC4C.git
git clone https://github.com/doe300/VC4CL.git
git clone https://github.com/doe300/VC4CLStdLib.git

mkdir VC4C/build
cd VC4C/build

goldsborough

#include <cudnn.h>
#include <cassert>
#include <cstdlib>
#include <iostream>
#include <opencv2/opencv.hpp>

#define checkCUDNN(expression)                               \
  {                                                          \
    cudnnStatus_t status = (expression);                     \
    if (status != CUDNN_STATUS_SUCCESS) {                    \

moodoki

import os, argparse

import tensorflow as tf
from tensorflow.python.framework import graph_util

dir = os.path.dirname(os.path.realpath(__file__))

def freeze_graph(model_folder, output_nodes='y_hat', 
                 output_filename='frozen-graph.pb', 
                 rename_outputs=None):

bnb

/*
Usage: 
  Adding this script to your doc: 
    - Tools > Script Manager > New
    - Select "Blank Project", then paste this code in and save.
  Running the script:
    - Tools > Script Manager
    - Select "ConvertToMarkdown" function.
    - Click Run button.
    - Converted doc will be mailed to you. Subject will be "[MARKDOWN_MAKER]...".

robertpainsi

Commit Message Guidelines

Short (72 chars or less) summary

More detailed explanatory text. Wrap it to 72 characters. The blank
line separating the summary from the body is critical (unless you omit
the body entirely).

Write your commit message in the imperative: "Fix bug" and not "Fixed
bug" or "Fixes bug." This convention matches up with commit messages