Vaibhav Tulsyan xennygrimmato

Using Linear Algebra to Convert a Large Code Model

Background

The SalesForce CodeGen models are a family of large language models trained on a large amount of natural language data and then fine-tuned on specialized datasets of code. Models of size 350M, 2B, 6B, and 16B parameters are provided in three flavors:

nl, the base model trained on The Pile, a large natural language dataset compiled by EleutherAI
multi, which is fine-tuned from the nl model on a dataset of code in multiple languages, scraped from GitHub, and
mono, which is fine-tuned from the multi model on Python code only.

	import time
	# https://github.com/niklasb/ctf-tools/blob/master/pwnlib/tools.py
	from pwnlib.tools import *

	TARGET=('localhost',5000)
	INTERVAL=0

	offset_free = 549184
	offset_system = 0x456d0

	'''
	CRC is affine.
	CRC(x) = L(x) + C, where L is linear.
	We want CRC(x) = L(x) + C = x.
	Write as L(x)+x = C.
	Solve matrix equation.
	'''

	from sage.all import *

	#include <iostream>
	#include <stdlib.h>
	using namespace std;

	#define REP(i,x) for(int i = 0; i < (int)x; i++)
	#define M 8

	int N;
	string s[1000];
	long q[M], p[M], hs[M][1000], hr[M][1000];