Andrew Luetgers andrewluetgers

The MIT License (MIT)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

High-Performance Matrix Multiplication

This is a short post that explains how to write a high-performance matrix multiplication program on modern processors. In this tutorial I will use a single core of the Skylake-client CPU with AVX2, but the principles in this post also apply to other processors with different instruction sets (such as AVX512).

Intro

Matrix multiplication is a mathematical operation that defines the product of

	# train_grpo.py
	#
	# See https://github.com/willccbb/verifiers for ongoing developments
	#
	import re
	import torch
	from datasets import load_dataset, Dataset
	from transformers import AutoTokenizer, AutoModelForCausalLM
	from peft import LoraConfig
	from trl import GRPOConfig, GRPOTrainer

	import boto3
	import apsw

	BLOCK_SIZE = 64 * 1024
	EMPTY_BLOCK = b"".join([b"\x00"] * BLOCK_SIZE)

	# Inheriting from a base of "" means the default vfs
	class S3VFS(apsw.VFS):
	def __init__(self, s3, bucket, vfsname=f"s3vfs", basevfs=""):
	self.vfsname = vfsname

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="utf-8" />
	<meta name="viewport" content="width=device-width, initial-scale=1" />
	<title>scratch</title>
	<style>
	body {
	font-family: Hack, Menlo, Monaco, 'Droid Sans Mono', 'Courier New', monospace;
	white-space: pre;

	import React, { useReducer, useEffect, useRef } from 'react';
	import firebase from 'firebase/app';
	import equal from 'deep-equal';

	function filterKeys(raw, allowed) {
	if (!raw) {
	return raw;
	}
	let s = new Set(allowed);
	return Object.keys(raw)

	/** @license React v16.2.0
	* react-dom.development.js
	*
	* Copyright (c) 2013-present, Facebook, Inc.
	*
	* This source code is licensed under the MIT license found in the
	* LICENSE file in the root directory of this source tree.
	*/

	'use strict';