brian piercy bjpcjp

Raschka, 12 notable 2024 papers

Mixture of Experts (MoE), Mixtral: https://arxiv.org/abs/2401.04088
DoRA (weight decomposed low-rank adaptation): https://arxiv.org/abs/2402.09353
Continual pretraining: https://arxiv.org/abs/2403.08763
DPO (direct preference optimization) vs PPO (proximal policy optimization): https://arxiv.org/abs/2404.10719
LoRA learns less, forgets less: https://arxiv.org/abs/2405.096730
FineWeb (15T tokens!) dataset: https://arxiv.org/abs/2406.17557
Llama3 herd of models: https://arxiv.org/abs/2407.217830
LLM scaling (test time compute vs model params): https://arxiv.org/abs/2408.03314

Group	Benchmark	Summary Explanation	Link
English	MMLU (EM)	Measures multi-task learning across diverse knowledge domains to evaluate language models' general academic proficiency.	MMLU Benchmark
English	MMLU-Redux (EM)	A reduced version of MMLU focusing on key topics or subsets of academic questions.	No dedicated link available.

The compgen command in Linux is a shell built-in used primarily for generating possible completions for commands, functions, files, or other shell elements. It is part of the Bash shell and is often utilized in scripts for tab-completion or command suggestion mechanisms.

Syntax:

compgen [option] [word]

	# https://www.kdnuggets.com/10-python-one-liners-change-coding-game

	# 1. Lambda Functions
	price_after_discount = lambda price: price*0.9


	# 2. Map Operations on Lists
	discounted_prices = list(map(price_after_discount, prices))

	<!DOCTYPE html>
	<meta charset="utf-8">
	<style>
	body {
	font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;
	margin: auto;
	padding-top: 40px;
	position: relative;
	width: 800px;
	}

	#!/usr/bin/env bash

	# source: https://sharats.me/posts/shell-script-best-practices/

	set -o errexit
	set -o nounset
	set -o pipefail
	if [[ "${TRACE-0}" == "1" ]]; then
	set -o xtrace
	fi

	from blankly import Alpaca, CoinbasePro # supports stocks, crypto, and forex
	import numpy as np
	from math import sqrt


	def cagr(start_value: float, end_value: float, years: int):
	return (end_value / start_value) ** (1.0 / years) - 1

	def sharpe(account_values: np.array, risk_free_rate, annualize_coefficient):
	diff = np.diff(account_values, 1) / account_values[1:] # this gets our pct_return in the array

	# Author: Tom Dupre la Tour
	#
	# License: BSD 3 clause
	import time
	import sys

	try:
	import annoy
	except ImportError:
	print("The package 'annoy' is required to run this example.")

	<svg width="10" height="10">
	<rect x="0" y="0" width="10" height="10" fill="blue" />
	</svg>

brian piercy bjpcjp

Raschka, 12 notable 2024 papers

Syntax:

Key Features: