- EfficientZero V2: A general sample-efficient RL framework excels in diverse control tasks (discrete/continuous, visual/low-dimensional) outperforming SoTA, including DreamerV3, in 50/66 benchmarks.
- Gambling-Based Confidence Sequences: A novel gambling framework constructs tight, non-asymptotic confidence sequences for bounded random vectors, including categorical and probability-vector-valued observations, outperforming existing methods like the posterior-prior ratio martingale.
- In-Context Learning Circuits: Mechanistic study reveals how induction heads, key to in-context learning in transformers, emerge through interactions of three identified sub-circuits during training.
- Explaining Probabilistic Models with Distributional Values: This paper introduces distributional values, generalising cooperative game theory and value operators to provide fine-grained explanations of probabilistic models like vision and language models by tracking changes in model output rather than s
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def universal_summary(x, depth=0, max_depth=4, max_items=5, max_chars=350): | |
| """ | |
| Provides a concise summary of any Python object. | |
| Args: | |
| x: Any Python object | |
| depth: Current recursion depth (internal use) | |
| max_depth: Maximum recursion depth | |
| max_items: Maximum number of items to show for collections | |
| max_chars: Maximum characters in the final output |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import torch | |
| from torch.utils.cpp_extension import load_inline | |
| # Define C++ source code | |
| cpp_source = """ | |
| #include <torch/extension.h> | |
| torch::Tensor add_one(torch::Tensor input) { | |
| return input + 1; | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * my-validator.test.ts | |
| * | |
| * Test suite for my-validator.ts library | |
| */ | |
| import { objSchema, arrSchema, unionSchema, validate, Infer } from './my-validator' | |
| // Test 1: Basic object schema validation | |
| function testBasicObject() { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import mpmath as mp | |
| import math | |
| # Set high precision | |
| mp.mp.dps = 50 | |
| def calculate_collision_probability(n, d): | |
| """ | |
| Calculate probability of at least one collision when choosing n items from d possibilities | |
| Using the birthday paradox/collision probability formula: 1 - exp(-n(n-1)/(2d)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** Run different types of jobs via postgres queue. | |
| * Aims for exactly-once completion of (job, key) pair. | |
| */ | |
| import 'dotenv/config' | |
| import { Pool, PoolClient } from 'pg' | |
| import { JobStatus } from 'shared' | |
| type StatusObj<TData = any, TResult = any> = { | |
| jobId: string | |
| jobType: string |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curl https://rclone.org/install.sh | sudo bash | |
| mkdir ~/.config/rclone | |
| echo " | |
| [r2] | |
| type = s3 | |
| provider = Cloudflare | |
| access_key_id = $a | |
| secret_access_key = $b | |
| region = auto | |
| endpoint = https://$c.r2.cloudflarestorage.com |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Why does this run instantly after first trial? | |
| import subprocess | |
| import time | |
| import shutil | |
| import os | |
| def run_rclone_command(command, num_trials=3, dest_path_base="/home/ubuntu/hff/405b-base"): | |
| """ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # modal run issue.py | |
| import os | |
| import subprocess | |
| import time | |
| import modal | |
| import modal.experimental | |
| import modal.gpu | |
| import modal.volume |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| pip install huggingface_hub[hf_transfer] | |
| export HF_HUB_ENABLE_HF_TRANSFER=1 | |
| model_name=meta-llama/Llama-3.1-405B | |
| localdir=$(realpath ~/hff/405b) | |
| huggingface-cli download --max-workers=8 --include="model-???[02468][02468]-of-?????.safetensors" --local-dir=$localdir $model_name & sleep 6 | |
| huggingface-cli download --max-workers=8 --include="model-???[02468][13579]-of-?????.safetensors" --local-dir=$localdir $model_name & sleep 6 | |
| huggingface-cli download --max-workers=8 --include="model-???[13579][02468]-of-?????.safetensors" --local-dir=$localdir $model_name & sleep 6 | |
| huggingface-cli download --max-workers=8 --include="model-???[13579][13579]-of-?????.safetensors" --local-dir=$localdir $model_name & sleep 0 |