vikhyat

Photon vs vLLM on NVIDIA B200 (final shipping board)

Status: Final 28-cell board against vLLM 0.27.1. The board retains 26 validated, unchanged cells from 7f0dddc2, retains the frozen Gemma E4B C8 measurement from e1d6c1dd, and replaces Gemma E4B C4 with the final fast/correct release measurement from 5c06c090. Every Photon/vLLM pair is internally matched on driver, model revision, harness, ordered request stream, request count, and B200 hardware class; exact engine commits are recorded per cell.

ChartQA multimodal inference with reasoning enabled on NVIDIA B200. The primary metric is natural-output tokens completed per active end-to-end service second; accuracy and output-length differences are reported alongside it.

At a glance

• Photon leads in 28/28 end-to-end output-token service-throughput cells.
• Photon leads in 28/28 completed-request-rate cells.

vikhyat

Non-Neural Drafting Baseline for Qwen Math Traces

This gist contains a small reproducibility bundle for measuring a deterministic draft program on qwen35 math reasoning traces.

The measurement is an offline speculative-decoding analogue: at each response position, a causal program sees only the prompt plus already accepted response prefix, drafts 16 tokens, and counts how many leading tokens match the held-out trace continuation.

vikhyat

from typing import List, Dict, Tuple

import torch
import torch.nn as nn

from einops import rearrange
from transformers import pipeline, Owlv2Processor, Owlv2ForObjectDetection
from transformers.image_transforms import center_to_corners_format
from flash_attn.modules.mha import FlashSelfAttention

vikhyat

import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation

act, filename = nn.GELU, 'gelu_training.gif'

# Step 1: Generate Data

vikhyat

# Copyright (c) Microsoft Corporation.
# Licensed under the MIT license.
#
# Copyright (c) 2022, Tri Dao, trid@cs.stanford.edu.
# Licensed under the BSD 3-Clause License.

from __future__ import annotations

import math
from dataclasses import dataclass, field

vikhyat

tok_embeddings.weight torch.Size([32000, 4096])
norm.weight torch.Size([4096])
output.weight torch.Size([32000, 4096])
layers.0.attention_norm.weight torch.Size([4096])
layers.0.attention.wq.weight torch.Size([4096, 4096])
layers.0.attention.wk.weight torch.Size([1024, 4096])
layers.0.attention.wv.weight torch.Size([1024, 4096])
layers.0.attention.wo.weight torch.Size([4096, 4096])
layers.0.feed_forward.gate.weight torch.Size([8, 4096])
layers.0.ffn_norm.weight torch.Size([4096])

vikhyat

test

vikhyat

Here's how it's going to look like from the end user perspective:

1 hour before my PRO membership ends, if I am on a recurring plan 
Hummingbird will try to charge my credit card. If it succeeds 
everything is good. If it fails my subscription will run out and 
Hummingbird will try to charge my credit card again tomorrow, 3 
days from now and 5 days from now. I will receive an email from 
Hummingbird if the charge is successful, and a different email 
every time charging my card fails.

vikhyat

function readAnime(results) {
  return _.map(results.anime, function(anime) {
    anime.linked = {};
    _.each(Object.keys(anime.links), function(rel) {
      var relVal;
      if (Array.isArray(anime.links[rel])) {
        relVal = _.find(results.linked[rel], function(candidate) {
          // return whether candidate.id is in anime.links[rel]
        });
      } else {

vikhyat

int fib(n) {
  return n < 2 ? n : fib(n-1) + fib(n-2);
}

int main() {
  int t, n;
  scanf("%d\n", &t);
  while (t--) {
    scanf("%d\n", &n);
    printf("%f\n", fib(n+2) / pow(2, n));