Skip to content

Instantly share code, notes, and snippets.

View vgoklani's full-sized avatar

Vishal Goklani vgoklani

View GitHub Profile
@Birch-san
Birch-san / llama_flash.py
Last active January 22, 2024 06:05
Loading llama with Flash Attention
from transformers import (
AutoConfig,
AutoTokenizer,
BitsAndBytesConfig,
GenerationConfig,
AutoModelForCausalLM,
LlamaTokenizerFast,
PreTrainedModel,
TextIteratorStreamer,
StoppingCriteria,
@mara004
mara004 / pypdfjs.py
Last active July 1, 2026 16:07
PDF rendering with pdf.js, from Python
# Four lines intentionally left blank
# SPDX-FileCopyrightText: 2025 geisserml <geisserml@gmail.com>
# SPDX-License-Identifier: Apache-2.0 OR MPL-2.0
# See also https://github.com/extremeheat/JSPyBridge/blob/master/examples/python/pdfjs.py
@codekansas
codekansas / benchmark_self_attention.py
Last active March 11, 2023 18:34
Benchmarking script for attention
import argparse
import contextlib
import logging
import math
import random
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Callable
@Chillee
Chillee / 1-pw_op_fusion.py
Last active November 4, 2025 08:34
PT 2.0 Benchmarks
import torch
import torch._inductor.config
import time
torch._inductor.config.triton.cudagraphs = False
torch.set_float32_matmul_precision('high')
def bench(f, name=None, iters=100, warmup=5, display=True, profile=False):
for _ in range(warmup):
f()
Work Details
Augmenting convnets with aggregated attention Tutorial by Aritra
Train a Vision Transformer on small datasets Tutorial by Aritra
MobileViT Tutorial by Sayak
Compact Convolutional Transformers Tutorial by Sayak
Data efficient image transformers TF implementation, TF pre-trained models, tutorial by Sayak
Class attention image transformers TF implementation, TF pre-trained models by Sayak
Masked Autoencoders TF implementation, tutorial by Aritra and Sayak,
Contribution to Hugging Face Transformers by Aritra and Sayak
Probing the representation of ViTs
@simonster
simonster / attention_distance.py
Last active May 29, 2025 12:37
Mean attention distance
# Copyright 2022 Google LLC.
# SPDX-License-Identifier: Apache-2.0
# Author: Maithra Raghu <maithra@google.com>
def compute_distance_matrix(patch_size, num_patches, length):
"""Helper function to compute distance matrix."""
distance_matrix = np.zeros((num_patches, num_patches))
@madlag
madlag / speed2.py
Created September 30, 2020 17:35
Pytorch CUDA speed test for various data types, with and without AMP
#!/usr/bin/env python
# Any copyright is dedicated to the Public Domain.
# https://creativecommons.org/publicdomain/zero/1.0/
# Written by Francois Fleuret <francois@fleuret.org>
# Modified by François Lagunas <francois.lagunas@m4x.org>
import time, torch
@jstheoriginal
jstheoriginal / SearchBar.swift
Created June 20, 2019 00:42
A simple SwiftUI search bar
struct SearchBar : View {
@Binding var searchText: String
var body: some View {
HStack {
Image(systemName: "magnifyingglass").foregroundColor(.secondary)
TextField(
$searchText,
placeholder: Text("Search")) {
UIApplication.shared.keyWindow?.endEditing(true)
@dannguyen
dannguyen / _house-public-disc-scraper-README.md
Last active March 29, 2026 20:25
Simple scraper of the ASPX search form for U.S. Congress House financial disclosure results

Simple scraper of the ASPX search form for U.S. Congress House financial disclosure results

The following script, given someone's last name, prints a CSV of financial disclosure PDFs (the first 20, for simplicity's sake) as found on the House Financial Disclosure Reports. It's meant to be a proof-of-concept of how to scrape ASPX (and other "stateful" websites) with using plain old requests -- without too much inconvenience -- rather than resorting to something heavy like the selenium websdriver

The search page can be found here: http://clerk.house.gov/public_disc/financial-search.aspx

Here's a screenshot of what it does when you search via web browser:

screenshot of disclosure search for "king"

import numpy as np
from numba import jit
from numba import float64
from numba import int64
@jit((float64[:], int64), nopython=True, nogil=True)
def _ewma(arr_in, window):
r"""Exponentialy weighted moving average specified by a decay ``window``
to provide better adjustments for small windows via: