Skip to content

Instantly share code, notes, and snippets.

View jeffmcjunkin's full-sized avatar

Jeff McJunkin jeffmcjunkin

200 followers · 14 following
View GitHub Profile
@timinar
timinar / GLM-5.2-on-8xA100-vLLM.md
Last active August 21, 2026 23:10
Running GLM-5.2 (753B DeepSeek-Sparse-Attention MoE) on 8x A100 80GB with vLLM — TRITON_MLA_SPARSE backend (PR #38476), no-recompile install, benchmarks

Running GLM-5.2 (753B DSA MoE) on 8× A100 80GB with vLLM

TL;DR. GLM-5.2 (glm_moe_dsa — DeepSeek Sparse Attention) does not run on Ampere (A100, sm_80) with stock vLLM: the sparse-MLA attention backend (FLASHMLA_SPARSE) and the lightning-indexer's fp8_mqa_logits (DeepGEMM) are Hopper/Blackwell-only. vLLM PR #38476 (issue #38006) adds a Triton sparse-MLA backend (TRITON_MLA_SPARSE) + a bf16 Triton indexer fallback that run on Ampere. Cherry-picking it onto current main is a Python-only change — no CUDA recompile. Result: GLM-5.2 AWQ-INT4 serves on 8× A100 at ~56 tok/s single-stream and ~625 tok/s aggregate decode (32-way), with coherent output.

This is an independent 8× A100 confirmation of PR #38476 (the author validated on 32× A100), plus a no-recompile install note. Credit to @haosdent for the PR.

Requirements

  • 8× A100 80GB (sm_80). ~410 GiB VRAM used at TP=8, so all 8 GPUs.
@joswr1ght
joswr1ght / irplaybook.txt
Created November 10, 2025 19:16
AI Prompt to Generate Incident Response Playbooks
# Role and Objective/Task
You are an expert-level cybersecurity incident response analyst. Your task is to leverage best practice guidance to assist users in developing incident response playbooks that guide users through complex analysis tasks following an observed Event of Interest (EOI).
# Instructions
Assist the user in developing an incident response playbook for the supplied EOI.
Ask questions when the answer is needed to create a high-quality playbook. These questions could include information about IT infrastructure and systems, existing defense mechanisms, existing organizational policies, and organizational information. If the user provides insufficient detail, ask targeted, technical follow-up questions to clarify the EOI, affected platforms, and org environment before proceeding.
@rahimnathwani
rahimnathwani / write_a_story.py
Created April 12, 2025 01:05
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.11"
# dependencies = [
# "python-dotenv",
# "openai",
# ]
# ///
import os
@jlia0
jlia0 / agent loop
Last active September 2, 2026 21:01
Manus tools and prompts
You are Manus, an AI agent created by the Manus team.
You excel at the following tasks:
1. Information gathering, fact-checking, and documentation
2. Data processing, analysis, and visualization
3. Writing multi-chapter articles and in-depth research reports
4. Creating websites, applications, and tools
5. Using programming to solve various problems beyond development
6. Various tasks that can be accomplished using computers and the internet
@chriselgee
chriselgee / ClickhouseForCreds.sh
Last active February 27, 2025 04:08
### This assumes you have a credential file called BreachData.txt in the format of email:password, one per line
# Setup and start Clickhouse
mkdir clickhouse && cd clickhouse
curl https://clickhouse.com/ | sh
./clickhouse server
# IN A NEW TERMINAL
# Create and populate the database (assumes the cred file is one level up)
./clickhouse client 'CREATE DATABASE creds'
./clickhouse client 'CREATE TABLE creds.logins(`email` String,`password` String) ENGINE = MergeTree ORDER BY email'
@startergo
startergo / KVM_in_WSL2.md
Last active September 3, 2026 05:53
KVM on WSL2 Windows 11
  • In WSL2 run:
sudo apt update
sudo apt install qemu qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils cpu-checker \
network-manager iptables-persistent linux-headers-generic \
qemu uml-utilities virt-manager git \
wget libguestfs-tools p7zip-full make dmg2img tesseract-ocr \
tesseract-ocr-eng genisoimage vim net-tools screen firewalld libncurses-dev -y
sudo apt install virt-manager
sudo addgroup kvm
@ZephrFish
ZephrFish / 14_RTX_4090_v6.2.6.Benchmark.txt
Last active May 20, 2025 20:18
Hashcat v6.2.6 benchmark on 14x Nvidia RTX 4090
Benchmark is from an AI Cloud Rig:
https://cloud.vast.ai/?ref_id=127244
hashcat (v6.2.6) starting in benchmark mode
Benchmarking uses hand-optimized kernel code by default.
You can use it in your cracking session by setting the -O option.
Note: Using optimized kernel code limits the maximum supported password length.
To disable the optimized kernel code in benchmark mode, use the -w option.
@nullenc0de
nullenc0de / block_sec.sh
Created February 16, 2024 16:49
Looks up IP addresses to companies and blocks them via IP tables.
#!/bin/bash
apt install golang -y
GOROOT="/usr/local/go"
PATH="${PATH}:${GOROOT}/bin"
GOPATH=$HOME/go
PATH="${PATH}:${GOROOT}/bin:${GOPATH}/bin"
go install github.com/projectdiscovery/asnmap/cmd/asnmap@latest
@Scoubi
Scoubi / Linux.md
Last active February 7, 2024 18:34
Cross Compile Rust Banaries from Mac OSX (M1)
  1. brew install SergioBenitez/osxct/x86_64-unknown-linux-gnu
  2. brew install x86_64-linux-gnu-binutils
  3. rustup target add x86_64-unknown-linux-gnu
  4. Edit ~/.cargo/config and add
[target.x86_64-unknown-linux-gnu]
linker = "x86_64-unknown-linux-gnu-gcc"
  1. source ~/.cargo/config
  2. Edit Cargo.toml and include OpenSSL crate dependency This is the part missing from many how-to
@testanull
testanull / SharePwn_public.py
Created December 15, 2023 07:31
SharePoint Pre-Auth Code Injection RCE chain CVE-2023-29357 & CVE-2023-24955 PoC
# -*- coding: utf-8 -*-
import hashlib
import base64
import requests, string, struct, uuid, random, re
import sys
from collections import OrderedDict
from sys import version
from urllib3.exceptions import InsecureRequestWarning
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)
# too lazy to deal with string <-> bytes confusion in python3 so forget it ¯\_(ツ)_/¯