Skip to content

Instantly share code, notes, and snippets.

@jcohen66

jcohen66/ai_local_builds.md

Created June 1, 2026 00:24
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jcohen66/63e9c97cf352c7c77e7833ebf22073b4 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jcohen66/63e9c97cf352c7c77e7833ebf22073b4 to your computer and use it in GitHub Desktop.
Download ZIP
AI Local Builds #ai #hardware #vram #gpu #local #build #counter #
Raw
ai_local_builds.md

https://www.youtube.com/watch?v=P-Fmo_CCIbY

Starter

  • 7B-8B models
  • RTX 4060 Ti with 16g of counter space.
  • Not the 8g version
    • Load a real model, start a conversation, dishes pile up instantly
  • Ryzen 5 cpu
  • 64g RAM
  • 2T SSD
  • Good airflow and power

Mac

  • 7B-8B models
  • 16g Unified memory
  • Counter and storage are one big space
  • Bit slower than dedicated NVIDA graphics card

Coding Agents

Path 1

  • RTX 4070 Ti Super
  • 16g of counter space
  • Faster hands than 4060 Ti
  • Better for agent loops
  • Think, Execute, Think again

Path 2

  • 32B models with room left for conversation
  • Used RTX 3090
  • 24g of VRAM
  • Ryzen 7
  • 64GB RAM
  • 2T SSD

Tier 3 Powerhouse

  • 32B models like butter
  • RTX 4090
  • 24G of counter space
  • 128GB RAM
  • Massive storage
  • Fast chef
  • Big counterr
  • Long conversations
  • Complex agent chains

Mac

  • Mac Studio or Mac Ultra
  • 96G unified memory/counter space
  • Draws under 100 watts

Model Formats

Mac

GGUF

Not Mac

AWQ

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment