Skip to content

Instantly share code, notes, and snippets.

@justinlevi

justinlevi/AI Infrastructure and Tools.md

Last active December 12, 2024 13:39
Show Gist options
  • Save justinlevi/a8fe6d7631eb31832bcb6c8499bb18ff to your computer and use it in GitHub Desktop.
Save justinlevi/a8fe6d7631eb31832bcb6c8499bb18ff to your computer and use it in GitHub Desktop.
Download ZIP
Raw
AI Infrastructure and Tools.md

AI Infrastructure and Tools Overview

Vector Database Storage Frameworks/Services

A specialized storage system designed to efficiently handle and query high-dimensional vector data, enabling similarity search and AI applications

  • Qdrant - Open-source vector database written in Rust offering high-performance similarity search with cloud-native scalability

  • Pinecone - Serverless vector database platform optimized for machine learning applications with enterprise-grade security

  • Weaviate - Fast, flexible AI-native vector database with built-in model serving and multi-tenant capabilities

  • PGVector - PostgreSQL extension that enables vector similarity search with ACID compliance and SQL integration

  • Supabase - Postgres-based platform that includes vector search capabilities alongside other database features

LLM Serving Frameworks/Tools

Frameworks and platforms that enable efficient deployment and serving of large language models, optimizing for performance and scalability in production environments

  • vLLM - High-performance inference engine using PagedAttention for optimal serving throughput

  • Ollama - Framework for running and serving large language models locally with cross-platform support

  • LM Studio - Local LLM development environment with built-in model management and serving capabilities

  • Groq - Cloud platform offering ultra-fast LLM inference with specialized hardware acceleration

  • MLX - Apple's machine learning framework optimized for Apple Silicon

  • KServe - Kubernetes-based model serving platform supporting multiple frameworks and auto-scaling

Agent Development Frameworks

Software infrastructures that support the creation and management of autonomous agents, providing tools for development, deployment, and interaction between AI agents

  • LangChain - Comprehensive framework for building and connecting LLM-powered applications

  • LangGraph - Framework specialized in creating stateful, multi-agent workflows

  • LlamaIndex - Framework for building RAG applications with data connection capabilities

  • AutoGen/Magnetic-One - Microsoft's framework enabling sophisticated multi-agent collaboration patterns

  • CrewAI - Platform for creating role-based AI agent teams with specialized tasks and collaboration

MLOps Pipeline Tools

Software components that enable continuous integration, delivery, and automation of machine learning workflows, including model training, testing, and deployment

  • Apache Kafka - Distributed streaming platform for building real-time data pipelines

  • Google Cloud MLOps - Comprehensive suite of tools for ML model deployment and management

  • Databricks MLflow - End-to-end platform for managing the ML lifecycle with experiment tracking

  • Kubeflow - Kubernetes-native platform for deploying ML workflows

  • Delta Lake - Storage layer that brings ACID transactions to data lakes

Monitoring and Observability

Platforms and solutions that provide visibility into AI system performance, helping detect issues, analyze behavior, and ensure reliability of AI deployments

  • Arize Phoenix - Open-source platform specifically designed for LLM observability and evaluation

  • Evidently - Tool for ML model monitoring and evaluation in production

  • Seldon - Enterprise platform for deploying and monitoring ML models at scale

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment