Skip to content

Instantly share code, notes, and snippets.

@soasme
Last active May 9, 2026 04:31
Show Gist options
  • Select an option

  • Save soasme/3e359bbe0d644fde116f5c8df601453a to your computer and use it in GitHub Desktop.

Select an option

Save soasme/3e359bbe0d644fde116f5c8df601453a to your computer and use it in GitHub Desktop.

AI Engineering: A Complete Overview of LLMs, RAG, MCP, Agents, Fine-Tuning, and Quantization

Artificial Intelligence has rapidly evolved from a niche research field into one of the most transformative technologies in software engineering. Today, developers across Android, backend, web, DevOps, and cloud engineering are increasingly expected to understand AI systems and how to integrate them into real-world products.

This article walks through the core concepts behind modern AI engineering — including Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), MCP (Model Context Protocol), tools, embeddings, and the architecture of AI-powered systems.


Machine Learning vs AI Engineering

Before discussing modern AI systems, it is important to understand the distinction between Machine Learning Engineering and AI Engineering.

These two areas are related, but they solve very different problems.


Machine Learning Engineering: Building the Model

Machine learning engineering focuses on creating and training models.

A machine learning engineer typically:

  • Writes training code
  • Designs algorithms
  • Works with mathematics and optimization
  • Trains models on datasets
  • Produces model weights after training

At a high level, the process looks like this:

  1. Create a model.py
  2. Feed large amounts of training data
  3. Run training
  4. Generate trained parameters (weights)

Those weights are then stored in a file such as parameters.bin.

This process involves deep learning, transformer architectures, optimization algorithms, GPU training, and mathematical modeling. The result is a trained model.


AI Engineering: Using the Model

AI engineering is different.

AI engineers generally do not train foundation models from scratch.

Instead, they:

  • Use existing models
  • Integrate models into products
  • Build AI applications
  • Create RAG systems
  • Add tools and external APIs
  • Fine-tune and quantize models
  • Design scalable AI architectures

AI engineering is fundamentally about:

Making AI usable in real products.

An Android developer, backend engineer, DevOps engineer, or web developer can all become AI engineers because AI engineering focuses on integrating and operationalizing models rather than inventing them from scratch.


Building an AI Tutor

To understand the ecosystem, imagine building an AI Tutor application.

The interface might resemble ChatGPT:

  • A chat box
  • File upload support
  • A conversational UI
  • PDF attachments
  • Question-answering functionality

This AI tutor could run on Android, iOS, or the web. The frontend communicates with a backend server through APIs, and the backend communicates with AI systems and external services.

This becomes the foundation for understanding the rest of the architecture.


The World Before LLMs

Before large language models became mainstream, search systems worked very differently.

If a user searched L1 and L2 loss function, the system would typically return blue links, blog articles, and search engine results. The architecture was simple:

UI → Backend → Google API

The backend simply fetched search results and returned them to the frontend. The user had to open pages, read articles, and extract meaning manually.

This was the traditional web search experience.


The World After LLMs

Modern AI systems changed this completely.

Instead of returning links, systems now return summaries, explanations, structured answers, and contextual responses. If a user asks Explain L1 and L2 loss functions, the system directly generates definitions, comparisons, and examples.

This shift is powered by Large Language Models.


What Is a Language Model?

A language model is a system that understands text, learns language structure, predicts text, and generates text. It learns words, sentences, grammar, patterns, and relationships between tokens.

In essence:

A language model understands language and generates language.


Why Is It Called a Large Language Model?

Two things transformed ordinary language models into Large Language Models (LLMs):

1. Massive Training Datasets

Modern LLMs are trained on enormous datasets — internet data, books, articles, documentation, websites, and public conversations. The scale is dramatically larger than older systems.

2. Massive Numbers of Parameters

Training produces a huge set of learned numerical weights called parameters. These parameters are stored in files such as parameters.bin and may number in the millions, billions, or hundreds of billions.

Because both the dataset and the parameter count are massive, the system becomes a Large Language Model.


Understanding Model Weights

To understand model weights intuitively, consider a real estate pricing example.

Suppose we have historical housing data:

Bedrooms Square Feet Balcony Price
2 1200 1 500K
3 1500 2 700K

We can model pricing as:

Price = W1 × Bedrooms + W2 × Square Feet + W3 × Balcony

The values W1, W2, and W3 are learned during training. Initially:

W1 = 0
W2 = 0
W3 = 0

After training, the system discovers optimized values. Those learned values become the weights.

The exact same idea scales up inside LLMs — except instead of 3 weights, there may be hundreds of billions.


What Happens During Training?

At a high level:

  1. A model.py defines the architecture
  2. Data is fed into the model
  3. Training adjusts weights
  4. Optimized weights are saved to parameters.bin

This file contains the learned intelligence of the model. The architecture code itself is often relatively small. The real value is inside the trained parameters — which is why model weights are considered extremely valuable.


Why Open-Source Model Weights Matter

When companies release open-source models, they typically release model.py and parameters.bin. The parameters are the most valuable component, because retraining a frontier-scale model requires massive GPU clusters, tens or hundreds of millions of dollars, and enormous infrastructure.

If someone already provides trained weights, developers can immediately run advanced models without paying training costs.


Understanding Model Size

Suppose a model contains 100 billion parameters. Each parameter requires memory.

If parameters use 32-bit floating point precision (4 bytes each):

100B × 4 bytes = 400 GB

If we reduce to 16-bit precision (2 bytes each):

100B × 2 bytes = 200 GB

Reducing precision decreases memory usage and hardware requirements. This idea becomes extremely important later when discussing quantization.


LLMs Work Completely Offline

One of the most important concepts in AI engineering is this:

A Large Language Model works completely offline.

An LLM does not inherently browse the internet, make API calls, access databases, or search Google. An LLM is simply model.py and parameters.bin inside a directory. You can download an open-source model, disconnect your internet, and run the model locally — and it still works.

This is critical to understand because many people incorrectly assume LLMs themselves access the internet. They do not.


LLMs Are Like Trained Humans

An LLM is like a child who has already learned from books and the internet. After learning, the child is placed in a room with no internet access and can only answer based on prior knowledge.

The model understands language, predicts text, and generates responses — but cannot independently fetch new information.


Tokenization and Embeddings

Computers do not naturally understand text — they understand numbers. So language must be converted into numerical form through a process called tokenization.

For example:

Word Token
he 12
is 7
good 42

The sentence He is good becomes the token sequence 12 7 42.

These tokens are then transformed into embeddings — numerical vector representations that capture semantic meaning. The model processes these vectors mathematically, generates output vectors, and finally decodes them back into text.

This is how LLMs internally operate.


The Problem With Static Knowledge

Suppose a model was trained in 2020. If you ask it What is the current Bitcoin price?, the model cannot reliably answer because it was trained years ago and has no internet access. Possible outcomes include outdated answers, hallucinated values, or refusal responses.

This creates a major limitation.


Why Not Retrain Constantly?

Retraining is impractical because training is expensive, takes massive compute, and data changes constantly — real-time retraining is infeasible.

A system needs another solution: RAG.


What Is RAG?

RAG stands for Retrieval-Augmented Generation. It combines external information retrieval with LLM generation.

Instead of forcing the model to know everything, we:

  1. Retrieve fresh information externally
  2. Feed that information into the model
  3. Let the model generate a grounded response

This solves the real-time knowledge problem.


How RAG Works

Suppose the user asks What is the current Bitcoin price?

The architecture becomes:

Client → Backend → Crypto API
                       ↓
                      LLM

The backend detects the request, calls a crypto API, gets live data, and sends that data to the LLM. The LLM then generates a natural response such as:

The current Bitcoin price is 65K USD based on real-time data.

The LLM is not fetching the price itself. The backend retrieves the data; the LLM understands context, frames the response, and generates human-readable output.


Tools in AI Systems

External systems used by AI applications are often called tools — search APIs, crypto APIs, PDF readers, YouTube transcript extractors, and so on.

The LLM itself cannot use these directly. Instead, the backend invokes tools, gathers results, and passes those results into the LLM. This architecture powers modern AI systems.


The Scaling Problem

Initially, developers may implement hardcoded routing logic:

if crypto:
    call crypto API
if youtube:
    call YouTube API
if blogs:
    call Google API

But this becomes unmanageable. The backend slowly turns into a giant decision engine, full of branching logic that is difficult to maintain.

This motivates the need for MCP.


What Is MCP?

MCP stands for Model Context Protocol. It acts as a standardized communication layer for AI systems.

A useful analogy:

Traditional Web AI Systems
HTTP MCP

HTTP standardizes communication between machines. MCP standardizes communication between AI applications and external tools.


MCP Servers

An MCP server exposes tools, metadata, schemas, descriptions, and callable functions.

For example, a fetch_crypto_price tool might expose metadata such as:

{
  "name": "fetch_crypto_price",
  "description": "Fetches current cryptocurrency prices",
  "input_schema": {
    "symbol": "string"
  }
}

This metadata becomes critical — because the LLM can read descriptions written in plain English.


Why Metadata Matters

The LLM understands English. So when given multiple tools:

  • fetch_crypto_price
  • search_google_results
  • fetch_youtube_transcripts

the LLM can determine which tool is appropriate, what arguments are required, and when to invoke it.

The backend no longer needs massive if-else logic. Instead, the backend provides tool metadata and the LLM chooses tools intelligently — effectively moving decision-making from the backend into the LLM.


MCP Flow

The complete flow becomes:

  1. User sends query
  2. Backend gathers tool metadata from connected MCP servers
  3. Metadata + user query are sent to the LLM
  4. LLM recommends a tool and the arguments to use
  5. Backend executes the tool call
  6. Tool response is returned to the backend
  7. LLM generates a grounded final response

This transforms the LLM into the "brain" of the system. The backend becomes simpler and more generic.


Example: YouTube Search

Suppose the user asks Explain reflection using the top 5 YouTube videos.

The LLM reads available tool descriptions, identifies the YouTube MCP tool, constructs the proper arguments, and instructs the backend to call it. The backend executes the MCP request, gets the results, and sends them back to the LLM, which summarizes everything into a coherent answer.

This is modern AI orchestration.


Handling PDFs

Suppose a user uploads physics.pdf and asks Explain reflection from this PDF.

A naive solution would extract the entire PDF and feed everything into the LLM. But this creates major problems:

  • Slow inference
  • Context window limits
  • Excessive token usage

Instead, we chunk documents.


Chunking

A large PDF is divided into smaller sections — paragraphs, passages, or fixed-size chunks. For example:

100-page PDF
→ 300 chunks

Each chunk is converted into embeddings. Then semantic similarity search identifies which chunks are relevant to the query. Only the relevant chunks are passed into the LLM.

This dramatically improves speed, cost, scalability, and accuracy. This is the foundation of modern document RAG systems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment