VenkataSakethDakuri

import requests

import os

from dotenv import load_dotenv


#import the VAPI_API_KEY from the .env file
load_dotenv()
VAPI_API_KEY = os.getenv("VAPI_API_KEY")

VenkataSakethDakuri

Eval engineering is the discipline of designing and version-controlling automated test suites that rigorously quantify whether each model or prompt change is an improvement, acting like unit tests for AI systems.

Conditional Evals can save costs especially in workflows.
LLM as a judge, heuristics based evals, golden datasets.

In Lora finetuning can create a pipeline where we try out test suites for a certain set of hyperparameters and store the adapter along with the eval results. We can iterate over different values of rank, alpha, target modules based on eval results. 

Benchmarking refers to standardised comparison bw predefined tasks. Evaluation refers to the overall model performance and suitability for intended task.

RAG

VenkataSakethDakuri

from typing import Any, Dict, List, Tuple

import torch
from diffusers import (
    AutoencoderKLCogVideoX,
    CogVideoXDPMScheduler,
    CogVideoXImageToVideoPipeline,
    CogVideoXTransformer3DModel,
)
from diffusers.models.embeddings import get_3d_rotary_pos_embed

VenkataSakethDakuri

HORIZONTAL Scaling vs VERTICAL Scaling
1. LOAD BALANCING REQUIRED,   1. N/A.
2. RESILIENT,   2. Single point of failure.
3. Network calls (RPC),   3. Inter process Communication.
4. DATA INCONSISTENCY.,   4. Consistent
5. SCALES WELL. AS USERS INCREASE,   5. Hardware limit.


Tradional vs Consistent hashing:
The problem with tradional hashing is that whenever we change the no. of servers ( up or down ), lot of data has to be reshuffled across the available servers. This is inefficient, for ex: we may have some cache stored in a particular server and that may become obsolete.