VenkataSakethDakuri

import requests

import os

from dotenv import load_dotenv


#import the VAPI_API_KEY from the .env file
load_dotenv()
VAPI_API_KEY = os.getenv("VAPI_API_KEY")

VenkataSakethDakuri

Eval engineering is the discipline of designing and version-controlling automated test suites that rigorously quantify whether each model or prompt change is an improvement, acting like unit tests for AI systems.

Conditional Evals can save costs especially in workflows.
LLM as a judge, heuristics based evals, golden datasets.

In Lora finetuning can create a pipeline where we try out test suites for a certain set of hyperparameters and store the adapter along with the eval results. We can iterate over different values of rank, alpha, target modules based on eval results. 

Benchmarking refers to standardised comparison bw predefined tasks. Evaluation refers to the overall model performance and suitability for intended task.

RAG

VenkataSakethDakuri

from typing import Any, Dict, List, Tuple

import torch
from diffusers import (
    AutoencoderKLCogVideoX,
    CogVideoXDPMScheduler,
    CogVideoXImageToVideoPipeline,
    CogVideoXTransformer3DModel,
)
from diffusers.models.embeddings import get_3d_rotary_pos_embed