André Pedersen andreped

RAGAS Evals - Fine Tuning Guide

This guide provides a streamlined approach to implementing RAGAS evaluation while managing OpenAI API rate limits effectively. It's designed to be straightforward, visual, and actionable.

Quick Overview

RAGAS (Retrieval Augmented Generation Assessment) is a framework for evaluating RAG systems with:

Objective metrics without human annotations
Synthetic test data generation
Comprehensive evaluation workflows

	from nonconformist.cp import IcpRegressor
	from nonconformist.cp import IcpClassifier
	from nonconformist.nc import NcFactory
	import lightgbm as lgbm
	# Create the underlying model
	model = lgbm.LGBMRegressor()
	# Default nonconformity measure
	nc = NcFactory.create_nc(model)
	# Inductive conformal regressor
	icp = IcpRegressor(nc)

	/**
	* QuPath script to threshold a single channel of an image.
	* This can also be useful to convert a binary image into QuPath annotations.
	*
	* First written for https://forum.image.sc/t/rendering-wsi-as-overlay-on-top-of-another-wsi/52629/24?u=petebankhead
	*
	* @author Pete Bankhead
	*/

	int channel = 0 // 0-based index for the channel to threshold

	import json

	import base64
	import asyncio
	import pyaudio
	import websockets

	SAMPLE_RATE=16000
	FRAMES_PER_BUFFER = 3200
	API_KEY = '<your AssemblyAI Key goes here>'