AIAnytime

FROM ./Llama-3-ELYZA-JP-8B-q4_k_m.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"

AIAnytime

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from openai import OpenAI
from dotenv import load_dotenv
import os

load_dotenv()

app = FastAPI()
client = OpenAI()

AIAnytime

import requests

# Base endpoint
base_url = "https://r.jina.ai/"

# Input URL to be appended
input_url = "https://www.stateof.ai/"

# Full URL with the input URL appended after a plus (+) sign
full_url = base_url + input_url

AIAnytime

import streamlit as st

# App title
st.title('EcoOptimizer Carbon Footprint Calculator')

# Introduction
st.write('''
This tool calculates the Software Carbon Intensity (SCI) for EcoOptimizer, 
an AI-powered energy management system for commercial buildings by EcoTech Solutions.
''')

AIAnytime

training_arguments = TrainingArguments(
        output_dir="./results",
        per_device_train_batch_size=4,
        per_device_eval_batch_size=4,
        gradient_accumulation_steps=2,
        optim="adamw_8bit",
        logging_steps=50,
        learning_rate=1e-4,
        evaluation_strategy="steps",
        do_eval=True,

AIAnytime

## With Evaluation Harness
!pip install git+https://github.com/EleutherAI/lm-evaluation-harness.git
!pip install bitsandbytes
!pip install --upgrade transformers
!pip install auto-gptq optimum autoawq

!lm_eval --model hf --model_args pretrained=google/gemma-7b --tasks winogrande,hellaswag,arc_challenge --device cuda:0 --num_fewshot 1 --batch_size 8 --output_path ./eval_harness/gemma-7b
  
!lm_eval --model hf --model_args pretrained=google/gemma-7b --tasks winogrande,hellaswag,arc_challenge --device cuda:0 --num_fewshot 5 --batch_size 8 --output_path ./eval_harness/gemma-7b-5shot
  

AIAnytime

import os
#set you runpod key as a environment variable
os.environ['RUNPOD_API_KEY'] = "your_runpod_api_key"

import runpod
from IPython.display import display, Markdown

runpod.api_key = os.getenv("RUNPOD_API_KEY", "your_runpod_api_key")

if runpod.api_key == "your_runpod_api_key":

AIAnytime

Hello everyone, I am Sneha Roy, working as a full-stack AI Engineer and recently I have been working on Generative AI projects. 
Today, I will be showcasing a solution that we have developed.........."GCC KB Chatbot". 
it is an intelligent assistant tailored to simplify how call centers operate, in this case.. the GCC team.

GCC faces a significant challenge managing a high volume of tickets daily. 
Each of these tickets is needing assistance, and each minute spent resolving one is critical. 
Traditionally, our colleagues at GCC relied on a comprehensive but cumbersome Knowledge Base (KB) to find solutions, 
which significantly contributed to longer turnaround times.

That's where our solution comes in...  it drastically reduces the time taken to resolve tickets. Acting as a virtual agent, 

AIAnytime

import torch
from transformers import BertTokenizer, BertModel

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')

def encode(text):
    tokens = tokenizer.tokenize(text)
    tokens = ['[CLS]'] + tokens + ['[SEP]']
    return tokenizer.convert_tokens_to_ids(tokens)

AIAnytime

Nucleus sampling is a technique used in large language models to control the randomness and diversity of generated text. It works by sampling from only the most likely tokens in the model’s predicted distribution.

The key parameters are:

Temperature: Controls randomness, higher values increase diversity.

Top-p (nucleus): The cumulative probability cutoff for token selection. Lower values mean sampling from a smaller, more top-weighted nucleus.

Top-k: Sample from the k most likely next tokens at each step. Lower k focuses on higher probability tokens.