Hamel Husain hamelsmu

Export Traces

Export to Notebook

See our data API guide for function and argument level information.

After logging data to Arize, if you want to retrieve that data to use in code for evaluation, you can visit the LLM Tracing tab to see your traces and use the export button to get your export code.

Add Attributes, Metadata and Tags to Span

You might want to track additional application details manually. This is particularly useful for actions outside standard frameworks or LLM clients.

Add attributes to a span

Attributes let you attach key/value pairs to a spans so it carries more information about the current operation that it's tracking.

Mandatory Packages

The below script will print all the packages modal installs in every image. If these packages cannot be installed, the image will not be built and you will have a bad time.

from modal import Image, App
app = App("01a_modal_requirements")

def see_reqs():

Here is my personal opinion about the questions I posed in this tweet:

I think that fine-tuning is still very valuable in many situations. I’ve done some more digging and I find that people who say that fine-tuning isn't useful are indeed often working on products where fine-tuning isn't likely to be useful:

They are making developer tools - foundation models have been trained extensively on coding tasks.
They are building foundation models and testing for the most general cases. But the foundation models themselves are also being trained for the most general cases.
They are building a personal assistant that isn’t scoped to any type of domain or use case and is essentially similar to the same folks building foundation models.

Axolotl Notes For Prompt Construction

the function parse_instruction_fields generates a tuple (instruction,input,response) The key to making a new input format is to make sure you can parse your input into these parts and deal with them.

prompt_tokenizers.InstructionPromptTokenizingStrategy

    def tokenize_prompt(self, prompt):
        (
            instruction,
            input,  # pylint: disable=redefined-builtin

You are an autoregressive language model that has been fine-tuned with instruction-tuning and RLHF. You carefully provide accurate, factual, thoughtful, nuanced answers, and are brilliant at reasoning. If you think there might not be a correct answer, you say so. Since you are autoregressive, each token you produce is another opportunity to use computation, therefore you always spend a few sentences explaining background context, assumptions, and step-by-step thinking BEFORE you try to answer a question. Your users are experts in AI and ethics, so they already know you're a language model and your capabilities and limitations, so don't remind them of that. They're familiar with ethical issues in general so you don't need to remind them about those either. Don't be verbose in your answers, but do provide details and examples where it might help the explanation. When showing Python code, minimise vertical space, and do not include comments or docstrings; you do not need to follow PEP8, since your users' organiz

{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "I want to read more books\nCan you please generate one option for how to accomplish this?\nPlease make the option very short, at most one line."}], "model": "gpt-3.5-turbo", "max_tokens": 1000, "n": 1, "stream": true, "temperature": 1.0, "top_p": 1.0}

{"messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "I want to read more books\nCan you please generate one option for how to accomplish this?\nPlease make the option very short, at most one line."}], "model": "gpt-3.5-turbo", "max_tokens": 1000, "n": 1

	from fastapi import Request, HTTPException
	from pydantic import BaseModel, BaseModel, HttpUrl
	from modal import Secret, App, web_endpoint, Image
	from typing import Optional, List
	from example import proposal
	import os

	app = App(name="circleback", image=Image.debian_slim().pip_install("openai", "pydantic", "fastapi"))

	class Attendee(BaseModel):

	base_model: mistralai/Mistral-7B-v0.1
	model_type: MistralForCausalLM
	tokenizer_type: LlamaTokenizer
	is_mistral_derived_model: true

	load_in_8bit: false
	load_in_4bit: true
	strict: false

	lora_fan_in_fan_out: false