Adeel Ahmad adeelahmad

After pulling everything together, I thought it would be a good idea to document what ended up working for me with the following setup:

*darr apps
Some 3d printers
MobilRaker
NZB360

This stack requires the following

Protecting the sites from unauthorized access

macOS Live Text has a very good quality/speed tradeoff.

Compared to Tesseract, it has much higher quality and is up to 3x as fast.

Finetune instruct that can enhance LLM capabilities and safety.

flowchart
	StartScriptExecution["Start Script Execution"]
	ReadTextFile["Read Text File"]
	ChunkTextIntoSegments["Chunk Text Into Segments"]
	ForEachTextChunk["For Each Text Chunk"]
	PerformNamedEntityRecognition["Perform Named Entity Recognition (NER)"]

Tokenizer Notes

Praxis Maldevide - Draft A

Introduction

This document is a collection of thoughts and observations about the tokenizers used in llama-rooted large language models.

The Tokenizer

Most language models use the LlamaTokenizer.

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix

LiteLLM: Bypass Cert Verifications

Overview

If you use LiteLLM to proxy requests to Ollama.ai in corporate environments, you may encounter the following error in your Python application:

httpcore.ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1006)

	import os
	import re
	from copy import deepcopy
	from functools import cache
	from typing import Any, AsyncIterator, Protocol, cast

	from mistralai import Mistral
	from openai import AsyncOpenAI, OpenAI

	from unmute.kyutai_constants import LLM_SERVER

	[TASK(s)]

	1.

	[INPUT]


	Design a chatbot interface where, instead of responding with plain text, the AI provides action buttons based on the user’s message. The chatbot should analyze the user input and dynamically generate relevant options as clickable buttons.

	Key Features:

	"""
	Conversation prompt templates.

	We kindly request that you import fastchat instead of copying this file if you wish to use it.
	If you have any changes in mind, please contribute back so the community can benefit collectively and continue to maintain these valuable templates.
	"""

	import base64
	import dataclasses
	from enum import auto, IntEnum