Mert Bozkir mertbozkir

Install with Homebrew:

$ brew install postgresql@14

(The version number 14 needs to be explicitly stated. The @ mark designates a version number is specified. If you need an older version of postgres, use postgresql@13, for example.)

NOTE: commands and UI are deprecated

Content:

Negative Engineering
What is workflow orchestration?
Introduction to Prefect 2.0
First Prefect flow and Basics

Workflow Orchestration

4 Steps in Running LLaMA-7B on a M1 MacBook

The large language models usability

The problem with large language models is that you can’t run these locally on your laptop. Thanks to Georgi Gerganov and his llama.cpp project, it is now possible to run Meta’s LLaMA on a single computer without a dedicated GPU.

Running LLaMA

There are multiple steps involved in running LLaMA locally on a M1 Mac after downloading the model weights.

Creating a chatbot using Alpaca native and LangChain

Let's talk to an Alpaca-7B model using LangChain with a conversational chain and a memory window.

Setup and installation

Install python packages using pip. Note that you need to install HuggingFace Transformers from source (GitHub) currently.

$ pip install git+https://github.com/huggingface/transformers

Multiple Ollama Containers on a single host (with multiple GPUs)

I don't want model RELOAD

I have a large machine with 2 GPUs and a considerable amount of RAM.
I was trying to use ollama to server llava and mistral BUT it would reload the models every time I switched model requests.
So this is the solution that appears to be working: Multiple Containers, each serving a different model, on different ports.

Ollama model working dir:

I have many models already downloaded on my machine so I mount the host ollama working dir to the containers.
Linux (At least on my linux machine) - /usr/share/ollama/.ollama

	import sys
	from awsglue.transforms import *
	from awsglue.utils import getResolvedOptions
	from pyspark.context import SparkContext
	from awsglue.context import GlueContext
	from awsglue.job import Job

	## @params: [JOB_NAME]
	args = getResolvedOptions(sys.argv, ['JOB_NAME'])
	bucketpathparam = getResolvedOptions(sys.argv, ['s3_path'])

	import modal

	vllm_image = modal.Image.debian_slim(python_version="3.10").pip_install(
	[
	"vllm==0.5.3post1", # LLM serving
	"huggingface_hub==0.24.1", # download models from the Hugging Face Hub
	"hf-transfer==0.1.8", # download models faster
	]
	)