mrseanryan’s gists

mrseanryan / toxic-or-neutral-labelled.ipynb

Created June 9, 2024 19:14

notebook--toxic-or-neutral-labelled-dataset

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

mrseanryan / gist:2f168d1f268759e10601b09aa4b2db4a

Created January 31, 2024 12:02

Download an LLM for use with text-generation-webui on Ubuntu

	#!/bin/bash
	# Download an LLM for use with text-generation-webui on Ubuntu

	pushd text-generation-webui

	echo "=== === Downloading an LLM ..."
	# python3 download-model.py TheBloke/CodeLlama-13B-GGUF
	python3 download-model.py mistralai/Mistral-7B-v0.1

	echo "=== === [done]"

mrseanryan / gist:f1f572f8a16c0dd37741bdb232bb4c61

Last active January 31, 2024 12:01

Basic NON interactive install of text-generation-webui to host LLM on Ubuntu - for 24GB GPU such as AWS EC2 G4 or G5 box

	#!/bin/bash
	# Basic NON interactive install of text-generation-webui to host LLM on Ubuntu

	GPU_GB=${1:-24}

	#sudo apt update
	#sudo apt install python3-pip

	git clone https://github.com/oobabooga/text-generation-webui.git

mrseanryan / create-ec2-gpu-llm.yaml

Last active February 1, 2024 09:39

CloudFront script to create an EC2 GPU instance hosting an LLM

	AWSTemplateFormatVersion: 2010-09-09
	Resources:
	LlmEc2KeyPair:
	Type: AWS::EC2::KeyPair
	Properties:
	KeyName: !Join [ "-", [ "llmkeys", !Ref "AWS::StackName" ] ]
	LlmSshSecurityGroup:
	Type: AWS::EC2::SecurityGroup
	Properties:
	GroupName: !Join [ "-", [ "llmsshsg", !Ref "AWS::StackName"]]

mrseanryan / describe_instance_types.sh

Last active January 22, 2024 10:49

AWS: describe available instance types with GPU memory

aws ec2 describe-instance-types --instance-types g4dn.xlarge g5.xlarge --query "InstanceTypes[].GpuInfo" --no-cli-pager

mrseanryan / gist:a7d8ed4bfda2f3f6e495bfe719e8d35c

Created January 19, 2024 15:25

Bash script to lint Python code via flake8 - shows how to disable rules.

	# so future errors halt the script.
	set -e

	echo Linting ...

	# Disable less useful errors
	# - ref https://www.flake8rules.com/
	#
	# E302 Expected 2 blank lines
	# E305 Expected 2 blank lines after end of function or class

mrseanryan / gist:2dcdb8182c05d1ea116e6bd0b772ba3d

Last active January 22, 2024 12:14

Create script to start existing install of text-generation-webui

	#!/bin/bash -e
	GPU_GB=${1:-24}
	pushd /home/ubuntu/text-generation-webui
	GPU_CHOICE=A USE_CUDA118=FALSE LAUNCH_AFTER_INSTALL=TRUE INSTALL_EXTENSIONS=TRUE ./start_linux.sh --public-api --extensions openai --api-port 5000 --auto-devices --gpu-memory $GPU_GB
	# --share

mrseanryan / gist:6bf26b78c6af43fe38d322533c8e10b5

Last active January 22, 2024 16:38

AWS EC2 script to install text-generation-webui as a Ubuntu service

	#!/bin/bash -e
	GPU_GB=${1:-24}
	OUT_FILE=/etc/systemd/system/llmtextgen.service
	if [ -e $OUT_FILE ]
	then
	echo LLM servic 'llmtextgen.service' already installed - exiting.
	exit 0;
	fi

	echo Downloading script to /home/ubuntu/start-llm.sh

mrseanryan / gist:dd1e945316c1b0c4a763aa65eccd3fb8

Created January 16, 2024 16:28

Launch text-generation-webui as a Ubuntu service

	#/etc/systemd/system/llmtextgen.service
	[Unit]
	After=network.target

	[Service]
	Type=simple
	ExecStart=/bin/bash /home/ubuntu/start-llm.sh

	User=ubuntu
	Group=ubuntu

mrseanryan / gist:1f6ee66fde867ac37e9c57da2d7a3f09

Last active January 22, 2024 16:39

Install text-generation-webui with GPU support - via llama-cpp-python

	sudo apt update
	sudo apt install python3-pip

	conda create -n textgen python=3.10.9
	conda activate textgen
	conda init bash
	pip install torch torchvision torchaudio
	git clone https://github.com/oobabooga/text-generation-webui
	cd text-generation-webui
	pip install -r requirements.txt

Sean Ryan mrseanryan