Skip to content

Instantly share code, notes, and snippets.

@rain-1
rain-1 / llama-home.md
Last active December 27, 2025 05:31
How to run Llama 13B with a 6GB graphics card

This worked on 14/May/23. The instructions will probably require updating in the future.

llama is a text prediction model similar to GPT-2, and the version of GPT-3 that has not been fine tuned yet. It is also possible to run fine tuned versions (like alpaca or vicuna with this. I think. Those versions are more focused on answering questions)

Note: I have been told that this does not support multiple GPUs. It can only use a single GPU.

It is possible to run LLama 13B with a 6GB graphics card now! (e.g. a RTX 2060). Thanks to the amazing work involved in llama.cpp. The latest change is CUDA/cuBLAS which allows you pick an arbitrary number of the transformer layers to be run on the GPU. This is perfect for low VRAM.

  • Clone llama.cpp from git, I am on commit 08737ef720f0510c7ec2aa84d7f70c691073c35d.
@hrishioa
hrishioa / load_and_process_open_source_licenses.ts
Created May 6, 2023 06:23
Simple Typescript file demonstrating chunked, chained LLM calls to process large amounts of text.
// Requires the gpt library from https://github.com/hrishioa/socrate and the progress bar library.
// Created by Hrishi Olickel ([email protected]) (@hrishioa). Reach out if you have trouble running this.
import { ThunkQueue } from '../../utils/simplethrottler';
import {
AcceptedModels,
Messages,
askChatGPT,
getMessagesTokenCount,
getProperJSONFromGPT,
@virattt
virattt / agent_with_custom_tool.ipynb
Last active March 15, 2025 14:26
agent_with_custom_tool.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jasonjmcghee
jasonjmcghee / cached_chroma.py
Last active November 2, 2025 10:35
Cached embeddings in Chroma made easy.
from abc import ABC
from typing import List, Optional, Any
import chromadb
from langchain.docstore.document import Document
from langchain.embeddings.base import Embeddings
from langchain.vectorstores import Chroma
class CachedChroma(Chroma, ABC):

IAM Tips

Those tips where posted between June and July 2022 on LinkedIn by Roberto Migli.

#IAM tip #1: There are 4 main types of IAM policies: Identity-based policies, resource-based policies, permissions boundaries, Organizations SCPs and Session Policies. Matt Luttrell's blog post will guide you through when and how to use them.

HowAndWhenWithRolesBlog

@dabit3
dabit3 / basicmarket.sol
Last active January 9, 2024 08:54
Basic NFT marketplace
// SPDX-License-Identifier: MIT OR Apache-2.0
pragma solidity ^0.8.4;
import "@openzeppelin/contracts/utils/Counters.sol";
import "@openzeppelin/contracts/token/ERC721/extensions/ERC721URIStorage.sol";
import "@openzeppelin/contracts/security/ReentrancyGuard.sol";
import "@openzeppelin/contracts/token/ERC721/ERC721.sol";
contract NFT is ERC721URIStorage {
using Counters for Counters.Counter;
@divyajyotiuk
divyajyotiuk / get_twitter_bookmarks.py
Last active February 10, 2024 05:40
Python code to get text and link of the bookmarked tweets and save in markdown
import json
import glob
all_bookmarks = []
md_file = open("bookmarks.md", "w+") # saving in markdown file, if no file exists using '+' creates one
files = [file for file in glob.glob("JSONBookmarks/*")] # using glob to read all files from the folder
for file_name in files:
print(file_name)
with open(file_name) as bk:
@velniukas
velniukas / bellman.py
Created February 26, 2019 13:59 — forked from joninvski/bellman.py
Bellman ford python implementation
import pdb
"""
The Bellman-Ford algorithm
Graph API:
iter(graph) gives all nodes
iter(graph[u]) gives neighbours of u
graph[u][v] gives weight of edge (u, v)
"""
import PySimpleGUI as sg
import os
import chess
import chess.pgn
import copy
import time
button_names = ('close', 'cookbook', 'cpu', 'github', 'pysimplegui', 'run', 'storage', 'timer')
CHESS_PATH = '.' # path to the chess pieces
@dkn22
dkn22 / bellman_ford.py
Created September 17, 2018 19:20
Bellman-Ford algorithm for shortest paths
import numpy as np
import numba
@numba.jit
def bellman_ford(graph, start_vertex):
n = len(graph.nodes)
A = np.zeros((n+1, n))
A[0, :] = np.inf
A[0, start_vertex] = 0