markuskreitzer

Deploying Ollama and OpenWebUI to Kubernetes

Deployment Order

• Create the PVs and PVCs for ollama and open-webui.
• Deploy the ollama Deployment and Service.
• Deploy the open-webui Deployment and Service.
• Finally, apply the Ingress for open-webui.

kubectl apply -f ollama-pv.yaml

markuskreitzer

Sync Time on Linux

Some quirky systems out there doesn't have the usual utilities to sync time. If they have access to the internet, here's a quick way. I found this somewhere on the internet, but don't remember where.

DATE=$(wget -qSO- --max-redirect=0 google.com 2>&1 | grep Date: | cut -d' ' -f5-8)
sudo date -s "${DATE}Z"

markuskreitzer

import re
from typing import List, Tuple
import requests
from pathlib import Path
from ollama import AsyncClient
import asyncio
from tqdm import tqdm 
import sys
import json

markuskreitzer

import gitlab
import argparse
from typing import Iterable

def fix_cicd(project: Project, replacements:Iterable, branch='gitlab-ultimate-migration'):
    f = project.files.get('.gitlab-ci.yml', ref=branch)
    f_new = f.content
    for replacement in replacements:
        f_new.replace(replacement[0], repalcement[1])

markuskreitzer

Running Ollama on Docker Compose

Assuming you start with a vanilla Ubuntu, you need to install Nvidia stuff

Add Nvidia apt registries:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

markuskreitzer

version: '3.6'
services:
  web:
    image: 'gitlab/gitlab-ee:latest'
    platform: 'linux/amd64'
    restart: always
    hostname: 'g.example.dev'
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://g.example.dev:8443'

markuskreitzer

Create a secure SSH server with a static IP or DNS entry. It is important to set up keys and cache public key of destination server prior to these steps. There are many guides on doing this, so we'll skip over those.

The Script

Create a script with the following content. You can save it as /root/revese_ssh.sh for this example.

#!/usr/bin/env bash

REMOTE_PORT=22

markuskreitzer

#!/usr/bin/env python3
from pathlib import Path
import sys
import hashlib 

my_path = Path(sys.argv[1])
b = {}

def filehash(filename: Path):
    with filename.open("rb") as f:

markuskreitzer

#!/bin/bash
# Adapted from Nicolas Boyer by elec3647
# Original post: https://dev.to/nicolasboyer/migrate-all-of-your-repos-from-github-to-gitea-3fk
# Changelog:
# 1. Added ability to pull a user's personal repos or organization.
# 2. Updated `sed` command to work on Linux.
# TODO: Add some logic to pick between org or user.
# TODO: Give ownership to org in Gitea if pulling org.

GITHUB_USERNAME=""

markuskreitzer

https://docs.microsoft.com/en-us/windows/dev-environment/javascript/nodejs-on-wsl