Maziyar Panahi maziyarpanahi

Master: Dependencies

apt update && apt upgrade -y
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF

Just documenting docs, articles, and discussion related to gRPC and load balancing.

https://github.com/grpc/grpc/blob/master/doc/load-balancing.md

Seems gRPC prefers thin client-side load balancing where a client gets a list of connected clients and a load balancing policy from a "load balancer" and then performs client-side load balancing based on the information. However, this could be useful for traditional load banaling approaches in clound deployments.

https://groups.google.com/forum/#!topic/grpc-io/8s7UHY_Q1po

gRPC "works" in AWS. That is, you can run gRPC services on EC2 nodes and have them connect to other nodes, and everything is fine. If you are using AWS for easy access to hardware then all is fine. What doesn't work is ELB (aka CLB), and ALBs. Neither of these support HTTP/2 (h2c) in a way that gRPC needs.

http://mojodna.net/2014/05/14/kvm-libvirt-and-ubuntu-14-04.html

Get a qcow2 image from here: https://cloud-images.ubuntu.com/releases/16.04/release/
Convert it with qemu-img
Resize it (we'll add 50GB). This will be a "thin" resize.

Example: https://cloud-images.ubuntu.com/releases/16.04/release/ubuntu-16.04-server-cloudimg-amd64-disk1.img

The basics

This is a collection of basic "recipes", many using twurl (the Swiss Army Knife for the Twitter API!) and jq to query the Twitter API and format the results. Also, some scripts to test or automate common actions.

Importing and exporting CSV files with PostgreSQL

Let’s see how to use PostgreSQL to import and export CSV files painlessly with the COPY command.

Import CSV into table t_words:

COPY t_words FROM '/path/to/file.csv' DELIMITER ',' CSV;

You can tell quote char with QUOTE and change delimiter with DELIMITER.

##VGG16 model for Keras

This is the Keras model of the 16-layer network used by the VGG team in the ILSVRC-2014 competition.

It has been obtained by directly converting the Caffe model provived by the authors.

Details about the network architecture can be found in the following arXiv paper:

Very Deep Convolutional Networks for Large-Scale Image Recognition

K. Simonyan, A. Zisserman

	import os
	import time
	from pynput import keyboard
	from datetime import datetime
	import subprocess
	import threading
	import tkinter as tk
	import queue

	# ML imports

	base_model: meta-llama/Meta-Llama-3-70B
	model_type: LlamaForCausalLM
	tokenizer_type: AutoTokenizer

	load_in_8bit: false
	load_in_4bit: true
	strict: false

	datasets:
	- path: /home/migel/ai_datasets/tess-v1.5b-chatml.jsonl

	#!/bin/sh

	log() {
	format="$1"; shift
	# shellcheck disable=SC2059
	>&2 printf "$format\n" "$@"
	}

	usage() {
	>&2 cat <<EOF

	import org.apache.spark.ml.feature.{CountVectorizer, RegexTokenizer, StopWordsRemover}
	import org.apache.spark.mllib.clustering.{LDA, OnlineLDAOptimizer}
	import org.apache.spark.mllib.linalg.Vector

	import sqlContext.implicits._

	val numTopics: Int = 100
	val maxIterations: Int = 100
	val vocabSize: Int = 10000