Thomas Kim Pham thpham

How to install Hadoop on your local Kubernetes cluster

Okey this is not the easiest way of running Hadoop on your local computer and probably you should instead just install it locally.

However if you really insist doing this here's how:

Install kubectl, minikube and Docker if you don't already have it. I recommend using package-manager like Chocolatey. Minikube should install with VirtualBox as default driver which I recommend. When starting minikube we should increase its memory limit since our Hadoop node's pods need at least 2GB: minikube --memory 4096 --cpus 2 start (minikube's default is 1GB). NOTE: actually the Hadoop cluster by default uses about 10GB in memory limits and about 3GB running memory. From what I looked my k8s will overprovision to 300% of its capacity limits but use far less.
Install helm. Then run helm init.
Now you

You need at least WeeChat 3.2-dev

Enable mouse

/mouse enable

FWIW: I'm not the author of the content presented here (which is an outline from Edmond Lau's book). I've just copy-pasted it from somewhere over the Internet, but I cannot remember what exactly the original source is. I was also not able to find the author's name, so I cannot give him/her the proper credits.

Effective Engineer - Notes

By Edmond Lau
Highly Recommended 👍
http://www.theeffectiveengineer.com/

What's an Effective Engineer?

Lima (Linux virtual machines, on macOS) installation guide for M1 Mac.

Sep. 27th 2021 UPDATED

Now we can install patched version of QEMU via Homebrew (thank you everyone for the info!). Here is the updated instruction with it:

Used M1 Mac mini 2020 with macOS Big Sur Version 11.6.

1. Install QEMU & Lima

Electron apps

Slide deck frameworks supporting Markdown source

Running Ubuntu 21.04 on QEMU MicroVM

I mean, in this era of containers, it's not like you'd need this very often, but if you do, you do.

Download the required image files

https://cloud-images.ubuntu.com/releases/hirsute/release/unpacked/ubuntu-21.04-server-cloudimg-amd64-vmlinuz-generic https://cloud-images.ubuntu.com/releases/hirsute/release/unpacked/ubuntu-21.04-server-cloudimg-amd64-initrd-generic https://cloud-images.ubuntu.com/releases/hirsute/release/ubuntu-21.04-server-cloudimg-amd64-disk-kvm.img

	{ stdenv, fetchFromGitHub, fetchzip }:

	stdenv.mkDerivation rec {
	pname = "it930x-firmware";
	version = "1";

	src = fetchFromGitHub {
	owner = "nns779";
	repo = "px4_drv";
	rev = "90e0a4b30b812e7e5fff4483144f165de8914157";

	###########################################################
	# Automation of Everything #
	# How To Combine Argo Events, Workflows, CD, and Rollouts #
	# https://youtu.be/XNXJtxkUKeY #
	###########################################################

	# Requirements:
	# - k8s v1.19+ cluster with nginx Ingress

	# Replace `[...]` with the GitHub organization or the username

	#!/bin/bash

	# !!WARNING!!
	# This will DELETE all efforts you have put into configuring nix
	# Have a look through everything that gets deleted / copied over

	nix-env -e '.*'

	rm -rf $HOME/.nix-*
	rm -rf $HOME/.config/nixpkgs

	# Instructions for fresh install

	$ sh <(curl -L https://nixos.org/nix/install) --darwin-use-unencrypted-nix-store-volume --daemon
	# reboot
	$ source /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
	$ echo 'export NIX_PATH=darwin-config=$HOME/.nixpkgs/darwin-configuration.nix:$HOME/.nix-defexpr/channels${NIX_PATH:+:}$NIX_PATH' \| tee -a ~/.zshrc
	$ echo 'source $HOME/.nix-profile/etc/profile.d/hm-session-vars.sh' \| tee -a ~/.zshrc
	$ nix-channel --add https://nixos.org/channels/nixpkgs-unstable
	$ nix-channel --add https://github.com/LnL7/nix-darwin/archive/master.tar.gz darwin
	$ nix-channel --add https://github.com/nix-community/home-manager/archive/master.tar.gz home-manager