zhiyue zhiyue

How to install Hadoop on your local Kubernetes cluster

Okey this is not the easiest way of running Hadoop on your local computer and probably you should instead just install it locally.

However if you really insist doing this here's how:

Install kubectl, minikube and Docker if you don't already have it. I recommend using package-manager like Chocolatey. Minikube should install with VirtualBox as default driver which I recommend. When starting minikube we should increase its memory limit since our Hadoop node's pods need at least 2GB: minikube --memory 4096 --cpus 2 start (minikube's default is 1GB). NOTE: actually the Hadoop cluster by default uses about 10GB in memory limits and about 3GB running memory. From what I looked my k8s will overprovision to 300% of its capacity limits but use far less.
Install helm. Then run helm init.
Now you

https://redplus.me/post/set-up-headless-windows-10-virtual-machine-on-remote-linux-server/ https://www.ostechnix.com/install-oracle-virtualbox-ubuntu-16-04-headless-server/ https://www.howtoforge.com/tutorial/running-virtual-machines-with-virtualbox-5.1-on-a-headless-ubuntu-16.04-lts-server/

Installation

$ sudo nano /etc/apt/sources.list
      deb http://download.virtualbox.org/virtualbox/debian stretch contrib
$ wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
$ sudo apt-get update

	# Command Line Interface
	xcode-select --install

	# Homebrew
	/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

	# Homebrew Cask
	brew tap caskroom/cask

	# TeX environment

	#!/usr/bin/env bash

	################################################################
	# Script_Name : install-xrdp-1.9.2.sh
	# Description : Perform an automated custom installation of xrdp
	# on ubuntu 16.04.4
	# Date : March 2018
	# written by : Griffon
	# Web Site :http://www.c-nergy.be - http://www.c-nergy.be/blog
	# Version : 1.9.2

	#!/bin/bash
	echo -e '#!/bin/bash\n' > list.sh

	echo -e '########### Brew Install List ###########' >> list.sh
	brew leaves \| sed 's/^/brew install /' >> list.sh &&

	echo -e '\n########### Brew Cask Install List ###########' >> list.sh &&
	brew cask list \| sed 's/^/brew cask install /' >> list.sh &&

	echo -e '\n########### Mac App Store Install List ###########' >> list.sh &&

	import serial
	import re
	import requests
	ser = serial.Serial('/dev/ttyUSB0')
	ser.write('AT+CSCS="UCS2"\n')
	while 1:
	read=ser.readline()
	match=re.match(r'\+CMTI: "ME",(\d+)',read)
	if(match):
	ser.write('AT+CMGR=' + match.group(1) + '\n')

	#!/usr/bin/env bash
	#
	# Fetch secrets for local development from Azure KeyVault
	# and print them to stdout as a bunch of env var exports.
	# These secrets should be added to your local .env file
	# to enable running integration tests locally.
	#
	KEY_VAULT=$1

	function fetch_secret_from_keyvault() {

	from __future__ import print_function

	import requests
	import threading

	import sys


	def eprint(args, *kwargs):
	print(args, file=sys.stderr, *kwargs)

	#!/bin/bash

	# Reference guides:
	# https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF
	# https://pve.proxmox.com/wiki/Pci_passthrough
	# https://pve.proxmox.com/wiki/Nested_Virtualization

	# Remember to turn on SVM in BIOS and disable CSM

	# Update packages

	"""This module contains the ``SeleniumMiddleware`` scrapy middleware"""

	import asyncio

	from pyppeteer import launch
	from scrapy import signals
	from scrapy.http import HtmlResponse
	from twisted.internet import defer

	from .http import PuppeteerRequest