Elton Stoneman sixeyed

Runbook - Elasticsearch Fluentd Indices

Fluentd indices filling in Elasticsearch.

Summary

Centralized log collection from application services goes through Fluentd to Elasticsearch.

Fluentd creates an index for each hour of each day's logs - we do not autmatically clear the indices.

Windows on Kube

NOTE: This is based on the gist https://gist.github.com/wsong/34beb37ebd5d22c1bf1266aaeea085d4. That gist was used to set up the Dockercon SF demo; this one is for Dockercon EU 2018.

The below instructions describe how to set up a cluster with one Linux manager and one Windows worker such that you can schedule Kubernetes pods with Windows images on the Windows worker.

These instructions are for a Windows Server 2019 node. You can create these nodes on Azure (as of November 27, 2018).

On Azure, I recommend VMs of size D4s v3 or larger (4 CPUs, 16GB memory)

	az aks nodepool add -g $rgName --cluster-name $aksName `
	-n $nodePoolName `
	--mode User `
	--enable-cluster-autoscaler `
	--node-count 1 `
	--min-count 0 `
	--max-count $nodeMaxCount `
	--node-osdisk-size $aksNodeDiskSize `
	--node-vm-size $nodeVmSize `
	--vnet-subnet-id $subnetId `

	from azure.keyvault.secrets import SecretClient
	from azure.identity import DefaultAzureCredential

	keyVaultName = input("Enter Key Vault name:")
	KVUri = f"https://{keyVaultName}.vault.azure.net"
	secretName = "mySecret"

	credential = DefaultAzureCredential()
	client = SecretClient(vault_url=KVUri, credential=credential)
	retrieved_secret = client.get_secret(secretName)

	param(
	[string] $sqlServerName,
	[string] $sqlPassword
	)

	Start-Transcript -path C:\vm-setup.log

	Write-Output '* Setup starting'

	Write-Output '** Installing Windows features'

	param(
	[string] $sqlServerName,
	[string] $sqlPassword
	)

	Write-Output '* Setup starting'

	Write-Output '** Installing Windows features'
	Install-WindowsFeature Web-Server,NET-Framework-45-ASPNET,Web-Asp-Net45

	Write-Output '* Installing Chocolatey'
	Set-ExecutionPolicy Bypass -Scope Process -Force
	[System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072
	Invoke-Expression ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))

	Write-Output '* Installing tools'
	choco install -y git
	choco install -y vscode

	Write-Output '-VM setup script done-'

	build_and_push() {

	ip=$1
	os=$2
	arch=$3

	docker \
	--host tcp://$ip:2376 --tlsverify --tlscacert $ca --tlscert $cert --tlskey $key \
	image build --pull -t org/repo:$os-$arch .