Örjan Lundberg oluies

WSL 2 + Windows Terminal + Oh My Zsh + Powerlevel10k

This gist takes heavy inspiration from kevin-smets / iterm2-solarized.md gist for iTerm2.

Windows Subsystem for Linux 2 (WSL 2)

The first thing that you will want to do is install WSL 2 and a Linux distro.

Permanent WSL DNS Fix (WSL 2.2.1+)

If you're encountering ping github.com failing inside WSL with a Temporary failure in name resolution, you're not alone — this has been a long-standing issue, especially when using VPNs or corporate networks.

This issue is now fixed robustly with DNS tunneling, which preserves dynamic DNS behavior and avoids limitations like WSL’s former hard cap of 3 DNS servers in /etc/resolv.conf.

DNS tunneling is enabled by default in WSL version 2.2.1 and later, meaning that if you're still seeing DNS resolution issues, the first and most effective fix is simply to upgrade WSL. Upgrading WSL updates the WSL platform itself, but does not affect your installed Linux distributions, apps, or files.

To upgrade WSL, follow these steps,

Spark HWC integration - HDP 3 Secure cluster

Prerequisites :

Kerberized Cluster
Enable hive interactive server in hive
Get following details from hive for spark or try this HWC Quick Test Script

Getting Started in Scala

This is my attempt to give Scala newcomers a quick-and-easy rundown to the prerequisite steps they need to a) try Scala, and b) get a standard project up and running on their machine. I'm not going to talk about the language at all; there are plenty of better resources a google search away. This is just focused on the prerequisite tooling and machine setup. I will not be assuming you have any background in JVM languages. So if you're coming from Python, Ruby, JavaScript, Haskell, or anywhere… I hope to present the information you need without assuming anything.

Disclaimer It has been over a decade since I was new to Scala, and when I was new to Scala, I was coming from a Java and Ruby background. This has probably caused me to unknowingly make some assumptions. Please feel free to call me out in comments/tweets!

One assumption I'm knowingly making is that you're on a Unix-like platform. Sorry, Windows users.

Getting the JVM

Applied Functional Programming with Scala - Notes

1. Mastering Functions

A function is a mapping from one set, called a domain, to another set, called the codomain. A function associates every element in the domain with exactly one element in the codomain. In Scala, both domain and codomain are types.

val square : Int => Int = x => x * x

	// Copyright (c) 2019 Andrey Akinshin
	// Licensed under The MIT License https://opensource.org/licenses/MIT
	using System;
	using System.Collections.Generic;
	using System.Linq;

	/// <summary>
	/// The ED-PELT algorithm for changepoint detection.
	///
	/// <remarks>

	# Docker-in-Docker Gitlab runners setup taken from:
	# https://medium.com/@tonywooster/docker-in-docker-in-gitlab-runners-220caeb708ca
	dind:
	restart: always
	privileged: true
	volumes:
	- /var/lib/docker
	image: docker:17.09.0-ce-dind
	command:
	- --storage-driver=overlay2

	# allow `mount` cmd without password
	echo "$USER ALL=NOPASSWD: /bin/mount" \| (sudo su -c 'EDITOR="tee -a" visudo')
	# add the mount directive to `fstab`
	sudo mkdir -p /c
	sudo sh -c "echo '/mnt/c /c none bind' >> /etc/fstab"
	# update to `.bashrc` to auto mount at login
	echo "sudo mount -a" >> ~/.bashrc
	# now reload it
	source ~/.bashrc

	import scalanative.native._
	import SDL._
	import SDLExtra._

	@extern
	@link("SDL2")
	object SDL {
	type Window = CStruct0
	type Renderer = CStruct0

	/**
	* Generate Case class from DataFrame.schema
	*
	* val df:DataFrame = ...
	*
	* val s2cc = new Schema2CaseClass
	* import s2cc.implicit._
	*
	* println(s2cc.schemaToCaseClass(df.schema, "MyClass"))
	*