Skip to content

Instantly share code, notes, and snippets.

@eddies
eddies / apache-spark.rb
Created January 2, 2019 04:22
Apache Spark 2.2.0 Homebrew
class ApacheSpark < Formula
desc "Engine for large-scale data processing"
homepage "https://spark.apache.org/"
url "https://archive.apache.org/dist/spark/spark-2.2.0/spark-2.2.0-bin-hadoop2.7.tgz"
version "2.2.0"
sha256 "97fd2cc58e08975d9c4e4ffa8d7f8012c0ac2792bcd9945ce2a561cf937aebcc"
head "https://github.com/apache/spark.git"
bottle :unneeded
@laughedelic
laughedelic / sbt-dependency-management-guide.md
Last active March 20, 2025 09:36
Explicit dependency management in sbt

Some of these practices might be based on wrong assumptions and I'm not aware of it, so I would appreciate any feedback.

  1. avoiding some dependency conflicts:

    • install sbt-explicit-dependencies globally in your ~/.sbt/{0.13,1.0}/plugins/plugins.sbt
    • run undeclaredCompileDependencies and make the obvious missing dependencies explicit by adding them to libraryDependencies of each sub-project
    • (optionally) run unusedCompileDependencies and remove some obvious unused libraries. This has false positives, so ; reload; Test/compile after each change and ultimately run all tests to see that it didn't break anything
    • (optionally) add undeclaredCompileDependenciesTest to the CI pipeline, so that it will fail if you have some undeclared dependencies
  2. keeping dependencies up to date and resolving conflicts:

    • install sbt-updates globally in your `~/.sbt/{0.13,1.0}/plugins/plugins.
@mskimorg
mskimorg / Reservoir.scala
Last active August 11, 2020 17:15
Reservoir Sampling for Scala Spark
import scala.reflect.ClassTag
import scala.util.Random
class Reservoir[T: ClassTag](
private val size: Int,
private val seed: Long = Random.nextLong()) extends Serializable {
private val rand = new Random(seed)
private val reservoir = new Array[T](size)
private var count = 0L
@melvic-ybanez
melvic-ybanez / akka_http_and_cats_validation.md
Last active January 22, 2024 18:29
Request Validation with Akka-HTTP and Cats

Request Validation with Akka-HTTP and Cats

Greg Beech wrote an article on Akka HTTP Entity Validation. This post is on the same topic, except we are going to use typeclass instances, as opposed to subtyping, for form validations.

Sample Problem: User Regsitration

To start with, let us suppose that we are working on some sort of user management service using Scala/Akka. However, here we are only going to focus on the registration part, in which the client is expected to send a user data, in JSON format; and the server needs to validate them, and save the user if the validations succeed, or reject the request with all the errors detected. That said, we should keep in mind that our validations may be used through out the entire codebase, not just in the user registration endpoint.

The rules for our validation are as follows:

@mayneyao
mayneyao / notion2blog.js
Last active June 19, 2025 19:02
Notion.so > Personal Blog | custom domain + disqus comment
const MY_DOMAIN = "agodrich.com"
const START_PAGE = "https://www.notion.so/gatsby-starter-notion-2c5e3d685aa341088d4cd8daca52fcc2"
const DISQUS_SHORTNAME = "agodrich"
addEventListener('fetch', event => {
event.respondWith(fetchAndApply(event.request))
})
const corsHeaders = {
"Access-Control-Allow-Origin": "*",
@peterwake
peterwake / RGeo_with_Proj.md
Last active March 22, 2021 08:34
Installation of RGeo with Proj support (also called Proj4)

Installation of RGeo with Proj support (also called Proj4)

This is a real pain as of writing (18 Oct 2019). I can't promise it'll work either :)

Reference

Let's start from the beginning...

If you've been hacking around already you should do:

@stettix
stettix / things-i-believe.md
Last active July 31, 2025 07:29
Things I believe

Things I believe

This is a collection of the things I believe about software development. I have worked for years building backend and data processing systems, so read the below within that context.

Agree? Disagree? Feel free to let me know at @JanStette.

Fundamentals

Keep it simple, stupid. You ain't gonna need it.

@nrktkt
nrktkt / ExampleSpec.scala
Last active February 27, 2020 07:34
AWS unit tests with localstack and scalatest
import cloud.localstack.docker.annotation.LocalstackDockerProperties
import cloud.localstack.TestUtils
import org.scalatest.{BeforeAndAfterEach, FlatSpec}
import ...LocalstackSpec
@LocalstackDockerProperties(services = Array("s3"))
class ExampleSpec
extends FlatSpec
with BeforeAndAfterEach
with LocalStackSpec {
@april
april / arena-macos-full-screen-fixes.sh
Last active July 16, 2025 17:52
Fixes Magic Arena's broken full screen implementation on macOS
#!/bin/bash
# this forces Arena into full screen mode on startup, set back to 3 to reset
# note that if you go into the Arena "Graphics" preference panel, it will reset all of these
# and you will need to run these commands again
defaults write com.wizards.mtga "Screenmanager Fullscreen mode" -integer 0
defaults write com.wizards.mtga "Screenmanager Resolution Use Native" -integer 0
# you can also replace the long complicated integer bit with any other scaled 16:9
# resolution your system supports.
let mapleader = " "
" :actionlist for all commands
set surround
set multiple-cursors
set history=1000
set visualbell
set noerrorbells
set incsearch " Highlight search results when typing
set hlsearch " Highlight search results
set relativenumber " relative numbers