Skip to content

Instantly share code, notes, and snippets.

View frgomes's full-sized avatar

Richard Gomes frgomes

View GitHub Profile
@frgomes
frgomes / SimilarSchema.scala
Last active December 18, 2022 03:58
Spark - Compare schemas, ignoring ``nullable`` settting
package object spark {
import org.apache.spark.sql.types.StructType
implicit class StructTypeExtension(schema: StructType) {
import org.apache.spark.sql.types.StructField
implicit def similar(other: StructType): Boolean = _similar(schema, other)
implicit val fieldOrdering: Ordering[StructField] = Ordering.by(field => field.name)
private final def _similar(_this: StructType, _other: StructType): Boolean =
@frgomes
frgomes / ProductExtension.scala
Created October 29, 2022 01:37
Scala - obtain field names and field types from case class employing TypeTag
import scala.reflect.runtime.universe.TypeTag
implicit class ProductExtension[T <: Product : TypeTag](o: T) {
def productElementNames: Iterator[String] = {
import scala.reflect.runtime.universe._
typeOf[T].members
.collect { case m: MethodSymbol if m.isCaseAccessor => m.name.toString }
.toList.reverse.toIterator
}
@frgomes
frgomes / SparkExampleTests.scala
Created April 10, 2022 21:50
Spark - template for integration tests
package com.example
import utest._
object SparkExampleTests extends TestSuite {
import scala.util.{Try,Success,Failure}
import org.apache.spark.sql.SparkSession
val _spark: Try[SparkSession] =
Try {
@frgomes
frgomes / Configs.scala
Last active April 10, 2022 12:47
SBT - multiproject build with projectmatrix and additional configurations
//file: project/Configs.scala
import sbt._
object Configs {
val FunctionalTest = config("ft") extend (Test)
val AcceptanceTest = config("at") extend (Test)
val PerformanceTest = config("pt") extend (Test)
val Tools = config("tools") extend (Test)
}
@frgomes
frgomes / sbt_disable_publishing.sbt
Last active March 28, 2022 23:49
SBT - disable publishing
def disablePublishing: Seq[Setting[_]] =
Seq(
publish/skip := true,
publishLocal/skip := true
)
@frgomes
frgomes / SlickConfig.scala
Created March 9, 2022 19:50
Scala - Read database credentials from file
trait SlickConfig {
import scala.util.{Try, Success, Failure}
private val RegexDB2 = "^jdbc:(db2):.*//(.*):(.*)/([^;]*)[?:;](.*)".r
private val RegexDerby = "^jdbc:(derby):.*//(.*):(.*)/([^;]*)[?:;](.*)".r
private val RegexH2 = "^jdbc:(h2):(?:mem):()()([^;]*)[?:;](.*)".r
private val RegexHsqlDB = "^jdbc:(hsqldb):.*//(.*):(.*)/([^;]*)[?:;](.*)".r
private val RegexSqlServer = "^jdbc:(sqlserver)://(.*):(.*);DatabaseName=([^;]*)[?:;](.*)".r
private val RegexjTDS = "^jdbc:(jtds):sqlserver://(.*):(.*)/([^;]*)[?:;](.*)".r
private val RegexMySQL = "^jdbc:(mysql):.*//(.*):(.*)/([^;]*)[?:;](.*)".r
@frgomes
frgomes / sequence.scala
Last active July 22, 2024 22:53
FP - sequence :: Seq[Try[T]] to Try[Seq[T]]
import scala.util.Try
implicit class SeqTryExtension[T](seq: Seq[Try[T]]) {
def sequence: Try[Seq[T]] =
seq
.foldRight(Try(List.empty[T])) {
case (item, acc) => for { a <- acc; i<- item } yield i :: a
}
}
@frgomes
frgomes / yaml_solve_references.rs
Created January 29, 2022 18:35
Rust - Render YAML resolving environment variables and YAML references.
#![allow(unused_parens)]
use anyhow::{Context,Result, anyhow};
use clap::{arg, App, AppSettings};
use std::ffi::OsStr;
fn main() -> Result<()> {
let matches = App::new("mkvm")
.about("Make virtual machines easily!")
.setting(AppSettings::SubcommandRequiredElseHelp)
@frgomes
frgomes / kvm_host_network_interfaces
Created January 10, 2022 01:42
Linux - Turn off AAAA queries in Debian
## /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
source /etc/network/interfaces.d/*
# The loopback network interface
auto lo
iface lo inet loopback
System Information
-------------------
Manufacturer: HP
Product Name: HP ENVY x360 Convertible 15-ee0xxx