Yordan Georgiev YordanGeorgiev

1-daydreaming 2-know what you really want 3-commit to do it 4-know how-to to do it 5-can do it 6-do it actually 7-no regrets having done it

YordanGeorgiev / delete-file-if-it-exists.scala

Created September 10, 2018 14:04

[delete file if exists in spark] how-to delete a file if it exists in hdfs spark scala

-    val fs = org.apache.hadoop.fs.FileSystem.get(spark.sparkContext.hadoopConfiguration)
-    if (fs.exists(new org.apache.hadoop.fs.Path(cleanedAdpFileUri)))
-      fs.delete(new Path(cleanedAdpFileUri), true)

YordanGeorgiev / chain-of-boolean-funcs.scala

Created September 7, 2018 08:02

[chaining bool returning functions] control flow for bool returning functions in chain #scala #functions #bool #chain #control-flow

YordanGeorgiev / how-to-test-for-empty-string-in-bash.sh

Created August 17, 2018 08:35

[bash test for empty string] how-to test for empty string in bash #bash #if #string #else

	if [ -z "$Emails" ]; then
	msg="Emails to set mail to not set !!! you need to export Emails=list-of-emails-to-send-mail-to-comma-delimited" ;
	export exit_code=1 ;
	doExit "$msg";
	exit 1 ;
	fi

YordanGeorgiev / aws-s3cmd-how-to.sh

Last active August 17, 2018 08:13

[aws s3cmd how-to] aws s3cmd how-toc #aws #s3cmd #config #s3cmd-config

	# set those var names ( not the values ;o) in your shell
	# good luck trying those ...
	export aws_profile=my-profile
	export bucket=phub-1525928545-127
	export endpoint=https://es-si-s3-z2.ecloud.comm.net:1010
	export aws_secret_access_key=56FDD9520FF3F9DA835A6E36f783903
	export aws_access_point_id=49585117C01847189BC8BFE0b50D561A
	export AWS_SECRET_ACCESS_KEY=56FDD9520ACD4D9DA835A6E36f783903
	export AWS_ACCESS_KEY_ID=49585217C06847189AC8BFE0b50D561A

YordanGeorgiev / create-empty-dataframe.scala

Created August 15, 2018 08:35

[create-empty-dataframe] how-to create empty data frame #scala #dataframe

	val rowsSeq: Seq[Row] = Seq()
	val rowsRDD: RDD[Row] = spark.sparkContext.parallelize(rowsSeq)
	spark.createDataFrame(rowsRDD, schema)

YordanGeorgiev / repositories

Last active January 15, 2019 08:55

[global sbt repositories file]global sbt repositories file #sbt #repositories

	# file: ~/.sbt/repositories
	[repositories]
	local
	sbt-releases-repo: http://repo.typesafe.com/typesafe/ivy-releases/,[organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
	sbt-plugins-repo: http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext]
	maven-central: http://repo1.maven.org/maven2/

	# check also the following credentials file
	# ~/.sbt/0.13/credentials.sbt

YordanGeorgiev / create-df-with-nullable-cols.scala

Last active August 10, 2018 11:14

[how-to create a hardcoded dataframe with nullable columns] how-to create a hardcoded dataframe with nullable columns #dataframe #nulls #nullable

	// obs ! you need the .asInstanceOf[<<Type>>] otherwise a serialization exception will be thrown
	// format: off
	val mdtDs = Seq(
	(Some(45.5806766), Some(10.4940080), null.asInstanceOf[Double], null.asInstanceOf[Double],
	null.asInstanceOf[Double])
	).toDS()

	val mdtDf = mdtDs.toDF(
	"latitude",
	"longitude",

YordanGeorgiev / how-to-edit-a-linux-user.sh

Created July 4, 2018 20:30

[how-to edit a linux user] how-to edit a linux user #linux #user

	# modify a user
	usermod -a -G $group $user

	# change the password for the specified user (own password)
	passwd $user
	#how-to forces to change password when logging in for the first time
	passwd -f login
	#change user pass to expire never
	chage -I -1 -m 0 -M 99999 -E -1 $user
	# and check results

YordanGeorgiev / how-to-add-a-linux-user.sh

Created July 4, 2018 20:29

[how-to add a linux user] how-to add a linux user #linux #user #user-management #useradd

	# START === user management
	#how-to add a linux group
	export group=usr_name
	export gid=10001
	sudo groupadd -g "$gid" "$group"
	sudo cat /etc/group \| grep --color "$group"

	export user=grp_name
	export uid=10001
	export home_dir=/home/$user

YordanGeorgiev / how-to-create-table.md

Created June 26, 2018 15:28

[how-to create table in md] how-to create a table in md format #table #md #readme

Name	Required	Type	Default	Description
name	true	[String, Number]		Name of the modal
delay	false	Number	0	Delay between showing overlay and actual modal box
pivotY	false	Number (0 - 1.0)	0.5	Vertical position in %, default is 0.5 (meaning that modal box wii
ll be in the middle (50% from top) of the window