Zoltan C. Toth zoltanctoth

MLOps, Scalable Data Platforms, Big Data, Databricks, dbt,

zoltanctoth / newmonth.sh

Last active April 14, 2018 07:40

create new month in datapao admin

gfind 2018-02\ február -type d  -print0 | sed 's/2018-02 február/2018-03 március/g' | xargs -0 -I {} mkdir -p {}

zoltanctoth / ggplot2-solutions.r

Created May 9, 2017 04:43

Here you can find the solutions for the exercises we used at the class.

	library(ggplot2)

	# Take a look at our example dataset
	View(diamonds)

	# Make a chart from scratch
	x = ggplot() +
	layer(
	data = diamonds, mapping = aes(x=carat,y=price),
	stat='identity', position="identity", geom="point"

zoltanctoth / ggplot2-examples.r

Last active January 17, 2021 18:10

ggplot2 examples and exercises

	library(ggplot2)

	# Take a look at our example dataset
	View(diamonds)

	# Make a chart from scratch
	x = ggplot() +
	layer(
	data = diamonds, mapping = aes(x=carat,y=price),
	stat='identity', position="identity", geom="point"

zoltanctoth / spark-kafka.scala

Last active February 6, 2017 20:23

How to use the Direct Kafka Source in Scala with offset Specification

	import org.apache.spark._
	import org.apache.spark.sql.Column
	import org.apache.spark.streaming._
	import _root_.kafka.serializer.StringDecoder
	import org.apache.spark.streaming._
	import org.apache.spark.sql.SparkSession
	import org.apache.spark.sql.SQLContext
	import org.apache.kafka.clients.consumer.ConsumerRecord
	import org.apache.kafka.common.TopicPartition
	import org.apache.kafka.common.serialization.StringDeserializer

zoltanctoth / spark-kafka.scala

Created February 6, 2017 20:09

How to use the Direct Kafka Source in Scala

	object Anomymizer extends App {

	val spark = SparkSession.builder
	.master("local[3]")
	.appName("Anonimizer")
	.getOrCreate()

	val salt = "SAALT"
	def anonimizeStr(a:Any) = {
	a match {

zoltanctoth / spark-kafka.scala

Created February 6, 2017 20:09

How to use the Direct Kafka Source in Scala

	object Anomymizer extends App {

	val spark = SparkSession.builder
	.master("local[3]")
	.appName("Anonimizer")
	.getOrCreate()

	val salt = "SAALT"
	def anonimizeStr(a:Any) = {
	a match {

zoltanctoth / spark-kafka.scala

Created February 6, 2017 20:09

How to use the Direct Kafka Source in Scala

	object Anomymizer extends App {

	val spark = SparkSession.builder
	.master("local[3]")
	.appName("Anonimizer")
	.getOrCreate()

	val salt = "SAALT"
	def anonimizeStr(a:Any) = {
	a match {

zoltanctoth / h2o-sparkling-water-deep-learning.scala

Created September 13, 2016 20:09

This is a Spark <-> H2O / Sparkling water deep learning prototype.

	import org.apache.spark.{SparkConf, SparkContext}
	import org.apache.spark.h2o.{H2OContext, H2OFrame}
	import org.apache.spark.sql.DataFrame

	import hex.deeplearning.DeepLearning
	import water.app.SparkContextSupport

	import hex.deeplearning.DeepLearningParameters
	import hex.deeplearning.DeepLearningParameters.Activation
	import org.apache.spark.h2o.{DoubleHolder, H2OContext, H2OFrame}

zoltanctoth / move-wordpress-to-different-domain.sh

Last active September 25, 2015 06:26

Moving wordpress to an other domain can be a hassle. Here is a script on how to do it in without the pain.

	#!/bin/bash -xeu
	# This script moves your wordrpress page under a different domain
	# Zoltan C. Toth
	export HISTCONTROL=ignorespace

	ORIGIN_DOMAIN=teszt2.gyulahus.hu
	TARGET_DOMAIN=teszt.gyulahus.hu
	ORIGIN_DIR=/home/gyulahus/public_html/$ORIGIN_DOMAIN
	TARGET_DIR=/home/gyulahus/public_html/$TARGET_DOMAIN
	TARGET_DB=teszt2_gyh

zoltanctoth / pyspark-udf.py

Last active July 15, 2023 13:23

Writing an UDF for withColumn in PySpark

	from pyspark.sql.types import StringType
	from pyspark.sql.functions import udf

	maturity_udf = udf(lambda age: "adult" if age >=18 else "child", StringType())

	df = spark.createDataFrame([{'name': 'Alice', 'age': 1}])
	df.withColumn("maturity", maturity_udf(df.age))

	df.show()