Skip to content

Instantly share code, notes, and snippets.

View zouzias's full-sized avatar

Anastasios Zouzias zouzias

51 followers · 52 following
View GitHub Profile
@zouzias
zouzias / README.txt
Last active April 23, 2017 17:25
Migrate PGP key for sbt to new computer (spark-lucenerdd)
Install pgptools https://gpgtools.org
@zouzias
zouzias / distanceGeoShapeQuery.json
Created March 29, 2017 06:36
Examples of Elasticsearch's geo_shape queries
{ "query":
{ "bool" : {
"must" : {
"match_all" : {}
},
"filter": {
"geo_shape": {
"coords": {
"shape": {
"type": "circle",
@zouzias
zouzias / output.txt
Last active March 28, 2017 11:27
LuceneRDD Question, run with `bin/spark-shell --packages org.zouzias:spark-lucenerdd_2.11:0.2.7`
(123ABC23,123ABC23QQ,AA-123ABC23-XYZ,123ABC23XYZ,AA-123ABC23AA)
(123XYZAA,)
(56789XY,)
@zouzias
zouzias / init-jenkins-ldap.groovy
Created January 21, 2017 11:20
Jenkins LDAP Groovy Configuration
import jenkins.*
import hudson.*
import com.cloudbees.plugins.credentials.*
import com.cloudbees.plugins.credentials.common.*
import com.cloudbees.plugins.credentials.domains.*
import com.cloudbees.jenkins.plugins.sshcredentials.impl.*
import hudson.plugins.sshslaves.*;
import hudson.model.*
import jenkins.model.*
import hudson.security.*
@zouzias
zouzias / Dockerfile
Last active November 24, 2016 15:16
OpenStreetMap (OSM) to Postgres
FROM postgres:9.6.1
RUN apt-get update
RUN apt-get install -y osmosis wget postgis
@zouzias
zouzias / mysqlJDBCSparkExample.scala
Last active May 5, 2019 23:40
Spark JDBC DataFrame Example
val jdbc = spark.read.format("jdbc").option("url", "jdbc:mysql://localhost:3306/wikipedia").option("dbtable", "wikipedia.page").option("user", "root").option("password", "XXXX").load
@zouzias
zouzias / nginxproxy.md
Created November 7, 2016 23:29 — forked from soheilhy/nginxproxy.md
How to proxy web apps using nginx?

Virtual Hosts on nginx (CSC309)

When hosting our web applications, we often have one public IP address (i.e., an IP address visible to the outside world) using which we want to host multiple web apps. For example, one may wants to host three different web apps respectively for example1.com, example2.com, and example1.com/images on the same machine using a single IP address.

How can we do that? Well, the good news is Internet browsers

@zouzias
zouzias / gist:beea2a445d55c1cecfa38cbc55623e31
Created November 4, 2016 09:48
Cellural Technologies
https://danielmiessler.com/study/cellular/#gs.EnPVl6A
@zouzias
zouzias / ShapeLuceneRDD.scala
Last active November 4, 2016 06:14
ShapeLuceneRDD: Max mind cities self spatial join example
import org.zouzias.spark.lucenerdd.spatial.shape._
import org.zouzias.spark.lucenerdd._
import org.zouzias.spark.lucenerdd.spatial.shape.ShapeLuceneRDD
val citiesDF = spark.read.parquet("s3://recordlinkage/world-cities-maxmind.parquet").repartition(40)
citiesDF.cache
val total = citiesDF.count
@zouzias
zouzias / DFvsLuceneRDD.scala
Last active October 20, 2016 16:25
Preliminary tests: DataFrame vs LuceneRDD example (2x r3.xlarge core nodes)
import org.zouzias.spark.lucenerdd.facets.FacetedLuceneRDD
import org.apache.spark.sql.DataFrame
val citiesDF = spark.read.parquet("s3://recordlinkage/world-cities-maxmind.parquet").repartition(8)
citiesDF.cache
val total = citiesDF.count
println(s"Cities: ${total}")
val fieldName = "Country"
val k: Int = 20