Skip to content

Instantly share code, notes, and snippets.

View saswata-dutta's full-sized avatar
💭
I may be slow to respond.

Saswata Dutta saswata-dutta

💭
I may be slow to respond.
View GitHub Profile
@saswata-dutta
saswata-dutta / github-to-bitbucket
Created April 15, 2019 09:33 — forked from sangeeths/github-to-bitbucket
Forking a Github repo to Bitbucket
Go to Bitbucket and create a new repository (its better to have an empty repo)
git clone git@bitbucket.org:abc/myforkedrepo.git
cd myforkedrepo
Now add Github repo as a new remote in Bitbucket called "sync"
git remote add sync git@github.com:def/originalrepo.git
Verify what are the remotes currently being setup for "myforkedrepo". This following command should show "fetch" and "push" for two remotes i.e. "origin" and "sync"
git remote -v
@saswata-dutta
saswata-dutta / Json.scala
Created April 9, 2019 16:10 — forked from pchiusano/Json.scala
Simple JSON parser combinator library that does not use zippers
// WARNING! totally untested, I have only compiled the code! :)
package json
import collection.immutable.Map
import scalaz.{\/, MonadPlus}
import scalaz.\/._
import scalaz.std.vector._
import scalaz.std.map._
import scalaz.std.list._
@saswata-dutta
saswata-dutta / query_athena.py
Created April 4, 2019 13:56 — forked from sysboss/query_athena.py
SQL Query Amazon Athena using Python
#!/usr/bin/env python3
#
# Query AWS Athena using SQL
# Copyright (c) Alexey Baikov <sysboss[at]mail.ru>
#
# This snippet is a basic example to query Athen and load the results
# to a variable.
#
# Requirements:
# > pip3 install boto3 botocore retrying
@saswata-dutta
saswata-dutta / spark_mergeSchema
Last active March 31, 2019 09:51
Demo schema evolution in parquet -- Merging schema
scala> val df = spark.read.option("mergeSchema", "true").parquet("/Users/saswatdutta/playground/df/df3/*"); df.printSchema
root
|-- id: long (nullable = true)
|-- created_at: long (nullable = true)
df: org.apache.spark.sql.DataFrame = [id: bigint, created_at: bigint]
scala> val df = spark.read.option("mergeSchema", "true").parquet("/Users/saswatdutta/playground/df/df1/*"); df.printSchema
root
|-- id: long (nullable = true)
@saswata-dutta
saswata-dutta / Example.java
Created March 29, 2019 16:07 — forked from RyanRamchandar/Example.java
Cancel a running or queued Call with OkHttp3
// ...
Request request = new Request.Builder()
.url(url)
.tag(TAG)
.build();
// Cancel previous call(s) if they are running or queued
OkHttpUtils.cancelCallWithTag(client, TAG);
// New call
@saswata-dutta
saswata-dutta / avro_describe_feilds.py
Last active March 27, 2019 16:52
get all Field's names from an Avro schema json.
# -*- coding: utf-8 -*-
import avro.schema
def get_schema_children(avro_schema):
children = []
if isinstance(avro_schema, avro.schema.UnionSchema):
children = avro_schema.schemas
if isinstance(avro_schema, avro.schema.RecordSchema):
@saswata-dutta
saswata-dutta / GADTs.scala
Created March 20, 2019 19:03 — forked from pchiusano/GADTs.scala
GADT support in Scala
/** GADTs in Scala and their limitations */
/** Background: what is an algebraic data type (ADT) ?
* ADT: (possibly) recursive datatype with sums and products
* In scala - a trait with case classes (case class is product, subtyping is sum)
*/
/** Motivation: untyped embedded DSL doesn't prevent nonsensical expressions */
sealed trait Expr {
def apply(other: Expr) = Ap(this, other)
@saswata-dutta
saswata-dutta / expr.scala
Created March 19, 2019 14:22 — forked from calincru/expr.scala
Expression problem in Scala
// The expression problem is a new name for an old problem. The goal is to
// define a datatype by cases, where one can add new cases to the datatype and
// new functions over the datatype, without recompiling existing code, and while
// retaining static type safety (e.g., no casts).
// (Philip Wadler)
import scala.language.implicitConversions
object ExpressionProblem extends App {
@saswata-dutta
saswata-dutta / curl.md
Created March 18, 2019 15:10 — forked from anzeljg/curl.md
A curl tutorial using GitHub's API

Introduction

An introduction to curl using GitHub's API

The Basics

Makes a basic GET request to the specifed URI

curl https://api.github.com/users/caspyin
@saswata-dutta
saswata-dutta / curl.md
Created March 18, 2019 15:09 — forked from btoone/curl.md
A curl tutorial using GitHub's API

Introduction

An introduction to curl using GitHub's API.

The Basics

Makes a basic GET request to the specifed URI

curl https://api.github.com/users/caspyin