marmbrus

First a disclaimer: This is an experimental API that exposes internals that are likely to change in between different Spark releases. As a result, most datasources should be written against the stable public API in org.apache.spark.sql.sources. We expose this mostly to get feedback on what optimizations we should add to the stable API in order to get the best performance out of data sources.

We'll start with a simple artificial data source that just returns ranges of consecutive integers.

/** A data source that returns ranges of consecutive integers in a column named `a`. */
case class SimpleRelation(
    start: Int, 
    end: Int)(
    @transient val sqlContext: SQLContext) 

apangin

_hashCode                               java/lang/Object.hashCode()I
_getClass                               java/lang/Object.getClass()Ljava/lang/Class;
_clone                                  java/lang/Object.clone()Ljava/lang/Object;
_dabs                                   java/lang/Math.abs(D)D
_dsin                                   java/lang/Math.sin(D)D
_dcos                                   java/lang/Math.cos(D)D
_dtan                                   java/lang/Math.tan(D)D
_datan2                                 java/lang/Math.atan2(DD)D
_dsqrt                                  java/lang/Math.sqrt(D)D
_dlog                                   java/lang/Math.log(D)D

mikesmullin

Mike's x86-64 Assembly (ASM) Notes

Assembling Binary Machine Code

Operating Modes:

These determine the assumed/default size of instruction operands, and restricts which opcodes are available, and how they are used.

Modern operating systems, booted inside Real mode,

perusio

    # Balancing over 4 nodes by hashing URI (consistently with md5) 
    # onto 16 shards. 

    upstream x0 { server 192.168.0.2;  server 192.168.0.3 backup; }
    upstream x1 { server 192.168.0.2;  server 192.168.0.4 backup; }
    upstream x2 { server 192.168.0.2;  server 192.168.0.5 backup; }
    upstream x3 { server 192.168.0.2;  server 192.168.0.3 backup; }

    upstream x4 { server 192.168.0.3;  server 192.168.0.4 backup; }