Skip to content

Instantly share code, notes, and snippets.

View rampage644's full-sized avatar

Sergei Turukin rampage644

View GitHub Profile
@rampage644
rampage644 / impalad-summary
Created August 24, 2014 09:59
Impalad summary
## Git repo
Find modified impala [here](https://github.com/rampage644/impala-cut). First, have a look at [this](https://github.com/rampage644/impala-cut/blob/executor/README.md) *README* file.
## Task description
Original task was to prune impalad to some sort of *executor* binary which only executes part of query. Two approaches were suggested: top-down and bottom-up. I used bottom-up approach.
My intention was to write unittest that whill actually test the behavior we need. So, look at `be/src/runtime/plan-fragment-executior-test.cc`. It contains all possible tests (that is, actual code snippets) to run part of query with or without data. Doing so helped me a lot to understand impalad codebase relative to query execution.
@rampage644
rampage644 / impala-pth
Created August 20, 2014 13:29
Impala Pth
# Gnu Pth as thread library for impalad
In short, it's impossible to use Gnu Pth library with `impalad` "AS IS", i.e. without modification.
Gnu Pth:
* Gnu Pth can't fully replace `pthreads`. It lacks some functions, some entities.
* It doesn't provide versioned symbols
There are some `*.so` libraries (system/thirdparty) which come precompiled and they are linked against versioned symbols. Be prepared to recompile them replace somehow or just do anything. Example:
@rampage644
rampage644 / seccomp.md
Created August 13, 2014 12:42
Seccomp

Benchmarking

Jump to Results:

Simple experiment showed seccomp-based syscall ~5 times slower than vanila one.

Calling write syscall directly:

const unsigned count = UINT_MAX / 10000;
	unsigned i = 0;
@rampage644
rampage644 / strace-analysis.py
Created August 10, 2014 13:03
Python strace log analysis script
#!/usr/bin/env python
import sys
import re
def main():
regexp = re.compile(r'^(\S+)\((.*)\)\s+=\s+(\d+)$')
whitelist = ['read', 'write', 'fstat', 'lseek', 'fcntl']
opened_fd = {}