Created
August 24, 2014 09:59
-
-
Save rampage644/4977c599b4808af5cff9 to your computer and use it in GitHub Desktop.
Impalad summary
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Git repo | |
Find modified impala [here](https://github.com/rampage644/impala-cut). First, have a look at [this](https://github.com/rampage644/impala-cut/blob/executor/README.md) *README* file. | |
## Task description | |
Original task was to prune impalad to some sort of *executor* binary which only executes part of query. Two approaches were suggested: top-down and bottom-up. I used bottom-up approach. | |
My intention was to write unittest that whill actually test the behavior we need. So, look at `be/src/runtime/plan-fragment-executior-test.cc`. It contains all possible tests (that is, actual code snippets) to run part of query with or without data. Doing so helped me a lot to understand impalad codebase relative to query execution. | |
## Achievements ? | |
First of all, I actually wrote unittest that executes part of query given the query itself and data. As side effect, i did some impalad codebase research (everything relating to query execution). There are a lot code left aside, though. | |
### Impalad peculiarities | |
1. MPPI-oriented. Uses lots of threads and even more. Vanilla impalad when doing nothing has ~183 threads created. | |
2. Some of functionality is done in JAVA via JNI. | |
3. Closely coupled code, a lot of *Singletones*. | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment