| # turtle.py | |
| # Basic Turtle graphics module for Pythonista | |
| # | |
| # When run as a script, the classic Koch snowflake is drawn as a demo. | |
| # The module can also be used interactively or from other scripts: | |
| # >>> from turtle import * | |
| # >>> right(30) | |
| # >>> forward(100) | |
| # ... |
| Trivia problem: Generate primes between 1 and 100 million with Scalding | |
| Input: Numbers from 1 to 100 million, 1 per line - use a simple C program for this | |
| Output: Filter input for primes using the filter function on pipes in Scalding | |
| Execution time: 13:11:00 to 13:25:54 = 15 minutes in hdfs-local mode | |
| Part files: 27 | |
| $ ls | |
| part-00000 part-00002 part-00004 part-00006 part-00008 part-00010 part-00012 part-00014 part-00016 part-00018 part-00020 part-00022 part-00024 part-00026 part-00001 part-00003 part-00005 part-00007 part-00009 part-00011 part-00013 part-00015 part-00017 part-00019 part-00021 part-00023 part-00025 | |
| $ tail part-00026 | |
| 99999787 |
Suppose you have a key like (page, geo, day) and you want to make rollups/datacube so you can query for all pages, or all geos or all days.
Here is how you do it:
def opts[T](t: T): Seq[Option[T]] = Seq(Some(t), None)
val p: TypedPipe[(String, String, Int)] = ...
p.sumByLocalKeys
| Write a program that does what it’s supposed to do | |
| Write idiomatic code | |
| Debug a program that you wrote | |
| Debug a program someone else wrote | |
| Debug the interaction between a system you wrote and one you didn’t | |
| File a good bug report | |
| Modify a program you didn’t write | |
| Test a program you wrote | |
| Test a program you didn’t write | |
| Learn a new programming language |