Skip to content

Instantly share code, notes, and snippets.

@manuphatak
Last active December 12, 2015 07:03
Show Gist options
  • Save manuphatak/03e3497a64988cb6ebd5 to your computer and use it in GitHub Desktop.
Save manuphatak/03e3497a64988cb6ebd5 to your computer and use it in GitHub Desktop.
Idiomatic text processing. Inverting reduce: instead of running one function on a list of values, run a list of functions on one value.
from functools import reduce
def pipeline(steps, initial=None):
def apply(result, step):
yield from step(result)
yield from reduce(apply, steps, initial)
if __name__ == '__main__':
# BEFORE
before = notify(write(modify(sanitize(concat(read(FILES))))))
# AFTER
process = read, concat, sanitize, modify, write, notify
after = pipeline(process, FILES)
# alternate, BEFORE
opened_files = read(FILES)
meaningless_variable_name = concat(opened_files)
etc = sanitize(meaningless_variable_name)
step4 = modify(etc)
step5 = write(step4)
before = notify(step5)
@manuphatak
Copy link
Author

python 2 conversion, convert yield from to a for loop:

From:

yield from reduce(apply, steps, initial)

To:

for item in reduce(apply, steps, initial):
    yield item

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment