Skip to content

Instantly share code, notes, and snippets.

@heynemann
Created August 13, 2012 14:33
Show Gist options
  • Save heynemann/3341253 to your computer and use it in GitHub Desktop.
Save heynemann/3341253 to your computer and use it in GitHub Desktop.
r3 sample mapper
#!/usr/bin/python
# -*- coding: utf-8 -*-
from r3.worker.mapper import Mapper
class CountWordsMapper(Mapper):
job_type = 'count-words'
def map(self, lines):
#time.sleep(0.5)
return list(self.split_words(lines))
def split_words(self, lines):
for line in lines:
for word in line.split():
yield word.strip().strip('.').strip(','), 1
r3-map --mapper-key="1" --mapper-class="my_app.mapper.CountWordsMapper"
r3-map --mapper-key="2" --mapper-class="my_app.mapper.CountWordsMapper"
r3-map --mapper-key="3" --mapper-class="my_app.mapper.CountWordsMapper"
r3-map --mapper-key="4" --mapper-class="my_app.mapper.CountWordsMapper"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment