Skip to content

Instantly share code, notes, and snippets.

@alvesjnr
Created June 21, 2011 13:30
Show Gist options
  • Select an option

  • Save alvesjnr/1037851 to your computer and use it in GitHub Desktop.

Select an option

Save alvesjnr/1037851 to your computer and use it in GitHub Desktop.
Firt try: mapreduce for counting
#coding: utf-8
def map(key, value):
for word in value:
yield (word,1)
def reduce(list_maps):
reduced = {}
for value in list_maps:
if value[0] in reduced:
reduced[value[0]] += value[1]
else:
reduced[value[0]] = value[1]
return [(key, reduced[key]) for key in reduced]
if __name__=='__main__':
"""
Attention: case sensitive!!!
"""
text = open('longtext.txt').read()
text = text.split() #remove all whitespaces
mapped_text = map(None,text)
reduced_text = reduce(mapped_text)
print reduced_text
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment