Skip to content

Instantly share code, notes, and snippets.

@copyninja
Created October 19, 2010 11:39
Show Gist options
  • Save copyninja/634062 to your computer and use it in GitHub Desktop.
Save copyninja/634062 to your computer and use it in GitHub Desktop.
Simple Driver program for Crawler
#!/usr/local/bin/python3
from crawler import Crawler
def crawling_done(url):
print(("Crawling {} done".format(url)))
if __name__ == "__main__":
c = Crawler(crawling_done,"http://kn.wikipedia.com/wiki/Karnataka","kn_IN")
c.crawl()
c.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment