Skip to content

Instantly share code, notes, and snippets.

@gidgid
Last active October 9, 2020 10:58
Show Gist options
  • Select an option

  • Save gidgid/06d5b31179a380d52845d43a9c130e13 to your computer and use it in GitHub Desktop.

Select an option

Save gidgid/06d5b31179a380d52845d43a9c130e13 to your computer and use it in GitHub Desktop.
More function parameters usually require a dict to pass them
def crawl_page(url, crawl_request):
output_file = crawl_request["output_file"] # 1
num_retries = crawl_request["num_retries"] # 1
is_polite = crawl_request["is_polite"] # 1
follow_redirects = crawl_request["follow_redirects"] # 1
# rest of the implementation excluded for the sake of brevity
crawl_request = {
"output_file": "page_content.csv",
"num_retries": 4,
"is_polite": True,
"follow_redirects": True,
} # 2
crawl_page(url="http://hopefully.doesnt.exi.st", crawl_request=crawl_request) # 2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment