Created
August 19, 2020 15:22
-
-
Save HebeHH/14382d210c89781c582af6a5a2dadb71 to your computer and use it in GitHub Desktop.
Extracting Headers from cURL request as a dictionary for the python Requests library
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# The function: | |
get_headers = lambda curl: {k:v for k, v in | |
[[i.strip().strip(r"'-H \$'") for i in | |
[x.split(': ')[0], ': '.join(x.split(': ')[1:])]] | |
for x in curl.split('\n') | |
if '-H' in x]} | |
# You get a curl request in this kind of format from eg: Chrome Developer Tools | |
# Important: all arguments must be on a new line and it must be a raw string (the r before the triple quotes) | |
example_curl = r"""curl 'https://urllib3.readthedocs.io/en/latest/_static/alabaster.css' \ | |
-H 'authority: urllib3.readthedocs.io' \ | |
-H 'dnt: 1' \ | |
-H 'accept: text/css,*/*;q=0.1' \ | |
-H 'sec-fetch-site: same-origin' \ | |
-H 'referer: https://urllib3.readthedocs.io/en/latest/reference/index.html' \ | |
--compressed""" | |
# Using the function: | |
get_headers(example_curl) | |
# Output from this request: | |
""" | |
{'authority': 'sg.element14.com', | |
'upgrade-insecure-requests': '1', | |
'dnt': '1', | |
'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.125 Safari/537.36', | |
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9', | |
'sec-fetch-site': 'same-origin', | |
'sec-fetch-mode': 'navigate', | |
'sec-fetch-user': '?1', | |
'sec-fetch-dest': 'document', | |
'referer': 'https://sg.element14.com/pro-signal/abm-715-rc/electret-microphone-omni-leads/dp/2066501?st=electret%20microphone', | |
'accept-language': 'en-US,en;q=0.9'} | |
""" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment