Created
April 9, 2020 16:55
-
-
Save u8sand/ebc1907c02f731165ebb4e1e6a7bf108 to your computer and use it in GitHub Desktop.
A simple pandas helper for cached reading of pandas files from urls
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import pandas as pd | |
def fetch_save_read(url, file, reader=pd.read_csv, sep=',', **kwargs): | |
''' Download file from {url}, save it to {file}, and subsequently read it with {reader} using pandas options on {**kwargs}. | |
''' | |
if not os.path.exists(file): | |
df = reader(url, sep=sep, index_col=None) | |
df.to_csv(file, sep=sep, index=False) | |
return pd.read_csv(file, sep=sep, **kwargs) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment