Skip to content

Instantly share code, notes, and snippets.

@u8sand
Created April 9, 2020 16:55
Show Gist options
  • Save u8sand/ebc1907c02f731165ebb4e1e6a7bf108 to your computer and use it in GitHub Desktop.
Save u8sand/ebc1907c02f731165ebb4e1e6a7bf108 to your computer and use it in GitHub Desktop.
A simple pandas helper for cached reading of pandas files from urls
import os
import pandas as pd
def fetch_save_read(url, file, reader=pd.read_csv, sep=',', **kwargs):
''' Download file from {url}, save it to {file}, and subsequently read it with {reader} using pandas options on {**kwargs}.
'''
if not os.path.exists(file):
df = reader(url, sep=sep, index_col=None)
df.to_csv(file, sep=sep, index=False)
return pd.read_csv(file, sep=sep, **kwargs)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment