Skip to content

Instantly share code, notes, and snippets.

@solidpple

solidpple/load_df_from_csv.py

Created December 29, 2017 12:24
Show Gist options
  • Save solidpple/033ea815937cb38872e8e9dc2b51f243 to your computer and use it in GitHub Desktop.
Save solidpple/033ea815937cb38872e8e9dc2b51f243 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
load_df_from_csv.py
def get_df(cols, filename, sep='\t'):
# filename = 'filename'
iter_csv = pd.read_csv(filename,iterator=True, chunksize=1000, sep=sep, names=cols)
# df = pd.concat([chunk[chunk['field'] > constant] for chunk in iter_csv])
df = pd.concat([chunk for chunk in iter_csv])
return df
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment