Skip to content

Instantly share code, notes, and snippets.

@ChengLuFred
Last active December 1, 2020 19:54
Show Gist options
  • Save ChengLuFred/f3ac09ec143f912a75c8db3b75670b81 to your computer and use it in GitHub Desktop.
Save ChengLuFred/f3ac09ec143f912a75c8db3b75670b81 to your computer and use it in GitHub Desktop.
[File Management in Python]file combine, zip extract, etc. #Python

1. Quick way to read file using pandas

file_name = 'file_name.csv'
tickers_info = pd.read_csv(file_name,
                           sep = ",",			 # ',' for csv file and '\t' for text file
						   header = 0,			 # row number used as head
                           usecols=[1, 2], 		 # column index to be included
                           index_col = [0], 	 # column index which apdated above
                           engine='c') 			 # c engine is faster than python

tickers_info = pd.read_csv(file_name, sep = ",", header = 0, usecols=[1, 2], index_col = [0], engine='c') 	

2. File Combination

combine all .csv files within a file_path

import glob
import pandas as pd

file_list = glob.glob(file_path + file_type)
data_combined = pd.concat([pd.read_csv(file) for file in file_list])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment