Created
March 27, 2020 02:25
-
-
Save rosdyana/20c6cbe930958c0b431ffd875ef7ab4d to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pandas as pd | |
| import sys | |
| CHUNK_SIZE = int(sys.argv[2]) | |
| rows = pd.read_csv(sys.argv[1], chunksize=CHUNK_SIZE) | |
| for i, chuck in enumerate(rows): | |
| # i is for chunk number of each iteration | |
| chuck.to_csv('out{}.csv'.format(i)) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment