Skip to content

Instantly share code, notes, and snippets.

@WillKoehrsen
Created August 15, 2018 14:40
Show Gist options
  • Save WillKoehrsen/f4177739841867982ef73320388b084e to your computer and use it in GitHub Desktop.
Save WillKoehrsen/f4177739841867982ef73320388b084e to your computer and use it in GitHub Desktop.
# Chunk size for 104 sections
chunk_size = len(clients) // 103
# Create list of lists of clients in each section
client_lists = [clients[i:i + chunk_size] for i in range(0, len(clients), chunk_size)]
def create_partition(client_list, partition_number):
"""Creates and saves a dataset with users in user_list"""
# Subset data
subset = data[client_list]
# Save subset data with partition number as identifier
subset.to_csv('p%s.csv' % partition_number)
# Iterate through each sublist of clients
for i, client_list in enumerate(client_lists):
create_partition(client_list, i)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment