Skip to content

Instantly share code, notes, and snippets.

@korkridake

korkridake/aws-sagemaker-data-splitting-ep01.py

Created December 31, 2019 18:09
Show Gist options
  • Save korkridake/45f427c754a860540e5926a805e8bb74 to your computer and use it in GitHub Desktop.
Save korkridake/45f427c754a860540e5926a805e8bb74 to your computer and use it in GitHub Desktop.
Download ZIP
AWS Cloud Journey - Analyzing to Deploying Models with Amazon SageMaker (Ep. 1)
Raw
aws-sagemaker-data-splitting-ep01.py
###############################################################
# Split into training, validation, and testing data
###############################################################
train_data, validation_data, test_data = np.split(model_data.sample(frac=1, random_state=1729), [int(0.7 * len(model_data)), int(0.9 * len(model_data))])
train_data.to_csv('train.csv', header=False, index=False)
validation_data.to_csv('validation.csv', header=False, index=False)
###############################################################
# Upload these files to S3
###############################################################
boto3.Session().resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'train/train.csv')).upload_file('train.csv')
boto3.Session().resource('s3').Bucket(bucket).Object(os.path.join(prefix, 'validation/validation.csv')).upload_file('validation.csv')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment