Skip to content

Instantly share code, notes, and snippets.

@DaisukeMiyamoto
Created November 20, 2018 06:12
Show Gist options
  • Select an option

  • Save DaisukeMiyamoto/3bd6595539bc6203cd189bae3f3622be to your computer and use it in GitHub Desktop.

Select an option

Save DaisukeMiyamoto/3bd6595539bc6203cd189bae3f3622be to your computer and use it in GitHub Desktop.
bash script for benchmark multi-file and multi-part for Amazon S3
#!/bin/bash
dd if=/dev/zero of=abcd10000mb_1 bs=1M count=10000
dd if=/dev/zero of=efgh10000mb_2 bs=1M count=10000
dd if=/dev/zero of=ijkl10000mb_3 bs=1M count=10000
dd if=/dev/zero of=mnop10000mb_4 bs=1M count=10000
dd if=/dev/zero of=qrst10000mb_5 bs=1M count=10000
dd if=/dev/zero of=uvwx10000mb_6 bs=1M count=10000
dd if=/dev/zero of=yzab10000mb_7 bs=1M count=10000
dd if=/dev/zero of=cdef10000mb_8 bs=1M count=10000
dd if=/dev/zero of=ghij10000mb_9 bs=1M count=10000
dd if=/dev/zero of=klmn10000mb_10 bs=1M count=10000
dd if=/dev/zero of=opqr10000mb_11 bs=1M count=10000
dd if=/dev/zero of=stuv10000mb_12 bs=1M count=10000
aws configure set default.s3.max_concurrent_requests 100
aws configure set default.s3.max_queue_size 1000
aws s3 cp abcd10000mb_1 s3://midaisuk-s3-test/ &
aws s3 cp efgh10000mb_2 s3://midaisuk-s3-test/ &
aws s3 cp ijkl10000mb_3 s3://midaisuk-s3-test/ &
aws s3 cp mnop10000mb_4 s3://midaisuk-s3-test/ &
aws s3 cp qrst10000mb_5 s3://midaisuk-s3-test/ &
aws s3 cp uvwx10000mb_6 s3://midaisuk-s3-test/ &
aws s3 cp yzab10000mb_7 s3://midaisuk-s3-test/ &
aws s3 cp cdef10000mb_8 s3://midaisuk-s3-test/ &
aws s3 cp ghij10000mb_9 s3://midaisuk-s3-test/ &
aws s3 cp klmn10000mb_10 s3://midaisuk-s3-test/ &
aws s3 cp opqr10000mb_11 s3://midaisuk-s3-test/ &
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment