Skip to content

Instantly share code, notes, and snippets.

@zxkane
Last active October 29, 2019 08:02
Show Gist options
  • Save zxkane/349200b4de2d9f11b309fcc31ace7d09 to your computer and use it in GitHub Desktop.
Save zxkane/349200b4de2d9f11b309fcc31ace7d09 to your computer and use it in GitHub Desktop.
Test AWS S3 download performance in Ubuntu LTS 18.04
#!/bin/bash -x
add-apt-repository ppa:plushuang-tw/uget-stable -y
apt install aria2 parallel iotop tmux python3-pip -y
pip3 install awscli --upgrade --user
export AWS_DEFAULT_REGION=ap-northeast-1
SECONDS=0
seq 1 5 | parallel --will-cite -j 5 'url=$(aws s3 presign s3://<your bucket>/test-files/file-{});aria2c --file-allocation=none -c -x 4 -s 4 -d /dev -o null $url'
duration=$SECONDS
RESULT=result-$(curl http://169.254.169.254/latest/meta-data/instance-type)-$(curl http://169.254.169.254/latest/meta-data/instance-id)-seconds-${duration}
touch /tmp/$RESULT
aws s3 cp /tmp/$RESULT s3://<your bucket>/test-files/rt/$RESULT
shutdown -t now
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment