Skip to content

Instantly share code, notes, and snippets.

@harshavardhana
Last active August 10, 2016 16:11
Show Gist options
  • Select an option

  • Save harshavardhana/1e0bda879afb798634dd to your computer and use it in GitHub Desktop.

Select an option

Save harshavardhana/1e0bda879afb798634dd to your computer and use it in GitHub Desktop.
s4cmd v/s mc

11MB object upload to a given bucket

$ time s4cmd put test s3://ferenginar/test6
test => s3://ferenginar/test6

real	0m14.545s
user	0m0.227s
sys	  0m0.143s
$ time mc cp test s3/ferenginar/test7
test:                         10.66 MB / 10.66 MB  ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓  100.00 % 1.73 MB/s 6s

real	0m8.543s
user	0m0.639s
sys	  0m0.283s

Recursive listing of all the objects in a given bucket

$ time s4cmd ls -r s3://ferenginar 1>/dev/null
real	1m29.988s
user	1m20.354s
sys	  0m13.123s
$ time mc ls -r s3/ferenginar/ 1>/dev/null
real	0m17.036s
user	0m2.446s
sys	  0m0.371s

Recursive du calculation across a given bucket

$ time s4cmd -r du s3://miniocloud
20344375        s3://miniocloud/Go/
192624368	s3://miniocloud/bin/
2742	s3://miniocloud/Readme First.txt
21918456	s3://miniocloud/clang
2742	s3://miniocloud/hello.txt
10	s3://miniocloud/helo helo.txt
107040	s3://miniocloud/man
10358436	s3://miniocloud/mc.darwin.amd64
10345472	s3://miniocloud/mc.exe
21918456	s3://miniocloud/myobject
13728	s3://miniocloud/nothing-like-anything
21918456	s3://miniocloud/test1
21918456	s3://miniocloud/testobject
27168	s3://miniocloud/true

real	0m6.265s
user	0m0.678s
sys	  0m0.310s
$ time mc ls --json -r s3/miniocloud | jq .size | awk '{sum+=$1} END {print sum}'
321499905

real	0m1.593s
user	0m0.173s
sys	  0m0.081s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment