- https://github.com/Azure/blobxfer
- https://github.com/Azure/blobxfer/blob/master/docs/10-cli-usage.md
Suppose you give --remote-path "container/1" to blobxfer in downloading blobs with blobxfer,
as blobxfer use internally of pathlib on remote paths, you'll get all blobs whose vdirpath names include "container/1" like this:
container/1/*
container/10/*
container/15/*
If you want to get only blobs under "container/1/" vdirpath, use a --include <vdirpath> filter, e.g., --include "1/*"
--strip-components N will strip the leading N components from the local file path on upload or remote file path on download. The default is 0.
Suppose you give --local-path "localdir", --remote-path "container/1" and --include "1/*" to blobxfer in downloading blobs with blobxfer,
you'll get all blobs under vdirpath "1/", however they are downloaded to local path including exactly the same vdirpath like this:
<storage> <local>
container/1/1.png --- downloaded ---> localdir/1/1.png
ctontainer/1/2/2.png --- downloaded ---> localdir/1/2/2.png
If you want to download files to local directory without keeping vdirpath, strip the vdirpath by using --strip-components N,
e.g.,
--strip-components 1 if vdirpath you want to strip is only 1 dir like 1/1.png, then you will get the file without vdirpath, localdir/1.png
--strip-components 2 if vdirpath you want to strip is 2 dirs like 1/2/2.png, then you will get the file without vdirpath, localdir/2.png
or
--strip-components 10 (big enough) if vdirpath you want to strip is 2 dirs like 1/2/3/3.png, then you will get the file vdirpath, localdir/3.png
You store blob files in your Azure blob storage with the following structure,
./container/
├── 1
│ ├── 1.png
│ └── 2
│ ├── 2.png
│ └── 3
│ └── 3.png
├── 10
│ ├── 10.png
│ └── 1010
│ ├── 1010-01.png
│ ├── 1010-02.png
│ └── 1010-03.png
└── 15
└── 155
└── 155.png
+───+───+───+
1 2 3 (N)
Suppose you give the following options to blobxfer in downloading blobs with blobxfer
--remote-path "container/1"--strip-components 0--local-path "localdir"
blobxfer downloads blob files to local like this:
./localdir
├── 1
│ ├── 1.png
│ └── 2
│ ├── 2.png
│ └── 3
│ └── 3.png
├── 10
│ ├── 10.png
│ └── 1010
│ ├── 1010-01.png
│ ├── 1010-02.png
│ └── 1010-03.png
└── 15
└── 155
└── 155.png
Suppose you give the following options to blobxfer in downloading blobs with blobxfer
--remote-path "container/1"--strip-components 0--local-path "localdir"--include "1/*"
blobxfer downloads blob files to local like this:
./localdir
└── 1
├── 1.png
└── 2
├── 2.png
└── 3
└── 3.png
Suppose you give the following options to blobxfer in downloading blobs with blobxfer
--remote-path "container/1"--strip-components 1--local-path "localdir"--include "1/*"
blobxfer downloads blob files to local like this:
./localdir
├── 1.png
└── 2
├── 2.png
└── 3
└── 3.png
Suppose you give the following options to blobxfer in downloading blobs with blobxfer
--remote-path "container/1"--strip-components 2--local-path "localdir"--include "1/*"
blobxfer downloads blob files to local like this:
./localdir
├── 1.png
├── 2.png
└── 3
└── 3.png
Suppose you give the following options to blobxfer in downloading blobs with blobxfer
--remote-path "container/1"--strip-components 3--local-path "localdir"--include "1/*"
blobxfer downloads blob files to local like this:
./localdir
├── 1.png
├── 2.png
└── 3.png
#!/bin/bash
STORAGE_ACCOUNT_NAME="<MY STORAGE ACCOUNT NAME>"
STORAGE_ACCOUNT_KEY="<MY STORAGE ACCOUNT KEY>"
REMOTE_CONTAINER="container" ## Container name
VDIRPATH="1" ## Virtual Dir Path
LOCAL_PATH="/Users/yoichika/localdir" ## Local dir path
blobxfer download --storage-account $STORAGE_ACCOUNT_NAME \
--storage-account-key $STORAGE_ACCOUNT_KEY \
--remote-path "$REMOTE_CONTAINER/$VDIRPATH" \
--local-path $LOCAL_PATH \
--include "$VDIRPATH/*" \
--recursive \
--strip-components 10 \
--mode auto \
--log-file out.txt \
--skip-on-lmt-ge \
--transfer-threads 10