Skip to content

Instantly share code, notes, and snippets.

@takeru

takeru/download.py

Last active October 12, 2023 13:27
Show Gist options
  • Save takeru/9f2d1b2b0b466356cceabfbb04e84c10 to your computer and use it in GitHub Desktop.
Save takeru/9f2d1b2b0b466356cceabfbb04e84c10 to your computer and use it in GitHub Desktop.
Download ZIP
GCEのサービスアカウントでGCSからファイルをGET
Raw
download.py
# python -m venv .venv
# .venv/bin/python3 -m pip install google-cloud-storage chardet
# .venv/bin/python3 download.py gs://xxx/xxx/xxx.txt xxx.txt
import sys
from google.cloud import storage
from urllib.parse import urlparse
_, gsurl, output_path = sys.argv
url = urlparse(gsurl)
bucket_name = url.netloc
blob_name = url.path[1:]
# print("bucket_name=", bucket_name)
# print("blob_name=", blob_name)
# print("output_path=", output_path)
client = storage.Client()
bucket = client.bucket(bucket_name)
blob = bucket.blob(blob_name)
blob.download_to_filename(output_path)
Raw
get_gcs_obj.sh
BUCKET_NAME=baketsu
OBJECT_NAME=path/to/file.txt
OUTPUT_PATH=./file.txt
ACCESS_TOKEN=$(curl -s -H 'Metadata-Flavor: Google' http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token | jq -r .access_token)
OBJECT_NAME=`echo "$OBJECT_NAME" | jq -Rr @uri`
url="https://storage.googleapis.com/storage/v1/b/$BUCKET_NAME/o/$OBJECT_NAME?alt=media"
curl -X GET -H "Authorization: Bearer $ACCESS_TOKEN" -o "$OUTPUT_PATH" "$url"
@takeru
Copy link
Author

takeru commented Oct 12, 2023 •
edited
Loading

https://stackoverflow.com/questions/64555078/copy-files-to-container-optimised-os-from-a-gcp-storage-bucket
https://stackoverflow.com/questions/64781844/google-cloud-storage-downloading-a-file-using-curl-request
https://zenn.dev/apstndb/articles/10349708f25985
https://cloud.google.com/storage/docs/request-endpoints?hl=ja

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment