Skip to content

Instantly share code, notes, and snippets.

@korakot
Last active September 29, 2022 21:40
Show Gist options
  • Save korakot/f3600576720206363c734eca5f302e38 to your computer and use it in GitHub Desktop.
Save korakot/f3600576720206363c734eca5f302e38 to your computer and use it in GitHub Desktop.
Mount GCS bucket on Colab
from google.colab import auth
auth.authenticate_user()
!echo "deb http://packages.cloud.google.com/apt gcsfuse-bionic main" > /etc/apt/sources.list.d/gcsfuse.list
!curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
!apt -qq update
!apt -qq install gcsfuse
!mkdir co-lab
!gcsfuse co-lab /content/co-lab
# this won't work with "requester pays" bucket, you need gsutil instead
# !gsutil -u "kora-id" cp gs://bucket_name/path_to/image.png .
@LightingMc
Copy link

LightingMc commented May 2, 2020

Thanks for making this tutorial. I need some help. I am trying to mount this dataset called NIH chest Xray to my google collab from google fuse. However, the new "Images" folder created seems to be empty. If I do, "!ls" I get an input/output error.

**
!echo "deb http://packages.cloud.google.com/apt gcsfuse-bionic main" > /etc/apt/sources.list.d/gcsfuse.list
!curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
!apt -qq update
!apt -qq install gcsfuse

!mkdir Images
!gcsfuse gcs-public-data--healthcare-nih-chest-xray Images
**

I changed the last line to

!gcsfuse gcs-public-data--healthcare-nih-chest-xray/content/ Images

and

!gcsfuse gcs-public-data--healthcare-nih-chest-xray/png/ Images

But to no avail. Can you help? My understanding was that the names of images/folders in this dataset would appear in the new directory.

@aaronjoseph
Copy link

I am facing the same issue. Were you able to debug this ?

@LightingMc
Copy link

LightingMc commented Sep 5, 2021 via email

@aaronjoseph
Copy link

Yeah, also, it doesn't make sense to mount GCS bucket. Latency is a big minus.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment