Auto Mount GCS in GCE

AutoMountGCSinGCE.md

Auto mount Google Cloud Storage in a Google Compute machine

The documentation covering this is pretty inartful when it comes to giving examples. Clearly the work of a slave, indentured servant or hostage.

But read it anyway to see if you agree:

Cloud Storage FUSE

Under normal circumstances the gcsfuse utility cannot mount GCS storage as root. For security reasons it will only work out of the box with a non-root user. What we want to do is get it to mount GCS storage on reboot, which necessarily is going to be done by root. But gcsfuse will take parameters specifying the user and group to use, as well as to allow a user not registered with GCS to mount. An added, but essential, bonus is that gcsfuse also lets you specify the file and directory permissions to apply.

NOTE: If you need the web server user (like "www-data") to be able to write to your cloud storage, then you should put that user's uid and gid in the parameter string (see example below).

In the example that follows the GCS bucket is called "www.example.com" (because it makes sense to name a bucket created to serve content for a specific site after the site). The local GCE login user is "myuser", and the mount point for the GCS storage will be "/d1/www/html".

Here are the steps:

1. Create a GCS bucket in same project as the GCE instances you want to share it with.

2. Grant the GCE service account (e.g., [email protected]) for the project "Storage Object Admin" rights over the bucket in the Cloud Storage Browser.

3. Install gcsfuse in each instance.

4. Create a directory on your virtual host writable by your GCE login user without their sudo rights (manual mounting as root will not work) like this:

$ sudo mkdir -p /d1/www/html
$ chown myuser:myuser /d1/www/html

5. Test mounting a bucket to that directory by your GCE login user:

myuser@instance-1:~$ gcsfuse www.example.com /d1/www/html

6. Unmount the bucket ("$ umount /d1/www/html")

7. Re-permission the mount directory so it is owned and writable by www-data:

$ sudo chown www-data:www-data /d1/www/html
$ sudo chmod g+rw /d1/www/html

8. Edit /etc/fstab in instance to add the following line:

www.example.com /d1/www/html gcsfuse rw,allow_other,uid=33,gid=33,file_mode=664,dir_mode=775

In order from left to right:

Name of Bucket - www.example.com
Path for mount - /d1/www/html
gcsfuse - type of filesystem
Parameters
  rw - read/write
  allow_other - allow mounting by other than a GCS authenticated user
  uid=33 - uid of specific user, in this case www-data
  gid=33 - gid of specific group, in this case www-data
  file_mod=664 - Read/Write by owning user and group, read by others
  dir_mode=775 - Read/Write/Execute (for cd) by owning user and group, read/execute by others

9. Test by trying to mount as sudo root (mounting entries in /etc/fstab requires root), "sudo mount /d1/www/html".

10. Reboot and enjoy the goodness that is cloud storage disk attached to your instance.

11. Adjust application configurations to point at this new mount as desired (for example, "root /d1/www/html" in your nginx vhost config)

The more refined example below subsitutes a web developer user and group for www-data so that the web server cannot write to the file system. This is how I usually set things up when possible.

www.example.com /d1/www/html gcsfuse rw,allow_other,uid=1010,gid=1010,file_mode=664,dir_mode=775

The user with uid and gid 1010 in this case is "wwwdev", just a normal user without any other special privileges other than having read/write access to /d1/www/html.