Google Cloud Shell: connecting to cloud storage buckets
Posted by jpluimers on 2021/09/28
One of the drawbacks of Google Cloud Shell is that it will delete the home directory after 120 days of not using it: [WayBack] Deletion notice for my Google Cloud Shell home directory – Stack Overflow
This is documented [WayBack] here:
If you do not access Cloud Shell for 120 days, we will delete your home disk. You will receive an email notification before we do so and simply starting a session will prevent its removal.
This only applies to the home directory of your Cloud Shell instance (you may want to store it on Cloud Storage anyway if you want to keep it). Any other Google services you use will be unaffected.
I hardly use the cloud shell, as it is a last resort to shell out from overly protected networks. Fewer and fewer environments restrict so much, so I’ve bumped into the home directory deletion a few times now.
I might use it more in the future, as I recently discovered there is a URL trick so you can start a cloud shell with parameters like an initial git repository: [WayBack] Open in Cloud Shell | Google Cloud
The Open in Cloud Shell feature allows you to publish a link that opens the Cloud Console and either automatically clones a Git repository into Cloud Shell or starts Cloud Shell with a custom image. It also allows for instructions to be printed to the terminal to help users interact with the content.
The Open in Cloud Shell feature helps developers experiment with code samples and APIs without having to worry about downloading Cloud SDK, installing required dependencies, or searching for relevant source files. This page explains how to add this feature to your Git repository.
Currently, only GitHub and Bitbucket repositories are whitelisted. If you would like to add a different repository, send feedback with the repository type you’d like to use with Open in Cloud Shell.
Setting up the home directory with my scripts can be a curse, so I have contemplated on these kinds of solutions:
- store scripts in Google Drive, and mount part of Google Drive into the Cloud Shell
- store scripts in Google Cloud Storage
- script the setup of the home directory via a bash script in a gist
Some links that will hopefully help me with that:
- [WayBack] Use the Google Cloud Shell to Upload files to Google Drive : DataHoarder
- [WayBack] Open Your Repository In Google Cloud Shell – Daisuke Maki – Medium
- [WayBack] gsutil – Google cloud storage – Download file from web – Stack Overflow
mount the bucket in your Cloud Shell using gcsfuse:
Create a directory in your Cloud Shell user home
mkdir ~/mybucketNow mount your bucket in that directory using gcsfuse:
gcsfuse bucket_name ~/mybucketChange the current directory to mount point directory:
cd mybucket(if you want to have some fun run “df -h .” to see how much space you got in that mount point)
…
seems to work for all file sizes:
curl http://speedtest.tele2.net/10GB.zip | gsutil cp - gs://YOUR_BUCKET_NAME/10GB.zipBasically curl “streams” the data directly to the bucket.
[WayBack] Limitations and restrictions | Cloud Shell | Google Cloud
Usage limits
Cloud Shell inactivity: If you do not access Cloud Shell for 120 days, your home disk will be deleted. You will receive an email notification before its deletion and simply starting a session will prevent its removal. Please consider a different solution on Google Cloud storage for sensitive data you wish to store long term.
Non-interactive usage: Cloud Shell is intended for interactive use only. Non-interactive sessions will be ended automatically after a warning. Note that Cloud Shell sessions are capped at 12 hours, after which sessions are automatically terminated. You can use a new session immediately after.
Weekly usage: Cloud Shell also has weekly usage limits. If you reach your usage limit, you’ll need to wait until the specified time (listed under Usage Quota, found under the three dots menu icon) before you can use Cloud Shell again.
Restoring a session after a service limit violation: If your session is terminated or cannot be established because you exceeded a service limit, Cloud Shell will display an error with a link to a form that allows you to appeal the limit violation. Click the feedback link and submit the form with more information about the tasks you were performing before your session was terminated.
–jeroen
The Wiert Corner - irregular stream of stuff 
Leave a Reply