| layout | toc | permalink | title | see-also | id | description |
|---|---|---|---|---|---|---|
page |
true |
cloud.html |
Globus and the Cloud |
true |
cloud |
Stanford has licensed various cloud add-ons for Globus, described here. |
Here is a list of what is (and is not) supported. Click on a cloud service's name for more information.
| Cloud Service | Available Today? |
|---|---|
| Amazon S3 | ✅ |
| S3-compatible storage | ✅ |
| Medicine Box | ❌ |
| Ceph | ❌ |
| Dropbox | ❌ |
| Google Cloud Storage | ✅ |
| Google Drive | ✅ |
| iRODS | ❌ |
| Microsoft Azure Blob Storage | ❌ |
| Microsoft OneDrive | ✅ |
| Wasabi | ✅ |
{% capture collection-under-construction %}
This collection is still under construction, and is likely going to change in the future. If you run into problems accessing it, contact us for support.
{% endcapture %}Amazon S3 buckets—in all regions other than China—may be accessed through the Stanford Amazon (AWS) S3 collection.
If you need to access a requester-pays bucket, you may do so through the Stanford Amazon (AWS) S3 Requester-Pays collection.
[Go here for detailed information on how to use Globus with Amazon S3]({{ "cloud/s3.html" | absolute_url }}).
Using Globus with Amazon S3 requires an IAM User with appropriate permissions to list buckets, list objects within buckets, download objects, (optionally) upload objects (which includes managing multi-part uploads), and (optionally) delete objects. Globus' documentation has an example IAM Policy, or [see the step-by-step instructions]({{ "cloud/s3.html" | absolute_url }}).
Globus for Amazon S3 supports working with buckets in other accounts (cross-account access), as long as access is granted directly to your IAM User. Access via IAM Roles is not supported. Both you and the account owner must give your IAM User access to the bucket.
Globus does not support Requester-Pays buckets, but this feature has been requested. An announcement will be posted when this feature is available.
Globus also does not support custom metadata/tags, ACLs, and additional checksum algorithms. Only the latest version of objects will be accessed. Uploaded objects will be placed into the S3 Standard storage class.
The Globus S3 connector supports more than just Amazon S3. If you run a storage platform that supports the S3 API, the Globus-for-S3 add-on may support it.
As of this page's last update, besides Amazon S3, we are aware of the Globus S3 connector being used with Dell EMC ECS on-prem, and with Wasabi (as mentioned later on this page).
If you would like to use the Globus S3 connector with your own storage platform, you may do so! You will need to set up a Globus Connect Server v5 endpoint, along with an S3 storage gateway and at least one mapped collection.
For compability across multiple products, the Globus S3 connector uses a limited subset of the S3 REST API. Your storage platform will need to support the following aspects and operations:
-
For authentication, v4 signatures are used with Access Keys and Secret Keys.
-
Globus uses path-style for bucket access, instead of virtual-host style.
-
Globus makes a
GetBucketLocationcall before anything else, to identify which Amazon S3 region's endpoint should be used. As long as your storage platform returns something (instead of an error), Globus will ignore responses it cannot parse. If your platform has multiple instances or endpoints, you may need a separate storage gateway for each. -
The
ListObjectsandGetObjectcalls are used for directory listings and downloads, respectively. Globus follows the convention of using the forward-slash character as a 'directory' separator. -
Uploading objects use the following operations:
PutObject,ListMultipartUploads,ListParts,CreateMultipartUpload,CompleteMultipartUpload,UploadPart, andAbortMultipartUpload. -
Deleting objects uses the
DeleteObjectoperation.
Unfortunately, SRCC are unable to host or administer a Globus endpoint for your storage connector, but you may still contact us for general advice.
Although Stanford's subscription does support the Box connector, University IT does not provide Box storage as a service.
The School of Medicine does offer a Box instance—known as "Medicine Box"—but Stanford's Globus subscription has not gone through the necessary Security & Privacy reviews for access to High Risk data, so access to Medicine Box is blocked.
{% include info-box.html header="Object Storage Only" content="The Ceph add-on for Globus only supports Ceph object storage. If you want to access Ceph file storage (CephFS), you should use a normal Globus (POSIX) collection." %}
Although Stanford's subscription does support the Ceph (object storage) connector, University IT does not provide Ceph storage as a service.
If you run your own Ceph service, you may be able to [run your own Globus endpoint]({{ "server5.html" | relative_url }}) with the Ceph connector. Contact us if you are interested in this.
Although Stanford's subscription does support the Dropbox connector, University IT does not provide Dropbox storage as a service.
Some groups within the University do use Dropbox. If you use Dropbox, you may be able to [run your own Globus endpoint]({{ "server5.html" | relative_url }}) with the Dropbox connector. Contact us if you are interested in this.
{% capture google-account-required %}
To access Google Cloud Storage, you need a Stanford Google account. If you have Stanford email, you have a Stanford Google account. If you do not have Stanford email, you & your sponsor should read about Stanford Google Workspace ID for Base SUNet IDs.
{% endcapture %} {% comment %} Capture is in the Google Cloud Storage section, above {% endcomment %} {% include info-box.html icon="id-card" raw=google-account-required %}{% include info-box.html icon="biohazard" header="No High Risk Access allowed" content="Some Google Cloud accounts are allowed to store High Risk data (including PHI). Such accounts are not allowed to use Globus." %}
Google Cloud Storage may be accessed through the Stanford Google Cloud collection.
[Go here for detailed information on how to use Globus with Google Cloud Storage]({{ "cloud/cloud.html" | absolute_url }}).
When using Google Cloud Storage, Globus will act on your behalf. You will be able to access any buckets outside of the VPC Service Perimeter. In other words, you will not be able to access buckets associated with Nero projects, and other High Risk projects.
{% capture drive-limits %}
Starting September 1, 2024: Google Drive spaces (including your 'My Drive' space and all Shared Drives) will have a 50 GB quota. Read the transition announcement.
{% endcapture %} {% include info-box.html icon="compress-arrows-alt" raw=drive-limits %}{% include info-box.html icon="id-card" header="Full SUNetID Required" content="To interact with Google Drive through Stanford, you must have a full or full-sponsored SUNetID. Therefore, using the Globus Google connector also requires a full (or full-sponsored) SUNetID." %}
{% include info-box.html icon="biohazard" header="No High Risk Access allowed" content="Some users are able to able to store High Risk data (including PHI) in Google Drive. If you are part of this group, you are not allowed to use Globus to access that data." %}
Google Drive may be accessed through the Stanford Google Drive collection.
[Go here for detailed information on how to use Globus with Google Drive]({{ "cloud/drive.html" | absolute_url }}).
When using Globus with Google Drive, Globus will act on your behalf. You will be able to access files on your own Drive, on Shared Drives (formerly known as Team Drives), and to folders & items Shared With You.
Globus does not support setting or copying custom permissions. Uploaded items will inherit the permissions of the parent folder.
Globus is also not able to copy Drive Shortcuts. Drive Shortcuts are similar to shortcuts on Windows or aliases on macOS. At this time, Globus is not able to copy or follow Shortcuts; attempting to copy a Shortcut will create an empty file at the destination, and generate a checksum (integrity-checking) error. Support for Shortcuts is currently on Globus' backlog for implementation.
Finally, Globus is not able to copy 'files' which are created by Google products, such as Docs, Sheets, Slides, etc.. Google does not actually store data for Google products in Drive; instead, Drive holds a 'pointer' to the specific Google product. Attempting to copy these 'files' will generate an error.
Although Stanford's subscription does support the iRODS connector, University IT does not provide iRODS storage as a service.
If you run your own iRODS installation, you may be able to [run your own Globus endpoint]({{ "server5.html" | relative_url }}) with the iRODS connector. Contact us if you are interested in this.
Stanford's subscription supports the Microsoft Azure Blob Storage connector.
Microsoft Azure Blob Storage has unique properties that make it different to Amazon S3 and Google Cloud Storage. For that reason, before we provide an Azure Blob Storage Globus service, we would like to hear from existing Azure Blob Storage users.
If you use Microsoft Azure Blob Storage within the University, and you are interested in accessing it through Globus, please reach out!
{% include info-box.html icon="id-card" header="Full SUNetID Required" content="To interact with Microsoft services through Stanford, you must have a full or full-sponsored SUNetID. Therefore, using the Globus OneDrive connector also requires a full (or full-sponsored) SUNetID." %}
{% include info-box.html icon="biohazard" header="No High Risk Access allowed" content="Some users are able to able to store High Risk data (including PHI) in Microsoft OneDrive. If you are part of this group, you are not allowed to use Globus to access that data." %}
Microsoft OneDrive may be accessed through the Stanford OneDrive collection.
[Go here for detailed information on how to use Globus with Microsoft OneDrive]({{ "cloud/onedrive.html" | absolute_url }}).
When using Globus with Microsoft OneDrive, Globus will act on your behalf. You will be able to access files on your own OneDrive, files shared with you by others in Stanford University, and files in Stanford University SharePoint sites.
Globus does not support setting or copying custom permissions. Uploaded items will inherit the permissions of the parent folder.
Globus is not able to access non-University content. That includes things shared by folks outside of Stanford University, and it also includes things shared by members of Stanford Health Care and Stanford Children's Health.
{% include info-box.html icon="wrench" raw=collection-under-construction %}
{% include info-box.html icon="biohazard" header="No High Risk Access allowed" content="Some Wasabi accounts are allowed to store High Risk data (including PHI). Such accounts are not allowed to use Globus." %}
Access to Wasabi is possible throguh region-specific collections:
- us-west-1: SRCC Oak Wasabi S3
You must use the collection specific to your Wasabi region. Wasabi accounts created through Stanford University IT are placed into the us-west-1 region by default. If you need access to a different region, let us know.