Issue

I use a web browser to access Google Cloud Storage. But some storages require login to access. When downloading or accessing these storages with gsutil, there will be an error: ServiceException: 401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket.

$ gsutil -m cp -r "gs://waymo_open_dataset_v_1_2_0/" .
ServiceException: 401 Anonymous caller does not have storage.objects.list access to the Google Cloud Storage bucket.
CommandException: 1 file/object could not be transferred.

Solution

The gsutil config command obtains access credentials for Cloud Storage and writes a boto/gsutil configuration file containing the obtained credentials along with a number of other configuration-controllable values. Refer to gsutil config Document.

Run as follows:

$ gsutil config
This command will create a boto config file at
C:\Users\Administrator\.boto containing your credentials, based on
your responses to the following questions.
Please navigate your browser to the following URL:
https://accounts.google.com/o/oauth2/auth?client_id=912341233072.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&access_type=offline&response_type=code
In your browser you should see a page that requests you to authorize access to Google Cloud Platform APIs and Services on your behalf. After you approve, an authorization code will be displayed.

Enter the authorization code: 4/1AX4XfWjTqe2CzuheTvIJ-Ibq123RHl7-kp2XygdJoXDz5BZbFjaCkLgflg

Please navigate your browser to https://cloud.google.com/console#/project,
then find the project you will use, and copy the Project ID string from the
second column. Older projects do not have Project ID strings. For such projects,
 click the project and then copy the Project Number listed under that project.

What is your project-id? quickstart-1123408928142

gsutil developers rely on user feedback to make improvements to the
tool. Would you like to send anonymous usage statistics to help
improve gsutil? [y/N] y

Boto config file "C:\Users\Administrator\.boto" created. If you need
to use a proxy to access the Internet please see the instructions in
that file.

Follow the prompt of the command, enter the authorization code and project-id.

After the configuration is complete, you can copy and other operations. For example, to quickly get the total size:

$ gsutil du -hs "gs://waymo_open_dataset_v_1_2_0/"
1.82 TiB     gs://waymo_open_dataset_v_1_2_0

gsutil Error: serviceexception 401 anonymous caller does not have storage objects list

Issue

Solution