Prevent Public Bucket Access in GCP

Unless you're specifically trying to give your data away (perhaps for a website or something), you absolutely should always prevent public access to your cloud storage buckets. For whatever reason, this never seems to be the default, even though it's really pretty important.

In Google Cloud Platform (GCP), they've made this surprisingly complex. There are no simple checkboxes that make a bucket private, but you can set it at an organisation level so that all of your buckets (old and new) confirm to your policy.

Organisation Policies

An organisation policy can be set at a "level" and all projects below that level will inherit the policy unless specifically set otherwise. As such, if you prevent public access at the "root" of your organisation, then you've made every bucket you've ever created private. That's probably the best approach for a lot of shops.

If you need to make an exception, then you have to do it at the project level. You can make a particular project an exception to your policy, which then makes all of its buckets (potentially) public. A bucket without special handling will have some default ACLs on its objects, which hopefully are set to private, but may not be (and it's possible individual files are set to public). To lock down the buckets in a project you're making into an exception, you need to lock down each one of them - and remember not to create any more without locking them down too.

Organisation Policy IAMs and Access

Google don't make fiddling with policies easy. There's a good reason for this, but it does mean you've got some hoops to jump through before you can prevent public access to your buckets. This seems a little counter-productive to us, but it's what is provided, so we have to play along.

If you want to enforce a policy using the web console, then your user must have some specific permissions. Having the "Organization Policy Administrator" role isn't sufficient (what!?) - you addtionally need to have the "Access Transparency Admin" role. If you think all of that is counter-intuitive and confusing, then you're not alone.

If you're more inclined to use Terraform to manage your GCP setup (and here at Pre-Emptive, we definitely think everyone should do this), then your task is possibly even harder - you can't use the same user that you use for the browser to do this. If you try, you'll get this error:

Error: Error creating Policy: failed to create a diff: failed to retrieve Policy resource: googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the orgpolicy.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/. If you are getting this error with curl or similar tools, you may need to specify 'X-Goog-User-Project' HTTP header for quota and billing purposes. For more information regarding 'X-Goog-User-Project' header, please check https://cloud.google.com/apis/docs/system-parameters.

No amount of IAM permissions will ever work. Instead, you've got to use a Service Account user, or possibly do some sort of hacking with the gcloud tool. With these hoops to jump through, you might be tempted to just use the mouse in the web UI - on this occasion, we'll pretend we didn't see and won't tell anyone ;-)

If you do persist and use Terraform, something like this will do it for you:

resource "google_org_policy_policy" "primary" {
  name   = "organizations/${var.org_id}/policies/constraints/storage.publicAccessPrevention"
  parent = "organizations/${var.org_id}"

  spec {
    reset = true
  }
}

The variable org_id here should be set to the numeric organisation ID to set the policy in the root. A variation of the above can be used with folder names in place of the org_id.

Pre-Emptive can help with cloud storage provision and security, or pretty much any cloud problems you may have. Contact us to see how we can help you.