Google Cloud Storage Signed URLs with Google App Engine

Question

It's frustrating to deal with the regular Signed URLs (Query String Authentication) for Google Cloud Storage.

Google Cloud Storage Signed URLs Example -> Is this really the only code available in the whole internet for generating Signed URLs for Google Cloud Storage? Should I read it all and adapt it manually for Pure Python GAE if needed?

It's ridiculous when you compare it with AWS S3 getAuthenticatedURL(), already included in any SDK...

Am I missing something obvious or does everyone face the same problem? What's the deal?

Answer 1

Here's how to do it in Go:

func GenerateSignedURLs(c appengine.Context, host, resource string, expiry time.Time, httpVerb, contentMD5, contentType string) (string, error) {
    sa, err := appengine.ServiceAccount(c)
    if err != nil {
        return "", err
    }
    expUnix := expiry.Unix()
    expStr := strconv.FormatInt(expUnix, 10)
    sl := []string{
        httpVerb,
        contentMD5,
        contentType,
        expStr,
        resource,
    }
    unsigned := strings.Join(sl, "\n")
    _, b, err := appengine.SignBytes(c, []byte(unsigned))
    if err != nil {
        return "", err
    }
    sig := base64.StdEncoding.EncodeToString(b)
    p := url.Values{
        "GoogleAccessId": {sa},
        "Expires": {expStr},
        "Signature": {sig},
    }
    return fmt.Sprintf("%s%s?%s", host, resource, p.Encode()), err
}

Answer 2

I came across this problem recently as well and found a solution to do this in python within GAE using the built-in service account. Use the sign_blob() function in the google.appengine.api.app_identity package to sign the signature string and use get_service_account_name() in the same package to get the the value for GoogleAccessId.

Don't know why this is so poorly documented, even knowing now that this works I can't find any hint using Google search that it should be possible to use the built-in account for this purpose. Very nice that it works though!

Answer 3

I have no idea why the docs are so bad. The only other comprehensive answer on SO is great but tedious.

Enter the generate_signed_url method. Crawling down the rabbit hole you will notice that the code path when using this method is the same as the solution in the above SO post when executed on GAE. This method however is less tedious, has support for other environments, and has better error messages.

In code:

def sign_url(obj, expires_after_seconds=60):

    client = storage.Client()
    default_bucket = '%s.appspot.com' % app_identity.get_application_id()
    bucket = client.get_bucket(default_bucket)
    blob = storage.Blob(obj, bucket)

    expiration_time = int(time.time() + expires_after_seconds)

    url = blob.generate_signed_url(expiration_time)

    return url

Answer 4

Check out https://github.com/GoogleCloudPlatform/gcloud-python/pull/56

In Python, this does...

import base64
import time
import urllib
from datetime import datetime, timedelta

from Crypto.Hash import SHA256
from Crypto.PublicKey import RSA
from Crypto.Signature import PKCS1_v1_5
from OpenSSL import crypto

method = 'GET'
resource = '/bucket-name/key-name'
content_md5, content_type = None, None

expiration = datetime.utcnow() + timedelta(hours=2)
expiration = int(time.mktime(expiration.timetuple()))

# Generate the string to sign.
signature_string = '\n'.join([
  method,
  content_md5 or '',
  content_type or '',
  str(expiration),
  resource])

# Take our PKCS12 (.p12) key and make it into a RSA key we can use...
private_key = open('/path/to/your-key.p12', 'rb').read()
pkcs12 = crypto.load_pkcs12(private_key, 'notasecret')
pem = crypto.dump_privatekey(crypto.FILETYPE_PEM, pkcs12.get_privatekey())
pem_key = RSA.importKey(pem)

# Sign the string with the RSA key.
signer = PKCS1_v1_5.new(pem_key)
signature_hash = SHA256.new(signature_string)
signature_bytes = signer.sign(signature_hash)
signature = base64.b64encode(signature_bytes)

# Set the right query parameters.
query_params = {'GoogleAccessId': 'your-service-account@googleapis.com',
                'Expires': str(expiration),
                'Signature': signature}

# Return the built URL.
return '{endpoint}{resource}?{querystring}'.format(
    endpoint=self.API_ACCESS_ENDPOINT, resource=resource,
    querystring=urllib.urlencode(query_params))

Answer 5

And if you don't want to write it by your own checkout this class on GitHub.

Really easy to use

GCSSignedUrlGenerator