-1

I read this page which explains how to access the google storage from python.

https://cloud.google.com/appengine/docs/flexible/python/using-cloud-storage

I don't want a marriage between my code and the cloud storage provider.

If I use amazon, google or a custom storage should be configuration and should be handled outside my source code.

Is there a way to write python code which uses a cloud storage, but does this in a implementation independent way?

guettli
  • 26,461
  • 53
  • 224
  • 476

1 Answers1

1

Unfortunately today it's not possible to make your code completely independent of the type of cloud storage, primarily because each cloud storage provider uses different kinds of configurations, authentications, access libraries, etc.

But what you can do is contain such dependencies in thin layers specific to each provider, hidden behind a common, more generic API you'd create.

The vast majority of your code would use this generic API and would not have to change when you add, delete or change the actual storage provider.

Dan Cornilescu
  • 37,297
  • 11
  • 54
  • 89
  • Thank you for your answer. AFAIK storage APIs are very simple: CRUD (Create, Read, Update, Delete). Yes, I could create a thin layer to keep my code clean. I am unsure if "not possible" is really true. For relational databases it's possible. And if you only do basic SQL your code is portable and the configuration does define the concrete connection. – guettli Nov 21 '16 at 18:55
  • Well, your question mentioned storage, not SQL (I admit, fundamentally they can be seen as the same thing, but at implementation level they're not). Taking GCS for example, you **have to** use a GCS-specific library's `open()` to open a file, you can't just call python's `open()` with a filename. The reverse is also true: you can't use GCS's `open()` with a regular filesystem file, let alone amazon or some other non-GCS cloud storage, an adaptation layer would be required to accomodate such differences. This is where my "not possible" comes from. – Dan Cornilescu Nov 21 '16 at 19:19
  • Yes, this question is about storage APIs. I was just comparing it to SQL. I guess there will be implementation independent APIs in two to five years. – guettli Nov 21 '16 at 20:43
  • Indeed, future standardized cloud storage access is a distinct possibility :) – Dan Cornilescu Nov 21 '16 at 20:56