0

I am getting an ImportError when I try to access a library inside the lib directory in my application root.

I did the necessary steps in the documentation and it worked with only one module but once I moved to this directory structure with multiple modules it no longer works:

<app root>
  |
  +-- appengine_config.py
  |
  +-- lib
  |
  +-- modules     
      +-- module1
      |
      +-- module2

In my appengine_config.py I add the line: vendor.add(os.path.join(os.path.dirname(__file__), 'lib'))

Any thoughts on what might be going on here?

Edit: This directory structure that seemed to work for me:

<app root>
  |
  +-- lib
  |     
  +-- module1
  |     +-- appengine_config.py
  |     |
  |     +-- app.yaml
  |     |
  |     +-- lib <symlink to app root lib>
  +-- module2 (same as module 1)

I thought that appengine_config.py was something to be defined at the main application level and not the module level. Is it correct to duplicate the same exact file and sym linking in each module.

user5831382
  • 15
  • 5
  • Where are your modules' `.yaml` files located? I mean did you follow the 1st or the 2nd app structure suggestion in the doc: https://cloud.google.com/appengine/docs/python/modules/#Python_Configuration – Dan Cornilescu Jan 24 '16 at 13:34
  • The top level yaml files like `queue.yaml`, `cron.yaml`, etc are in the app root directory while the `app.yaml` files are in `modules/module1`, `modules/module2` directories. I followed mostly the first app structure but nested all my modules inside the module directory. Sorry for the naming confusion as it may get mixed up with your standard Python modules. – user5831382 Jan 24 '16 at 16:54

2 Answers2

0

Why are you including os.path.dirname(file) in vendor.add ?

Look at the docs. https://cloud.google.com/appengine/docs/python/tools/libraries27?hl=en#vendoring

vendor.add('lib') or at worst vendor.add('lib/modules') is all you need.

Is modules really a module/package ?

Tim Hoffman
  • 12,908
  • 1
  • 15
  • 28
  • I had `vendor.add('lib')` originally but it didn't work either so I just tried the tip mentioned in the [docs](https://cloud.google.com/appengine/docs/python/tools/libraries27?hl=en#vendoring). `modules` is the directory where I kept all my modules. Wanted to organize all my different modules in one place. – user5831382 Jan 24 '16 at 16:51
  • I've also just tried moving all my app engine modules down a level from inside the modules directory to just inside the app root. I am testing with `dev_appserver.py` btw. – user5831382 Jan 24 '16 at 17:08
  • It actually seems like `appengine_config.py` isn't being run at all when it's in the app root. When I move it to inside module1 and create a symlink in module1 to point to the main lib directory it seems to actually run the file. Edited my original question for something that seems to help. – user5831382 Jan 24 '16 at 17:24
0

Each module directory is uploaded as a standalone application and it needs all its files inside that module directory (i.e. nothing from the module's parent directory is uploaded/visible in the module). The module directory is the one containing the module's .yaml file, so in your case they'll be the modules/module1 and modules/module2 dirs.

So symlink the lib dir and the appengine_config.py from the app's dir into the modules' dirs. In addition, for my app, as Tim mentions, the vendoring line that works is indeed just vendor.add('lib'). Copying them works just as well, but symlinking is better as you only need to maintain one copy in your version control system, in the DRY spirit. The appcfg.py script knows to follow the symlinks and upload the actuall files the symlinks point to.

If you don't need all but only some of the libs in a module you can create a lib dir in that module dir and only symlink the necessary subdirs of the app's lib dir into the respective module's lib dir (to not upload unnecessary files).

You might need to also symlink into some of the module dirs other app-level config files to keep some of the supporting scripts happy on the development server, see this answer: https://stackoverflow.com/a/34111170/4495081. Note that updating the app-level configs may not happen simply by uploading the modules' code as mentioned in the single-module app tutorials, they'll need to be requested explicitly with the corresponding arguments of appcfg.py, as mentioned in the cheat sheet in that answer.

Community
  • 1
  • 1
Dan Cornilescu
  • 37,297
  • 11
  • 54
  • 89