Luigi is a Python package that helps you build complex pipelines of batch jobs.
Luigi is a Python package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.
For further information, see the documentation at luigi.readthedocs.io.
Getting Luigi
Run pip install luigi to install the latest stable version from PyPI.
For bleeding edge code, git clone https://github.com/spotify/luigi and python setup.py install. Bleeding edge documentation can be found here.
If you want to run the central scheduler (highly recommended), you need to install Tornado which you can do from PyPI as well: pip install tornado.
