Metadata-Version: 1.2
Name: lor
Version: 0.0.2
Summary: A framework for rapidly developing pipelines with Luigi
Home-page: UNKNOWN
Author: Adam Kewley
Author-email: contact@adamkewley.com
License: Apache License 2.0
Description-Content-Type: UNKNOWN
Description: Luigi on Rails
        ==============
        
        |Build Status|
        
        ALPHA BUILD: A framework for rapidly developing pipelines with Luigi
        
        [`documentation <https://adamkewley.github.io/lor/>`_]
        
        Features:
        
        - Convention-over-configuration approach to Luigi projects
        - Standard workspace creation (``lor new``)
        - Configuration loading
        - A workspace-centric command-line (``new``, ``explain``, ``ls``, ``run``, etc.)
        - Utility tasks (e.g. ``EnsureExistsOnLocalFilesystemTask``)
        - Other utilities for managing long-running subprocesses, building a CLI, etc.
        
        
        Installation
        ------------
        
        To install use pip:
        
            $ pip install lor
        
        
        Or clone the repo:
        
            $ git clone https://github.com/adamkewley/lor.git
        
            $ python setup.py install
        
        
        Usage
        -----
        
        Once installed, ``lor`` can be used from the command-line:
        
            $ lor new foo
        
        This creates a ``lor`` workspace at ``foo/``. A ``lor`` workspace is a standard python3 pip project which depends on
        ``lor`` as a dependency (this can be removed if you don't need it). The ``lor`` command-line command can detect if the
        terminal is in a ``lor`` workspace and provides convenience commands for working in a standard workspace.
        
        Tasks written for ``lor`` are just standard Luigi tasks. The ``lor`` library provides various utility methods that a
        typical project *might* find useful (all opt-in). For example, ``lor`` provides a ``props`` module that you can use to
        load configuration properties at runtime:
        
        .. code:: python
        
            # foo/tasks/bar
        
            import lor.props
            import luigi
        
        
            class BarTask(luigi.Task):
                output_path = luigi.Parameter()
        
                def run(self):
                    with open(str(self.output_path), "w") as f:
                        config_prop = lor.props.get("CONFIG_PROP")
                        f.write(config_prop)
        
                def output(self):
                    return luigi.LocalTarget(self.output_path)
        
        The above is a standard Luigi task that uses ``lor`` to load a configuration property called ``CONFIG_PROP`` from the
        workspace (held at ``etc/properties.yml`` in the workspace). The task is a standard Luigi task, so it can be ran from
        Luigi directly:
        
            $ luigi --module foo.tasks.bar BarTask --output-path some/path
        
        The ``lor run`` command can also be used to run the task:
        
           $ lor run --module foo.tasks.bar BarTask --output-path some/path
        
        The ``lor run`` command runs ``luigi`` by proxy, so it effectively has the same interface. However, ``lor run`` also
        adds useful functionality, such as the ability to override a variable's value at runtime:
        
           $ lor run --properties CONFIG_PROP=overridden --module foo.tasks.bar BarTask --output-path some/path
        
        The task will then write "overridden" to the output file instead of whatever was loaded from the workspace's configuration
        file. This is because ``lor run`` bootstraps the workspace global with the override before running Luigi.
        
        TODO: This documentation is work in progress
        
        
        .. |Build Status| image:: https://travis-ci.org/adamkewley/lor.svg?branch=master
           :target: https://travis-ci.org/adamkewley/lor
        
Keywords: Luigi
Platform: UNKNOWN
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.4
