Metadata-Version: 2.1
Name: experimaestro
Version: 0.7.3
Summary: "Experimaestro is a computer science experiment manager"
Home-page: https://github.com/experimaestro/experimaestro-python
Author: Benjamin Piwowarski
Author-email: benjamin@piwowarski.fr
License: GPL-3
Keywords: experiment manager
Platform: any
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Requires-Dist: click
Requires-Dist: fasteners
Requires-Dist: psutil
Requires-Dist: websockets
Requires-Dist: jsonstreams
Requires-Dist: pytools
Requires-Dist: watchdog
Requires-Dist: marshmallow
Requires-Dist: fabric
Requires-Dist: rpyc
Provides-Extra: dev
Requires-Dist: docutils ; extra == 'dev'
Requires-Dist: Pygments ; extra == 'dev'
Provides-Extra: ssh
Requires-Dist: paramiko ; extra == 'ssh'
Provides-Extra: test
Requires-Dist: tox ; extra == 'test'

[![PyPI version](https://badge.fury.io/py/experimaestro.svg)](https://badge.fury.io/py/experimaestro)

Experimaestro is a computer science experiment manager whose goals are:

* To decompose experiments into a set of parameterizable tasks
* Schedule tasks and handle dependencies between tasks
* Avoids to re-run the same task two times by computing unique task IDs dependending on the parameters
* Handle experimental parameters through tags

Full documentation can be found in [https://experimaestro.github.io/experimaestro-python/](https://experimaestro.github.io/experimaestro-python/)

# Install

## With pip

You can then install the package using `pip install experimaestro`

## Develop

Checkout the git directory, then

```
pip install -e .
```

# Example

This very simple example shows how to submit two tasks that concatenate two strings.
Under the curtain, 

- A directory is created for each task (in `workdir/jobs/helloworld.add/HASHID`)
  based on a unique ID computed from the parameters
- Two processes for `Say` are launched (there are no dependencies, so they will be run in parallel)
- A tag `y` is created for the main task

<!-- SNIPPET: MAIN ARGS[%WORKDIR%] ENV[XPM_EXAMPLE_WAIT=0.001] -->
```python
# --- Task and types definitions

import logging
from pathlib import Path
from experimaestro import *
import click
import time
import os
from typing import List

# --- Just to be able to monitor the tasks

def slowdown(N: int):
    for i in range(N):
        time.sleep(float(os.environ.get("XPM_EXAMPLE_WAIT", 2)))
        progress((i+1)/N)


# --- Define the tasks

hw = Identifier("helloworld")

@argument("word", type=str, required=True, help="Word to generate")
@task(hw.say)
def Say(word: str):
    slowdown(len(word))
    print(word.upper(),)

@argument("strings", type=List[Say], help="Strings to concat")
@task(hw.concat)
def Concat(strings: List[Say]):
    # We access the file where standard output was stored
    says = []
    slowdown(len(strings))
    for string in strings:
        with open(string.stdout()) as fp:
            says.append(fp.read().strip())
    print(" ".join(says))


# --- Defines the experiment

@click.option("--debug", is_flag=True, help="Print debug information")
@click.option("--port", type=int, default=12345, help="Port for monitoring")
@click.argument("workdir", type=Path)
@click.command()
def cli(port, workdir, debug):
    """Runs an experiment"""
    logging.getLogger().setLevel(logging.DEBUG if debug else logging.INFO)

    # Sets the working directory and the name of the xp
    with experiment(workdir, "helloworld", port=12345) as xp:
        # Submit the tasks
        hello = Say(word="hello").submit()
        world = Say(word="world").submit()

        # Concat will depend on the two first tasks
        Concat(strings=[hello, world]).tag("y", 1).submit()


if __name__ == "__main__":
    cli()
```

which can be launched with `python test.py /tmp/helloworld-workdir`


