Metadata-Version: 2.1
Name: seml
Version: 0.5.2
Summary: Slurm Experiment Management Library
Author: DAML Group @ TUM
Author-email: Daniel Zügner <zuegnerd@in.tum.de>, Johannes Gsteiger <johannes.gasteiger@tum.de>, Nicholas Gao <n.gao@tum.de>, Dominik Fuchsgruber <d.fuchsgruber@tum.de>
Maintainer-email: Nicholas Gao <n.gao@tum.de>, Dominik Fuchsgruber <d.fuchsgruber@tum.de>
License: MIT License
        
        Copyright (c) 2023 Johannes Klicpera, Daniel Zügner, Nicholas Gao
        Technical University of Munich
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/TUM-DAML/seml
Project-URL: Documentation, https://github.com/TUM-DAML/seml/blob/master/docs.md
Project-URL: Repository, https://github.com/TUM-DAML/seml
Project-URL: Changelog, https://github.com/TUM-DAML/seml/releases/latest
Classifier: Programming Language :: Python
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.15
Requires-Dist: pymongo>=4.1
Requires-Dist: pandas>=2.2
Requires-Dist: sacred>=0.8.4
Requires-Dist: pyyaml>=5.1
Requires-Dist: jsonpickle>=2.2
Requires-Dist: munch>=4.0
Requires-Dist: debugpy>=1.2.1
Requires-Dist: requests>=2.28.1
Requires-Dist: typer>=0.12
Requires-Dist: rich>=13.0
Requires-Dist: omegaconf>=2.3.0
Requires-Dist: gitignore_parser>=0.1.11
Requires-Dist: setuptools>=69.2.0
Requires-Dist: importlib_resources>=5.7.0
Requires-Dist: typing_extensions>=4.10
Requires-Dist: deepdiff>=7.0.1
Provides-Extra: dev
Requires-Dist: ruff>=0.6.1; extra == "dev"
Requires-Dist: pytest>=8.3.2; extra == "dev"
Requires-Dist: pre-commit>=3.8.0; extra == "dev"
Requires-Dist: filelock>=3.15.4; extra == "dev"
Requires-Dist: sshtunnel>=0.4.0; extra == "dev"
Provides-Extra: ssh-forward
Requires-Dist: sshtunnel>=0.4.0; extra == "ssh-forward"
Requires-Dist: filelock>=3.13.3; extra == "ssh-forward"

![Github Actions](https://github.com/TUM-DAML/seml/workflows/Test/badge.svg)

# `SEML`: Slurm Experiment Management Library
**`SEML`** is the missing link between the open-source workload scheduling system `Slurm`, the experiment management tool `sacred`, and a `MongoDB` experiment database. It is lightweight, hackable, written in pure Python, and scales to thousands of experiments.

Keeping track of computational experiments can be annoying and failure to do so can lead to lost results, duplicate running of the same experiments, and lots of headaches.
While workload scheduling systems such as [`Slurm`](https://slurm.schedmd.com/overview.html) make it easy to run many experiments in parallel on a cluster, it can be hard to keep track of which parameter configurations are running, failed, or completed.
[`sacred`](https://github.com/IDSIA/sacred) is a great tool to collect and manage experiments and their results, especially when used with a [`MongoDB`](https://www.mongodb.com/). However, it is lacking integration with workload schedulers.

**`SEML`** enables you to
* very easily define hyperparameter search spaces using YAML files,
* run these hyperparameter configurations on a compute cluster using `Slurm`,
* and to track the experimental results using `sacred` and `MongoDB`.


In addition, **`SEML`** offers many more features to make your life easier, such as
* automatically saving and loading your source code for reproducibility,
* easy debugging on Slurm or locally,
* automatically checking your experiment configurations,
* extending Slurm with local workers,
* and keeping track of resource usage (experiment runtime, RAM, etc.).

## Get started
To get started, install **`SEML`** either via `pip`:
```bash
pip install seml
```
Then configure your MongoDB via:
```bash
seml configure
```
For convenience, you may create your first **`SEML`** project via:
```bash
# initialize the default template in the 'new_project' folder.
seml project init -t default new_project
```
in an empty directoy. **`SEML`** will automatically create a python package for you.


### SSH Port Forwarding
If your MongoDB is only accessible via an SSH port forward, **`SEML`** allows you to directly configure this as well if you install the `ssh_forward` dependencies via:
```bash
pip install seml[ssh_forward]
```
It remains to configure the SSH settings:
```bash
seml configure --ssh_forward
```

### Development
For development, we recommend [`uv`](https://docs.astral.sh/uv/) which you can install via
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
```
Setup the right environment use and activate it:
```bash
uv sync --locked
source .venv/bin/activate
```
Alternatively, you can install the repository in any Python environment via:
```bash
pip install -e .[dev]
```

#### Pre-commit hooks
Make sure to install the pre-commit hooks via
```bash
pre-commit install
```

## Documentation
Documentation is available in our [docs.md](docs.md) or via the CLI:
```python
seml --help
```

## Example
See our simple [example](examples) to get familiar with how **`SEML`** works.

## CLI completion
SEML supports command line completion. To install this feature run:
```bash
seml --install-completion {shell}
```

If you are using the zsh shell, you might have to append `compinit -D` to the `~/.zshrc` file (see this [issue](https://github.com/tiangolo/typer/issues/180#issuecomment-812620805)).

## Slurm version

SEML should work with Slurm 18.08 and above out of the box. Version 17.11 and earlier do not have a SIGNALING job state, which you have to remove from the SLURM_STATES defined in SEML's settings (`seml/settings.py`). Earlier versions have not been tested and might have other issues.

## Contact
Contact us at zuegnerd@in.tum.de, johannes.gasteiger@tum.de, or n.gao@tum.de for any questions.

## Cite
When you use SEML in your own work, please cite the software along the lines of the following bibtex:

```
@software{seml_2023,
  author = {Z{\"u}gner, Daniel and Gasteiger, Johannes and Gao, Nicholas and Dominik Fuchsgruber},
  title = {{SEML: Slurm Experiment Management Library}},
  url = {https://github.com/TUM-DAML/seml},
  version = {0.4.0},
  year = {2023}
}
```


Copyright (C) 2023
Daniel Zügner, Johannes Gasteiger, Nicholas Gao, Dominik Fuchsgruber
Technical University of Munich
