Metadata-Version: 2.4
Name: deephyper
Version: 0.10.1
Summary: Massively Parallel Hyperparameter Optimization for Machine Learning
Project-URL: Documentation, http://deephyper.readthedocs.io
Project-URL: Changes, https://github.com/deephyper/deephyper/releases
Project-URL: Forum, https://github.com/deephyper/deephyper/discussions
Project-URL: GitHub, https://github.com/deephyper/deephyper
Project-URL: Issues, https://github.com/deephyper/deephyper/issues
Author: Misha Salim, Romit Maulik, Venkat Vishwanath, Stefan Wild
Author-email: Romain Egele <regele@ornl.gov>, Prasanna Balaprakash <pbalapra@ornl.gov>
Maintainer: Gavin M. Wiggins
Maintainer-email: Romain Egele <regele@ornl.gov>, Prasanna Balaprakash <pbalapra@ornl.gov>, Brett Eiffert  <eiffertbc@ornl.gov>
License: BSD 3-Clause License
        
        Copyright (c) 2018, UChicago Argonne, LLC and the DeepHyper Development Team
        All Rights Reserved
        
        Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Requires-Dist: cloudpickle
Requires-Dist: configspace>=1.1.1
Requires-Dist: dm-tree
Requires-Dist: jinja2>=3.1.4
Requires-Dist: loky>=3.4
Requires-Dist: matplotlib
Requires-Dist: numpy>=1.26.0
Requires-Dist: packaging
Requires-Dist: pandas>=0.24.2
Requires-Dist: parse
Requires-Dist: psutil
Requires-Dist: pydantic>=2.10
Requires-Dist: pymoo>=0.6.0
Requires-Dist: pyyaml
Requires-Dist: scikit-learn>=0.23.1
Requires-Dist: scipy>=1.10
Requires-Dist: tqdm>=4.64.0
Provides-Extra: core
Requires-Dist: jax[cpu]>=0.3.25; extra == 'core'
Requires-Dist: numpyro[cpu]; extra == 'core'
Requires-Dist: torch>=2.0.0; extra == 'core'
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: gitpython; extra == 'dev'
Requires-Dist: ipython; extra == 'dev'
Requires-Dist: nbsphinx; extra == 'dev'
Requires-Dist: pydata-sphinx-theme==0.15.4; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: rstcheck; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: sphinx-book-theme==1.1.3; extra == 'dev'
Requires-Dist: sphinx-copybutton; extra == 'dev'
Requires-Dist: sphinx-design==0.6.1; extra == 'dev'
Requires-Dist: sphinx-gallery; extra == 'dev'
Requires-Dist: sphinx-lfs-content; extra == 'dev'
Requires-Dist: sphinx-togglebutton; extra == 'dev'
Requires-Dist: sphinx>=5; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Provides-Extra: jax-cpu
Requires-Dist: jax[cpu]>=0.3.25; extra == 'jax-cpu'
Requires-Dist: numpyro[cpu]; extra == 'jax-cpu'
Provides-Extra: jax-cuda
Requires-Dist: jax[cuda]>=0.3.25; extra == 'jax-cuda'
Requires-Dist: numpyro[cuda]; extra == 'jax-cuda'
Provides-Extra: mpi
Requires-Dist: mpi4py>=3.1.3; extra == 'mpi'
Provides-Extra: ray
Requires-Dist: ray[default]>=1.3.0; extra == 'ray'
Provides-Extra: redis
Requires-Dist: redis; extra == 'redis'
Provides-Extra: redis-hiredis
Requires-Dist: redis[hiredis]; extra == 'redis-hiredis'
Provides-Extra: torch
Requires-Dist: torch>=2.0.0; extra == 'torch'
Description-Content-Type: text/markdown

<p align="center">
<img src="docs/_static/logo/medium.png">
</p>

[![DOI](https://zenodo.org/badge/156403341.svg)](https://zenodo.org/badge/latestdoi/156403341)
![GitHub tag (latest by date)](https://img.shields.io/github/tag-date/deephyper/deephyper.svg?label=version)
[![Documentation Status](https://readthedocs.org/projects/deephyper/badge/?version=latest)](https://deephyper.readthedocs.io/en/latest/?badge=latest)
![License](https://img.shields.io/github/license/deephyper/deephyper)
![PyPI - Downloads](https://img.shields.io/pypi/dm/deephyper.svg?label=Pypi%20downloads)

## What is DeepHyper?

DeepHyper is a powerful Python package for automating machine learning tasks, particularly focused on optimizing hyperparameters, searching for optimal neural architectures, and quantifying uncertainty through the use of deep ensembles. With DeepHyper, users can easily perform these tasks on a single machine or distributed across multiple machines, making it ideal for use in a variety of environments. Whether you're a beginner looking to optimize your machine learning models or an experienced data scientist looking to streamline your workflow, DeepHyper has something to offer. So why wait? Start using DeepHyper today and take your machine-learning skills to the next level!

## Installation

Installation with `pip`:

```console
pip install deephyper
```

More details about the installation process can be found in our [Installation](https://deephyper.readthedocs.io/en/stable/install/) documentation.

## Quickstart

The black-box function named `run` is defined by taking an input job named `job` which contains the different variables to optimize `job.parameters`. Then the run-function is bound to an `Evaluator` in charge of distributing the computation of multiple evaluations. Finally, a Bayesian search named `CBO` is created and executed to find the values of config which **MAXIMIZE** the return value of `run(job)`.

```python
def run(job):
    # The suggested parameters are accessible in job.parameters (dict)
    x = job.parameters["x"]
    b = job.parameters["b"]

    if job.parameters["function"] == "linear":
        y = x + b
    elif job.parameters["function"] == "cubic":
        y = x**3 + b

    # Maximization!
    return y


# Necessary IF statement otherwise it will enter in a infinite recursion
# when loading the 'run' function from a child processes
if __name__ == "__main__":
    from deephyper.hpo import CBO, HpProblem
    from deephyper.evaluator import Evaluator

    # define the variable you want to optimize
    problem = HpProblem()
    problem.add_hyperparameter((-10.0, 10.0), "x") # real parameter
    problem.add_hyperparameter((0, 10), "b") # discrete parameter
    problem.add_hyperparameter(["linear", "cubic"], "function") # categorical parameter

    # define the evaluator to distribute the computation
    evaluator = Evaluator.create(
        run,
        method="process",
        method_kwargs={
            "num_workers": 2,
        },
    )

    # define your search and execute it
    search = CBO(problem, evaluator, random_state=42)

    results = search.search(max_evals=100)
    print(results)
```

Which outputs the following results where the best parameters are with `function == "cubic"`, 
`x == 9.99` and `b == 10`.

```verbatim
     p:b p:function       p:x    objective  job_id job_status  m:timestamp_submit  m:timestamp_gather
0      7      cubic -1.103350     5.656803       0       DONE            0.018402            1.548068
1      3      cubic  8.374450   590.312101       1       DONE            0.018485            1.548254
2      9     linear  8.787395    17.787395       3       DONE            1.564276            1.565336
3      6      cubic  4.680560   108.540056       2       DONE            1.564209            1.565440
4      2      cubic  4.012429    66.598442       5       DONE            1.575218            1.576076
..   ...        ...       ...          ...     ...        ...                 ...                 ...
96    10      cubic  9.986875  1006.067558      96       DONE            9.895560            9.995656
97     9      cubic  9.999787  1008.936159      97       DONE            9.995220           10.095534
98     9      cubic  9.997146  1008.143990      98       DONE           10.095102           10.195398
99     7      cubic  9.999389  1006.816600      99       DONE           10.194956           10.297452
100    9      cubic  9.997912  1008.373594     100       DONE           10.296981           10.412184
```

The code defines a function `run` that takes a RunningJob `job` as input and returns the maximized objective `y`. The `if` block at the end of the code defines a black-box optimization process using the `CBO` (Centralized Bayesian Optimization) algorithm from the `deephyper` library.

The optimization process is defined as follows:

1. A hyperparameter optimization problem is created using the `HpProblem` class from `deephyper`. In this case, the problem has three variables. The `x` hyperparameter is a real variable in a range from -10.0 to 10.0. The `b` hyperparameter is a discrete variable in a range from 0 to 10. The `function` hyperparameter is a categorical variable with two possible values.
2. An evaluator is created using the `Evaluator.create` method. The evaluator will be used to evaluate the function `run` with different configurations of suggested hyperparameters in the optimization problem. The evaluator uses the `process` method to distribute the evaluations across multiple worker processes, in this case, 2 worker processes.
3. A search object is created using the `CBO` class, the problem and evaluator defined earlier. The `CBO` algorithm is a derivative-free optimization method that uses a Bayesian optimization approach to explore the hyperparameter space.
4. The optimization process is executed by calling the `search.search` method, which performs the evaluations of the `run` function with different configurations of the hyperparameters until a maximum number of evaluations (100 in this case) is reached.
5. The results of the optimization process, including the optimal configuration of the hyperparameters and the corresponding objective value, are printed to the console.

## How do I learn more?

Check out our online documentation with API reference and examples: <https://deephyper.readthedocs.io>

## Citing DeepHyper

If you wish to cite the Software, please use the following:

```
@misc{deephyper_software,
    title = {"DeepHyper: A Python Package for Scalable Neural Architecture and Hyperparameter Search"},
    author = {Balaprakash, Prasanna and Egele, Romain and Salim, Misha and Maulik, Romit and Vishwanath, Venkat and Wild, Stefan and others},
    organization = {DeepHyper Team},
    year = 2018,
    url = {https://github.com/deephyper/deephyper}
} 
```

## How can I participate?

Questions, comments, feature requests, bug reports, etc. can be directed to Github Issues.

Patches through pull requests are much appreciated on the software itself as well as documentation.

More documentation about how to contribute is available on [deephyper.readthedocs.io/en/latest/developer_guides/contributing.html](https://deephyper.readthedocs.io/en/latest/developer_guides/contributing.html).

## Acknowledgments

* Scalable Data-Efficient Learning for Scientific Domains, U.S. Department of Energy 2018 Early Career Award funded by the Advanced Scientific Computing Research program within the DOE Office of Science (2018--Present)
* Argonne Leadership Computing Facility: This research used resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357.
* SLIK-D: Scalable Machine Learning Infrastructures for Knowledge Discovery, Argonne Computing, Environment and Life Sciences (CELS) Laboratory Directed Research and Development (LDRD) Program (2016--2018)

## Copyright and license

Copyright © 2019, UChicago Argonne, LLC

DeepHyper is distributed under the terms of BSD License. See [LICENSE](https://github.com/deephyper/deephyper/blob/master/LICENSE)

Argonne Patent & Intellectual Property File Number: SF-19-007
