Metadata-Version: 2.4
Name: easybench
Version: 0.1.34
Summary: A simple and easy-to-use Python benchmarking library
Project-URL: Homepage, https://github.com/smurak/easybench
Project-URL: Bug Tracker, https://github.com/smurak/easybench/issues
Project-URL: Documentation, https://easybench.readthedocs.io/
Author-email: smurak <smurak000@gmail.com>
License: The MIT License (MIT)
        
        Copyright (c) 2025 smurak
        
        Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
License-File: LICENSE
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Requires-Dist: pydantic>=2.0.0
Requires-Dist: typing-extensions; python_version < '3.11'
Provides-Extra: all
Requires-Dist: matplotlib; extra == 'all'
Requires-Dist: pandas; extra == 'all'
Requires-Dist: seaborn; extra == 'all'
Requires-Dist: tqdm; extra == 'all'
Provides-Extra: dev
Requires-Dist: black; extra == 'dev'
Requires-Dist: matplotlib; extra == 'dev'
Requires-Dist: mypy; extra == 'dev'
Requires-Dist: pandas; extra == 'dev'
Requires-Dist: pandas-stubs; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: seaborn; extra == 'dev'
Requires-Dist: tqdm; extra == 'dev'
Requires-Dist: types-seaborn; extra == 'dev'
Requires-Dist: types-tqdm; extra == 'dev'
Provides-Extra: docs
Requires-Dist: black; extra == 'docs'
Requires-Dist: mkdocs-material; extra == 'docs'
Requires-Dist: mkdocstrings[python]; extra == 'docs'
Provides-Extra: test
Requires-Dist: pytest; extra == 'test'
Requires-Dist: pytest-cov; extra == 'test'
Description-Content-Type: text/markdown

# EasyBench

[![Tests](https://github.com/smurak/easybench/actions/workflows/test.yml/badge.svg)](https://github.com/smurak/easybench/actions)
[![Docs](https://readthedocs.org/projects/easybench/badge/?version=latest)](https://easybench.readthedocs.io/)
[![PyPI version](https://badge.fury.io/py/easybench.svg)](https://pypi.org/project/easybench/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)
[![Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

[**Docs**](https://easybench.readthedocs.io/)

A simple and easy-to-use Python benchmarking library.

## Features

- Three benchmarking styles: decorator, class-based, and command-line
- Measure both execution time and memory usage ([see limitations](#memory-measurement-limitations))
- Rich visualizations (Boxplot, Violinplot, Lineplot, Histogram, Barplot)
- Advanced options: warmup runs, multiple loops per trial, outlier trimming
- Parametrized benchmarks for comparing function performance with different inputs
- pytest-like fixtures and lifecycle hooks (setup/teardown)
- Flexible configuration: time/memory units, progress tracking, filtering
- Multiple output formats (text tables, CSV, JSON, pandas.DataFrame)
- Extensible reporting system for custom outputs

## Installation

```bash
pip install easybench
```

### Optional Dependencies

EasyBench supports optional dependencies for additional features:

```bash
# Install with visualization support
pip install easybench[all]
```

The `all` option includes:

- `matplotlib`: For visualization and plotting benchmark results
- `seaborn`: For enhanced statistical visualizations
- `pandas`: For outputting benchmark results as DataFrames
- `tqdm`: For progress tracking during benchmark execution

## Quick Start

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/smurak/easybench/blob/main/notebooks/easybench_quickstart.ipynb)

There are 3 ways to benchmark with `easybench`:

1. `@bench` decorator

    ```python
    from easybench import bench
    
    # Add @bench with function parameters
    @bench(item=123, big_list=lambda: list(range(1_000_000)))
    def insert_first(item, big_list):
        big_list.insert(0, item)
    ```

    * When you need fresh data for each trial, use a function or lambda to generate new data on demand.  
      (like `lambda: list(range(1_000_000))` in the above)

2. `EasyBench` class

    ```python
    from easybench import EasyBench, BenchConfig
    
    class BenchListOperation(EasyBench):
        # Benchmark configuration
        bench_config = BenchConfig(
            trials=10,
            memory=True,
            sort_by="avg"
        )
    
        # Setup for each trial
        def setup_trial(self):
            self.big_list = list(range(1_000_000))
    
        # Benchmark methods (must start with bench_)
        def bench_insert_first(self):
            self.big_list.insert(0, 123)
    
        # You can define multiple benchmark methods
        def bench_pop_first(self):
            self.big_list.pop(0)
    
    if __name__ == "__main__":
        # Run benchmark
        BenchListOperation().bench()
    ```

3. `easybench` command

    1. Create a `benchmarks` directory
    2. Put `bench_*.py` scripts in the directory:

        ```python
        from easybench import fixture
        
        # Fixture for each trial
        @fixture(scope="trial")
        def big_list():
            return list(range(1_000_000))
        
        # Benchmark functions (must start with bench_)
        def bench_insert_first(big_list):
            big_list.insert(0, 123)
        
        # You can define multiple benchmark functions
        def bench_pop_first(big_list):
            big_list.pop(0)
        ```

    3. Run `easybench` command

        ```bash
        easybench --trials 10 --memory --sort-by avg
        ```

**Example of benchmark results:**

* Single benchmark

    ```
    Benchmark Results (5 trials):
    
    Function        Avg Time (s)  Min Time (s)  Max Time (s)
    --------------------------------------------------------
    insert_first        0.001568      0.001071      0.003265
    ```

* Multiple benchmarks

  ![EasyBench Benchmark Result](https://raw.githubusercontent.com/smurak/easybench/main/images/easybench_screenshot.png)

* Boxplot Visualization

  ![Boxplot Visualization](https://raw.githubusercontent.com/smurak/easybench/main/images/visualization_boxplot.png)

* Violinplot Visualization

  ![Violinplot Visualization](https://raw.githubusercontent.com/smurak/easybench/main/images/visualization_violinplot.png)

* Lineplot Visualization

  ![Lineplot Visualization](https://raw.githubusercontent.com/smurak/easybench/main/images/visualization_lineplot.png)

* Histogram Visualization

  ![Histplot Visualization](https://raw.githubusercontent.com/smurak/easybench/main/images/visualization_histplot.png)

* Barplot Visualization

  ![Barplot Visualization](https://raw.githubusercontent.com/smurak/easybench/main/images/visualization_barplot.png)

## Usage

For detailed usage instructions, please refer to the [**documentation**](https://easybench.readthedocs.io/).

## Memory Measurement Limitations

EasyBench uses Python's built-in `tracemalloc` module to measure memory usage.  
This has some important limitations:

- `tracemalloc` only tracks memory allocations made through Python's memory manager
- Memory allocated by C extensions (like NumPy, Pandas, or other native libraries) often bypasses Python's memory manager and won't be accurately measured
- The reported memory usage reflects Python objects only, not the total process memory consumption
For applications heavily using C extensions, consider using external profilers like `memory_profiler` or system monitoring tools for more accurate measurements.

## License

[MIT](https://github.com/smurak/easybench/blob/main/LICENSE)
