Metadata-Version: 2.1
Name: evalutils
Version: 0.2.1
Summary: evalutils helps challenge administrators easily create evaluation containers for grand-challenge.org.
Home-page: https://github.com/comic/evalutils
Author: James Meakin
Author-email: jamesmeakin@gmail.com
License: MIT license
Keywords: evalutils
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Requires-Python: >=3.6.0
Requires-Dist: pandas (!=0.24.0)
Requires-Dist: imageio
Requires-Dist: SimpleITK
Requires-Dist: cookiecutter
Requires-Dist: click
Requires-Dist: scipy
Requires-Dist: scikit-learn
Requires-Dist: numpy
Provides-Extra: test
Requires-Dist: pytest ; extra == 'test'
Requires-Dist: pytest-cov ; extra == 'test'

=========
evalutils
=========

.. image:: https://badge.fury.io/py/evalutils.svg
   :target: https://badge.fury.io/py/evalutils

.. image:: https://github.com/comic/evalutils/workflows/CI/badge.svg
   :target: https://github.com/comic/evalutils/actions?query=workflow%3ACI+branch%3Amaster
   :alt: Build Status

.. image:: https://codecov.io/gh/comic/evalutils/branch/master/graph/badge.svg
   :target: https://codecov.io/gh/comic/evalutils
   :alt: Code Coverage Status

.. image:: https://readthedocs.org/projects/evalutils/badge/?version=latest
   :target: https://evalutils.readthedocs.io/en/latest/?badge=latest
   :alt: Documentation Status

.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
   :target: https://github.com/ambv/black


evalutils helps challenge administrators easily create evaluation containers for grand-challenge.org.

* Free software: MIT license
* Documentation: https://evalutils.readthedocs.io.

Features
--------

* Generation your challenge evaluation project boilerplate using Cookiecutter_
* Scripts to build, test and export your generated docker container for grand-challenge.org
* Loading of CSV, ITK and Pillow compatible prediction files
* Validation of submitted predictions
* Interface to SciKit-Learn metrics and Pandas aggregations
* Bounding box annotations with Jaccard Index calculations


Getting Started
---------------

evalutils_ requires Python 3.6, and can be installed from `pip`. Please
see the `Getting Started`_ documentation for more details.


Credits
-------

This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.

.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
.. _evalutils: https://github.com/comic/evalutils
.. _`Getting Started`: https://evalutils.readthedocs.io/en/latest/usage.html


=======
History
=======

0.2.1 (2020-02-12)
------------------

* Added support for mha input for algorithms

0.2.0 (2020-02-11)
------------------

* Added experimental support for templating algorithms
  * Note that the `init` cli command has changed
* Added support for Python 3.8

0.1.17 (2019-05-23)
-------------------

* Fixes build and test scripts on windows
* Fixes BallTree execution with scipy 0.21.1

0.1.16 (2019-02-13)
-------------------

* Fixes bug in `export.sh` due to extra }

0.1.15 (2019-02-01)
-------------------

* Pins Pandas version due to a bug with dataframe conversion to dictionaries in Pandas 0.24.0
* Updates the export.sh template to gzip the docker container after creation
* Added Python 3.7 support

0.1.14 (2018-10-16)
-------------------

* Adds `evalutils.stats` for calculating common metrics in medical imaging
* Adds **experimental**  Windows support - we do not have CI on all of the windows tests so please report any errors
* Adds `evalutils.roc` for calculating bootstrapped roc curves

0.1.13 (2018-09-12)
-------------------

* Adds Segmentation and Detection challenges to the project generator
* File loaders now return lists of dictionaries rather than dictionaries
* Renamed `intersection_over_union` to `jaccard_index`
* Improved image memory management. Added `io.ImageLoader` and separate `load_image` and `hash_image` functions.
* `score_case` is no longer a static method


0.1.12 (2018-05-16)
-------------------

* Fixed a bug where the number of `false_negatives` could be less than 0.

0.1.11 (2018-05-15)
-------------------

* **Breaking change:** Renamed ``Evaluation`` to ``ClassificationEvaluation``
* Adds support for ``DetectionEvaluation``

0.1.10 (2018-04-19)
-------------------

* Simplifies the example template

0.1.9 (2018-04-19)
------------------

* **Breaking change:** Renamed ``bb2`` to ``other`` in ``BoundingBox()``
* ``ground_truth_path`` is no longer a required argument

0.1.8 (2018-04-18)
------------------

* Fixes template folder in distribution

0.1.7 (2018-04-18)
------------------

* Adds cookiecutter templating for generating new projects
* Adds equality check for ``BoundingBox``

0.1.6 (2018-03-30)
------------------

* Improves pandas csv handling

0.1.5 (2018-03-30)
------------------

* Corrects loading of some CSV files
* Adds logging and more tests
* Adds referencing to ``_ground_truth`` and ``_prediction`` in joined pandas tables


0.1.3 (2018-03-29)
------------------

* Adds basic implementation with
    * Full Evaluation workflow
    * CSV, SimpleITK, and ImageIO loaders
    * BoundingBox annotations with intersection, union and intersection over union metrics
    * Unique File Indices, Unique Image, Expected Column Names and Number of cases validators


0.1.0 (2018-03-22)
------------------

* First release on PyPI.


