Metadata-Version: 2.3
Name: confusion-visualization
Version: 1.0
Project-URL: Homepage, https://github.com/suksia/confusion-visualization
Project-URL: Issues, https://github.com/suksia/confusion-visualization/issues
Author-email: Aiden Ochoa <aio5165@psu.edu>
License: MIT License
        
        Copyright (c) 2024 Aiden Ochoa
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Dist: matplotlib>=3.9.1
Requires-Dist: opencv-python>=4.10.0.84
Requires-Dist: scikit-learn>=1.5.1
Description-Content-Type: text/markdown

# Overview
After training a neural network for binary segmentation, it will produce a grayscale prediction map where each pixel represents the probability of belonging to the positive class (i.e. 0 is negative, and 1 is positive). 

Given the human label, which acts as the ground-truth, the correctness of each pixel-wise prediction can be classified using the confusion matrix:

- True positive
- True negative
- False positive (type 1 error)
- False negative (type 2 error)

It may be informative to see where predictions were correct/incorrect given the human label, and to further see the type of error made by the machine. This visualizes the disagreement between human and machine and helps show biased labelling.

# Usage

The script ```src/demo.py``` performs a demo of the program using some provided data in ```data/```. It can also be used with custom data by supplying arguments that define paths to the top-level data directory and its sub-directories containing training images, labels, and output predictions. The user can also define a custom path (absolute, or relative) to a directory that will contain program output. Use ```demo.py --help``` to see more information about customizing supplied paths.

Predictions should be a result of binary segmentation; multiclass objectives are not supported. Each image/label/prediction pair should also have the same shape and filenames, though different (but consistent) extensions can be used.

# Output Results

The program creates a new directory that contains three sub-directories:

- ```confusion_masks/```
    - Standalone images with each pixel colored according to the confusion mask
- ```confusion_compare/```
    - Raw images, human labels, and confusion masks plotted together with the F1 found from Otsu thresholding
- ```prediction_compare/```
    - Raw images, human labels, and raw output predictions plotted together

