Metadata-Version: 2.4
Name: trtutils
Version: 0.6.0
Summary: Utilities for making TensorRT easier to use
Author-email: Justin Davis <davisjustin302@gmail.com>
Maintainer-email: Justin Davis <davisjustin302@gmail.com>
Project-URL: Homepage, https://github.com/justincdavis/trtutils
Project-URL: Bug Tracker, https://github.com/justincdavis/trtutils/issues
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Natural Language :: English
Classifier: Topic :: System :: Hardware
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Processing
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Classifier: Topic :: Software Development
Classifier: Topic :: Software Development :: Embedded Systems
Classifier: Topic :: Software Development :: Libraries
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: End Users/Desktop
Classifier: Typing :: Typed
Requires-Python: <=3.14,>=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: tensorrt>=8.0.0
Requires-Dist: cuda-python>=8.0.0
Requires-Dist: numpy<2.2.0,>=1.19.0
Requires-Dist: cv2ext>=0.1.0
Requires-Dist: opencv-python>=4.8.0
Requires-Dist: jetsontools>=0.1.0
Requires-Dist: tqdm>=4.67.1
Requires-Dist: typing_extensions>=4.0.0
Provides-Extra: all
Requires-Dist: trtutils[jit]; extra == "all"
Provides-Extra: jit
Requires-Dist: numba>=0.57.0; extra == "jit"
Provides-Extra: benchmark
Requires-Dist: torch>=2.0.0; extra == "benchmark"
Requires-Dist: ultralytics>=8.3.100; extra == "benchmark"
Requires-Dist: onnx>=1.14.0; extra == "benchmark"
Requires-Dist: onnxslim>=0.1.20; extra == "benchmark"
Requires-Dist: matplotlib>=3.0.0; extra == "benchmark"
Provides-Extra: ci
Requires-Dist: ruff>=0.11.13; extra == "ci"
Requires-Dist: mypy>=1.16.0; extra == "ci"
Requires-Dist: types-tqdm>=4.67; extra == "ci"
Provides-Extra: test
Requires-Dist: pytest>=6.2.0; extra == "test"
Provides-Extra: docs
Requires-Dist: sphinx>=7.0.0; extra == "docs"
Requires-Dist: sphinx_design>=0.6.1; extra == "docs"
Requires-Dist: furo>=2024.8.6; extra == "docs"
Requires-Dist: myst_parser>=4.0.0; extra == "docs"
Requires-Dist: matplotlib>=3.0.0; extra == "docs"
Provides-Extra: dev
Requires-Dist: trtutils[ci]; extra == "dev"
Requires-Dist: trtutils[test]; extra == "dev"
Requires-Dist: trtutils[docs]; extra == "dev"
Requires-Dist: twine>=4.0.0; extra == "dev"
Requires-Dist: wheel>=0.37.0; extra == "dev"
Requires-Dist: bumpver>=2023.1126; extra == "dev"
Requires-Dist: pyclean>=2.7.0; extra == "dev"
Requires-Dist: pyright>=1.1.348; extra == "dev"
Dynamic: license-file

# trtutils

[![](https://img.shields.io/pypi/pyversions/trtutils.svg)](https://pypi.org/pypi/trtutils/)
![PyPI](https://img.shields.io/pypi/v/trtutils.svg?style=plastic)
[![CodeFactor](https://www.codefactor.io/repository/github/justincdavis/trtutils/badge)](https://www.codefactor.io/repository/github/justincdavis/trtutils)

![MyPy](https://github.com/justincdavis/trtutils/actions/workflows/mypy.yaml/badge.svg?branch=main)
![Ruff](https://github.com/justincdavis/trtutils/actions/workflows/ruff.yaml/badge.svg?branch=main)
![PyPi Build](https://github.com/justincdavis/trtutils/actions/workflows/build-check.yaml/badge.svg?branch=main)
<!-- ![Black](https://github.com/justincdavis/trtutils/actions/workflows/black.yaml/badge.svg?branch=main) -->

A high-level Python interface for TensorRT inference, providing a simple and unified way to run arbitrary TensorRT engines. This library abstracts away the complexity of CUDA memory management, binding management, and engine execution, making it easy to perform inference with any TensorRT engine.

## Features

- Simple, high-level interface for TensorRT inference
- Automatic CUDA memory management
- Support for arbitrary TensorRT engines
- Built-in preprocessing and postprocessing capabilities
- Comprehensive type hints and documentation
- Support for both basic engine execution and end-to-end model inference

## Quick Start

### Basic Engine Usage

The `TRTEngine` class provides a simple interface for running any TensorRT engine:

```python
from trtutils import TRTEngine

# Load your TensorRT engine
engine = TRTEngine("path_to_engine")

# Get input specifications
print(engine.input_shapes)  # Expected input shapes
print(engine.input_dtypes)  # Expected input data types

# Run inference
inputs = read_your_data()
outputs = engine.execute(inputs)
```

## Installation

```bash
pip install trtutils
```

For additional features, you can install optional dependencies:

```bash
# For JIT compiler
pip install "trtutils[jit]"

# For development
pip install "trtutils[dev]"
```

## Documentation

For detailed documentation, including advanced usage, examples, and API reference, visit our [documentation site](https://trtutils.readthedocs.io/).

## Examples

Check out our [examples directory](examples/) for more detailed usage examples, including:
- Basic engine usage
- End-to-end model inference
- YOLO model implementation
- Benchmarking utilities

## Performance

| Device            | YOLOv8m                                                                 | YOLOv8n                                                                 |
|-------------------|-------------------------------------------------------------------------|-------------------------------------------------------------------------|
| OrinAGX-64GB      | ![OrinAGX-64GB YOLOv8m](benchmark/plots/OrinAGX-64GB/yolov8m.png)       | ![OrinAGX-64GB YOLOv8n](benchmark/plots/OrinAGX-64GB/yolov8n.png)       |
| OrinAGX-32GB      | ![OrinAGX-32GB YOLOv8m](benchmark/plots/OrinAGX-32GB/yolov8m.png)       | ![OrinAGX-32GB YOLOv8n](benchmark/plots/OrinAGX-32GB/yolov8n.png)       |
| OrinNX-16GB       | ![OrinNX-16GB YOLOv8m](benchmark/plots/OrinNX-16GB/yolov8m.png)         | ![OrinNX-16GB YOLOv8n](benchmark/plots/OrinNX-16GB/yolov8n.png)        |
| OrinNano-8GB      | ![OrinNano-8GB YOLOv8m](benchmark/plots/OrinNano-8GB/yolov8m.png)       | ![OrinNano-8GB YOLOv8n](benchmark/plots/OrinNano-8GB/yolov8n.png)       |
| XavierNX-8GB      | ![XavierNX-8GB YOLOv8m](benchmark/plots/XavierNX-8GB/yolov8m.png)       | ![XavierNX-8GB YOLOv8n](benchmark/plots/XavierNX-8GB/yolov8n.png)       |
| 3080Ti            | ![3080Ti YOLOv8m](benchmark/plots/3080Ti/yolov8m.png)                   | ![3080Ti YOLOv8n](benchmark/plots/3080Ti/yolov8n.png)                   |
| TitanRTX          | ![TitanRTX YOLOv8m](benchmark/plots/TitanRTX/yolov8m.png)               | ![TitanRTX YOLOv8n](benchmark/plots/TitanRTX/yolov8n.png)               |

## License

This project is licensed under the MIT License - see the LICENSE file for details.
