Metadata-Version: 2.4
Name: stormlog
Version: 0.3.3
Summary: A comprehensive GPU memory profiler for PyTorch and TensorFlow with CLI, visualization, and analytics
Author-email: Silas Asamoah <silasbempong@gmail.com>, Prince Agyei Tuffour <prince.agyei.tuffour@gmail.com>
Maintainer-email: Silas Asamoah <silasbempong@gmail.com>, Prince Agyei Tuffour <prince.agyei.tuffour@gmail.com>, Derrick Dwamena <derrickasante07@gmail.com>
License: MIT License
        
        Copyright (c) 2025 GPU Memory Profiler Team
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/Silas-Asamoah/stormlog
Project-URL: Documentation, https://github.com/Silas-Asamoah/stormlog/tree/main/docs
Project-URL: Repository, https://github.com/Silas-Asamoah/stormlog.git
Project-URL: Bug Tracker, https://github.com/Silas-Asamoah/stormlog/issues
Project-URL: Release Notes, https://github.com/Silas-Asamoah/stormlog/blob/main/CHANGELOG.md
Keywords: gpu,memory,profiler,pytorch,tensorflow,deep-learning,monitoring
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.19.0
Requires-Dist: pandas>=1.2.0
Requires-Dist: psutil>=5.8.0
Requires-Dist: scipy>=1.7.0
Provides-Extra: viz
Requires-Dist: matplotlib>=3.3.0; extra == "viz"
Requires-Dist: seaborn>=0.11.0; extra == "viz"
Requires-Dist: plotly>=5.0.0; extra == "viz"
Requires-Dist: kaleido>=0.2.1; extra == "viz"
Requires-Dist: dash>=2.0.0; extra == "viz"
Requires-Dist: dash-bootstrap-components>=1.6.0; extra == "viz"
Provides-Extra: torch
Requires-Dist: torch>=1.8.0; extra == "torch"
Provides-Extra: tf
Requires-Dist: tensorflow>=2.4.0; extra == "tf"
Provides-Extra: wandb
Requires-Dist: wandb>=0.19.0; extra == "wandb"
Provides-Extra: all
Requires-Dist: matplotlib>=3.3.0; extra == "all"
Requires-Dist: seaborn>=0.11.0; extra == "all"
Requires-Dist: plotly>=5.0.0; extra == "all"
Requires-Dist: kaleido>=0.2.1; extra == "all"
Requires-Dist: dash>=2.0.0; extra == "all"
Requires-Dist: dash-bootstrap-components>=1.6.0; extra == "all"
Requires-Dist: torch>=1.8.0; extra == "all"
Requires-Dist: tensorflow>=2.4.0; extra == "all"
Requires-Dist: textual>=0.57.0; extra == "all"
Requires-Dist: pyfiglet>=1.0.2; extra == "all"
Requires-Dist: wandb>=0.19.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-cov>=2.10.0; extra == "dev"
Requires-Dist: pytest-mock>=3.6.0; extra == "dev"
Requires-Dist: pytest-xdist>=2.4.0; extra == "dev"
Requires-Dist: pexpect>=4.9.0; extra == "dev"
Requires-Dist: pytest-textual-snapshot>=1.1.0; extra == "dev"
Requires-Dist: jsonschema>=4.0.0; extra == "dev"
Requires-Dist: black==24.3.0; extra == "dev"
Requires-Dist: flake8>=3.8.0; extra == "dev"
Requires-Dist: mypy>=0.910; extra == "dev"
Requires-Dist: isort>=5.9.0; extra == "dev"
Requires-Dist: pre-commit>=2.15.0; extra == "dev"
Requires-Dist: sphinx>=4.0.0; extra == "dev"
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "dev"
Requires-Dist: myst-parser>=0.15.0; extra == "dev"
Requires-Dist: jupyter>=1.0.0; extra == "dev"
Requires-Dist: ipython>=7.0.0; extra == "dev"
Requires-Dist: notebook>=6.4.0; extra == "dev"
Requires-Dist: coverage>=5.5.0; extra == "dev"
Requires-Dist: tox>=3.24.0; extra == "dev"
Requires-Dist: memory-profiler>=0.60.0; extra == "dev"
Requires-Dist: line-profiler>=3.3.0; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest>=8.0.0; extra == "test"
Requires-Dist: pytest-cov>=2.10.0; extra == "test"
Requires-Dist: pytest-mock>=3.6.0; extra == "test"
Requires-Dist: pytest-xdist>=2.4.0; extra == "test"
Requires-Dist: pexpect>=4.9.0; extra == "test"
Requires-Dist: pytest-textual-snapshot>=1.1.0; extra == "test"
Requires-Dist: jsonschema>=4.0.0; extra == "test"
Requires-Dist: coverage>=5.5.0; extra == "test"
Requires-Dist: numpy>=1.19.0; extra == "test"
Requires-Dist: pandas>=1.2.0; extra == "test"
Requires-Dist: scipy>=1.7.0; extra == "test"
Requires-Dist: memory-profiler>=0.60.0; extra == "test"
Requires-Dist: line-profiler>=3.3.0; extra == "test"
Provides-Extra: tui
Requires-Dist: textual>=0.57.0; extra == "tui"
Requires-Dist: pyfiglet>=1.0.2; extra == "tui"
Provides-Extra: docs
Requires-Dist: sphinx>=4.0.0; extra == "docs"
Requires-Dist: sphinx-rtd-theme>=1.0.0; extra == "docs"
Requires-Dist: myst-parser>=0.15.0; extra == "docs"
Dynamic: license-file

# Stormlog

[![CI](https://github.com/Silas-Asamoah/stormlog/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/Silas-Asamoah/stormlog/actions/workflows/ci.yml)
[![PyPI Version](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fpypi.org%2Fpypi%2Fstormlog%2Fjson&query=%24.info.version&label=pypi)](https://pypi.org/project/stormlog/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/Silas-Asamoah/stormlog/blob/main/LICENSE)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![PyTorch](https://img.shields.io/badge/PyTorch-1.8+-red.svg)](https://pytorch.org/)
[![TensorFlow](https://img.shields.io/badge/TensorFlow-2.4+-orange.svg)](https://tensorflow.org/)
[![Textual TUI](https://img.shields.io/badge/TUI-Textual-blueviolet)](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/tui.md)

<p align="center">
  <img src="https://raw.githubusercontent.com/Silas-Asamoah/stormlog/main/docs/tui-overview-current.png" alt="Stormlog overview tab" width="900">
  <br/>
  <em>Current Overview tab from the shipped Textual interface.</em>
</p>

Stormlog is a memory-profiling toolkit for day-to-day PyTorch and TensorFlow work. It combines Python APIs, CLI commands, and a Textual TUI so you can move from "what is using memory?" to saved artifacts and shareable diagnostics without switching tools.

Long-running `track` and `diagnose` flows are session-aware, so one capture can
be reconstructed deterministically across sink segments, diagnose bundles, and
OOM flight-recorder artifacts.

For task-oriented operational guidance, use the
[Production Cookbook](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/index.md).
The highest-signal entry points are the
[Always-on Tracking recipe](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/always_on.md),
[Incident Playbooks](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/incidents.md),
and
[CI and Release Qualification](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/ci_release.md).

## Why use this tool

- Catch memory growth before it becomes an OOM.
- Compare allocated vs reserved usage during training and inference.
- Export telemetry and diagnose bundles for CI or release triage.
- Load the same artifacts into a terminal UI for faster debugging.
- Keep workflows available on CPU-only and MPS systems, not just CUDA boxes.

## Installation

### From PyPI

```bash
pip install stormlog
pip install "stormlog[viz]"
pip install "stormlog[tui,torch]"
pip install "stormlog[torch]"
pip install "stormlog[tf]"
pip install "stormlog[all]"
```

`stormlog[all]` installs every runtime extra: visualization, TUI, PyTorch, and TensorFlow dependencies.

### Package and import names

`stormlog` is the distribution name on PyPI and the primary Python import root.
TensorFlow-specific APIs live under `stormlog.tensorflow`.

| Task | Use |
| --- | --- |
| Install the package | `pip install stormlog` |
| Launch the TUI | `stormlog` |
| Import PyTorch APIs | `from stormlog import GPUMemoryProfiler, MemoryTracker` |
| Import TensorFlow APIs | `from stormlog.tensorflow import TFMemoryProfiler` |
| Run CLI automation | `gpumemprof` or `tfmemprof` |

### From source

```bash
git clone https://github.com/Silas-Asamoah/stormlog.git
cd stormlog
pip install -e .
pip install -e ".[viz,tui,torch]"
```

If you want both framework extras in a development checkout:

```bash
pip install -e ".[dev,test,all]"
```

The `examples/` package and Markdown test guides are source-checkout only. A
plain `pip install stormlog` does not include them.

## Quick start

### CLI-first workflow

This is the fastest path to verify an environment and produce an artifact you can inspect later:

```bash
gpumemprof info
gpumemprof track --duration 2 --interval 0.5 --output /tmp/gpumemprof_track.json --format json
gpumemprof analyze /tmp/gpumemprof_track.json --format txt --output /tmp/gpumemprof_analysis.txt
gpumemprof diagnose --duration 0 --output /tmp/gpumemprof_diag

tfmemprof info
tfmemprof diagnose --duration 0 --output /tmp/tf_diag
```

If you reuse a sink directory across multiple runs, Stormlog separates those
captures by `session_id` and defaults analysis to the newest clean session.

### PyTorch API workflow

`GPUMemoryProfiler` currently targets PyTorch runtimes exposed through
`torch.cuda`, which covers NVIDIA CUDA and ROCm-backed builds. On Apple MPS or
CPU-only systems, use `MemoryTracker`, the CLI, or `CPUMemoryProfiler` instead.

```python
import torch
from stormlog import GPUMemoryProfiler

profiler = GPUMemoryProfiler()
device = profiler.device
model = torch.nn.Linear(1024, 128).to(device)

def train_step() -> torch.Tensor:
    x = torch.randn(64, 1024, device=device)
    y = model(x)
    return y.sum()

profile = profiler.profile_function(train_step)
summary = profiler.get_summary()

print(profile.function_name)
print(f"Peak memory: {summary['peak_memory_usage'] / (1024**3):.2f} GB")
```

### TensorFlow API workflow

`TFMemoryProfiler` works on GPU or CPU-backed TensorFlow runtimes.

```python
from stormlog.tensorflow import TFMemoryProfiler

profiler = TFMemoryProfiler(enable_tensor_tracking=True)

with profiler.profile_context("training"):
    model.fit(x_train, y_train, epochs=1, batch_size=32)

results = profiler.get_results()
print(f"Peak memory: {results.peak_memory_mb:.2f} MB")
print(f"Snapshots captured: {len(results.snapshots)}")
```

## Daily workflows

### ML engineer

- instrument a training step with `GPUMemoryProfiler` or `TFMemoryProfiler`
- switch to `track` when you need telemetry over time
- export plots or analyze saved telemetry later

### Researcher debugging regressions

- capture `track` output or a `diagnose` bundle
- open the same artifacts in the TUI diagnostics and visualizations tabs
- compare growth, gaps, and per-rank behavior before changing model code
- switch between discovered sessions instead of merging multiple captures from the same sink

### CI or release owner

These example-module commands require a source checkout plus `pip install -e .`.

- run `python -m examples.cli.quickstart`
- run `python -m examples.cli.capability_matrix --mode smoke --target both --oom-mode simulated`
- archive the emitted artifacts for later triage in the TUI

## Terminal UI

Install the optional TUI dependencies and launch:

```bash
pip install "stormlog[tui,torch]"
stormlog
```

The current TUI startup path imports PyTorch immediately, so `stormlog[tui]` alone is not enough yet.

The current TUI tabs are:

- `Overview`
- `PyTorch`
- `TensorFlow`
- `Monitoring`
- `Visualizations`
- `Diagnostics`
- `CLI & Actions`

<p align="center">
  <img src="https://raw.githubusercontent.com/Silas-Asamoah/stormlog/main/docs/tui-overview-current.png" alt="Stormlog overview tab" width="700">
  <br/>
  <em>Overview tab with current system summary and navigation guidance.</em>
</p>

<p align="center">
  <img src="https://raw.githubusercontent.com/Silas-Asamoah/stormlog/main/docs/tui-diagnostics-current.png" alt="Stormlog diagnostics tab" width="700">
  <br/>
  <em>Diagnostics tab with the current artifact loader, rank table, and timeline panes.</em>
</p>

Use the Monitoring tab to start live tracking, export CSV or JSON events to `./exports`, and tune warning or critical thresholds. In Visualizations, refresh the live timeline and save PNG or HTML exports under `./visualizations`. In Diagnostics, load live telemetry or artifact paths and rebuild rank-level diagnostics without leaving the terminal.

For screen-by-screen details, see the [TUI Guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/tui.md).

## Examples and walkthroughs

- [Documentation home](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/index.md)
- [Installation guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/installation.md)
- [Usage guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/usage.md)
- [CLI guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cli.md)
- [Production cookbook](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/index.md)
- [Always-on tracking](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/always_on.md)
- [Incident playbooks](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/incidents.md)
- [CI and release qualification](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cookbook/ci_release.md)
- [Examples guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/examples.md)
- [Testing guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/testing.md)
- [PyTorch guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/pytorch_testing_guide.md)
- [TensorFlow guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/tensorflow_testing_guide.md)
- [Long-form article](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/article.md)

## Launch QA scenarios

These commands require a source checkout:

```bash
python -m examples.cli.quickstart
python -m examples.cli.capability_matrix --mode smoke --target both --oom-mode simulated
python -m examples.scenarios.cpu_telemetry_scenario
python -m examples.scenarios.oom_flight_recorder_scenario --mode simulated
```

## CPU-only and laptop workflows

If CUDA is not available, Stormlog still supports:

- `gpumemprof info`
- `gpumemprof monitor`
- `gpumemprof track`
- `CPUMemoryProfiler`
- `CPUMemoryTracker`
- the TUI overview, monitoring, diagnostics, and CLI tabs

See the [CPU Compatibility Guide](https://github.com/Silas-Asamoah/stormlog/blob/main/docs/cpu_compatibility.md) for the CPU-only path.

## Contributing

See the [Contributing Guide](https://github.com/Silas-Asamoah/stormlog/blob/main/CONTRIBUTING.md) and [Code of Conduct](https://github.com/Silas-Asamoah/stormlog/blob/main/CODE_OF_CONDUCT.md).

## License

[MIT License](https://github.com/Silas-Asamoah/stormlog/blob/main/LICENSE)
