Metadata-Version: 2.4
Name: forensic-evidence
Version: 3.0.0
Summary: Evidence management
Author-email: Willy Pregliasco <willy.pregliasco@gmail.com>
License: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: pydlist
Requires-Dist: toml>=0.10.0
Requires-Dist: rich>=13.0.0
Requires-Dist: typer>=0.9.0
Requires-Dist: filetype>=1.0.0
Requires-Dist: duckdb>=1.0.0
Requires-Dist: ipykernel>=6.29.5
Requires-Dist: pillow>=10.4.0
Requires-Dist: ffmpeg-python>=0.2.0
Provides-Extra: media
Requires-Dist: Pillow>=8.0.0; extra == "media"
Requires-Dist: ffmpeg-python>=0.2.0; extra == "media"
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=2.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.8; extra == "dev"
Provides-Extra: all
Requires-Dist: evidence[dev,media]; extra == "all"

# Evidence - Audiovisual Forensics Suite

Evidence is a comprehensive audiovisual forensics suite for analyzing and processing multimedia files. It extends [`Dlist`](../dlist) with file scanning, metadata extraction, duplicate detection, and integrity verification.

## Contents

- [Quick start](#quick-start)
  - [Installation](#installation)
  - [Dependencies](#dependencies)
- [Modes](#modes)
- [CLI commands](#cli-commands)
  - [evidence-setup](#evidence-setup)
  - [evidence-build-database — Database builder](#evidence-build-database--database-builder)
  - [evidence-show — Database viewer](#evidence-show--database-viewer)
  - [idf — ID to filename](#idf--id-to-filename)
  - [fid — Filename to ID](#fid--filename-to-id)
  - [fname — Universal filename resolver](#fname--universal-filename-resolver)
  - [evidence-queue — Task queue](#evidence-queue--task-queue)
  - [evidence-obsidian — Obsidian vault generator](#evidence-obsidian--obsidian-vault-generator)
- [Queue system](#queue-system)
- [Evidence methods](#evidence-methods)
- [Quick reference](#quick-reference)
- [Project structure](#project-structure)
- [License](#license)

## Quick start

```python
from evidence import Evidence, Mode

ev = Evidence(mode=Mode.SCAN)    # scan all files, extract metadata
ev = Evidence(mode=Mode.LOAD)    # load previous scan
ev = Evidence(mode=Mode.RESCAN)  # add new files to existing database
ev = Evidence(mode=Mode.ZERO)    # empty instance

item = ev.idf('e42')             # look up by ID
print(item['file']['pathName'])
```

`Evidence` inherits from `Dlist`, so all Dlist operations work directly:

```python
ev.filter(file__mime__type='image')     # all images
ev.filter(type='E')                     # evidence items only
ev.partition('cat__disk')               # group by disk/directory
ev.tree()                               # visual structure overview
```

### Installation

```bash
uv venv
uv tool install -e .
```

With optional media dependencies:
```bash
uv tool install -e ".[media]"
```

For development:
```bash
uv tool install -e ".[dev]"
```

### Dependencies

- `dlist` — Data list management (local, from `../dlist`)
- `toml` — Configuration file parsing
- `rich` — Terminal formatting
- `typer` — CLI interface
- `Pillow` — Image processing (optional)
- `ffmpeg-python` — Video processing (optional)

---

## Modes

| Mode | Description |
|------|-------------|
| `DEFAULT` | Resolves to `LOAD` |
| `SCAN` | Full scan: reads all files, assigns IDs, extracts EXIF and media info, saves |
| `LOAD` | Loads previously saved database (DuckDB pivot join) |
| `RESCAN` | Detects new files on disk, runs full descriptors on them, appends to database |
| `ZERO` | Initializes empty database (backs up existing data) |

`ZERO` is a safe reset: it zips the current database into `backup/`, wipes all JSON files, and leaves an empty Evidence instance. Use it before a fresh `SCAN` when you want to start over, or to clean up corrupted data without losing the backup.

---

## CLI commands

### `evidence-setup`

Configuration manager for Evidence profiles (TOML-based). Add, edit, remove, select and list profiles.

Config file location:
- **Linux/macOS:** `~/.config/evidence/profiles.toml`
- **Windows:** `%APPDATA%\evidence\profiles.toml`

```bash
evidence-setup select              # interactive profile selection
evidence-setup add MyCase "D:\evidence" "D:\data" -d "Description"
evidence-setup list                # show all profiles
evidence-setup show                # show current profile
evidence-setup edit                # edit a profile interactively
evidence-setup remove              # remove a profile
```

> Multiple evidence paths use `;` as separator: `"path1;path2"`

### `evidence-build-database` — Database builder

Build or rebuild the Evidence database from the command line.

```bash
evidence-build-database            # auto: scan if no DB, otherwise load
evidence-build-database scan       # full scan from disk (backs up first)
evidence-build-database rescan     # add only new files (non-destructive)
evidence-build-database load       # load and verify existing DB (read-only)
```

| Subcommand | Destructive? | Description |
|------------|-------------|-------------|
| *(none)* | Safe if DB exists | Auto-detects: LOAD if DB exists, SCAN if not |
| `scan` | **Yes** (backs up first) | Full scan: wipes DB, walks all files, assigns IDs, saves |
| `rescan` | Appends only | Finds new files on disk, adds them to existing DB |
| `load` | **No** (read-only) | Loads DB and reports item count |

After `scan` or `rescan`, run `evidence-queue launch` to process background tasks (EXIF, media info, hashes).

### `evidence-show` — Database viewer

Display database contents.

```bash
evidence-show                      # default: file listing
evidence-show files                # list all files
evidence-show files --dirs         # list directories only
evidence-show files --dir photos   # list files in one directory
evidence-show struct               # data structure tree
evidence-show struct --depth 2     # tree with depth limit
```

### `idf` — ID to filename

Given an Evidence ID, returns the full pathname. Uses DuckDB to query JSON files directly — no Evidence class loaded.

```bash
idf e10    # → /full/path/to/SegCam Trelew.mp4
idf 42     # → /full/path/to/file.mp4
idf a3     # → /full/path/to/archive_file.mp4
```

### `fname` — Universal filename resolver

Resolves any input into a full pathname: IDs, partial filenames, relative paths, or absolute paths.

| Input | Resolution |
|---|---|
| Absolute path (`/full/path/file.mp4`) | Returned as-is |
| Evidence ID (`e10`, `42`, `a3`) | DuckDB lookup in `files.json` |
| Relative path with `/` (`tacho/vtests/test_5.mp4`) | Prepend EVIDENCE root |
| Bare name (`test_5`) | Fuzzy search in `files.json` |
| No match | Input returned unchanged |

```bash
fname e10        # → /full/path/to/SegCam Trelew.mp4
fname test_5     # → /full/path/to/tacho/vtests/test_5.mp4
fname 33         # → /full/path/to/tacho/vtests/test_5.mp4
```

#### Bash functions

Add to `~/.bashrc` for convenient media playback:

```bash
mpvf() { mpv "$(fname "$1")"; }
vlcf() { vlc "$(fname "$1")"; }
```

Then:

```bash
mpvf e10        # play by ID
mpvf test_5     # play by partial filename
mpvf 33         # play by number
```

### `fid` — Filename to ID

Given a filename or path, returns the Evidence ID. The reverse of `idf`.

```bash
fid "SegCam Trelew.mp4"           # → E0010
fid /full/path/to/file.mp4         # → E0042
```

### `evidence-queue` — Task queue

Manage the parallel task queue for long-running file processing.

```bash
evidence-queue launch              # start processing pending jobs
evidence-queue launch --workers 4  # with 4 parallel workers
evidence-queue status              # show running + pending commands
evidence-queue log                 # show full history
evidence-queue cancel              # stop after current jobs finish
```

### `evidence-obsidian` — Obsidian vault generator

Generate an [Obsidian](https://obsidian.md) vault from the Evidence database with thumbnails and rich metadata notes.

```bash
evidence-obsidian                  # build vault in data_path/obsidian/
```

Generates:
- JPEG thumbnails for images and videos (parallel, skips existing)
- Markdown notes with YAML frontmatter (tags, categories, EXIF, format info)
- Clickable file links (video files open with gotime/mpv, others with OS default)

Requires `ffmpeg` and `Pillow`.

---

## Queue system

Queue long-running tasks from Python, run them in parallel via CLI, and merge results back on completion. Resumable after shutdown.

```python
ev = Evidence(mode=Mode.LOAD)
ev.queue('exif',     inputL=['file__pathName'], outputL=['exif'],   query={'type': 'E'})
ev.queue('img_info', inputL=['file__pathName'], outputL=['format'], query={'file__mime__type': 'image'})
```

```bash
evidence-queue launch
```

Built-in tasks: `exif`, `snd_info`, `img_info`, `vid_info`, `hash`. Custom tasks can be added with the `@task` decorator.

See [docs/queue.md](docs/queue.md) for detailed documentation: dependency inference, custom task setup, standalone testing, file layout, and recovery.

---

## Evidence methods

Methods added by `Evidence` beyond what `Dlist` provides:

| Method | Description |
|--------|-------------|
| **Modes** | |
| `ev.scan()` | Full scan from disk: assigns IDs, type, cat, EXIF, media info |
| `ev.load(files=[], duplicates=None)` | Load saved database via DuckDB pivot join |
| `ev.rescan()` | Add new files with full descriptor pipeline |
| `ev.zero()` | Reset to empty (with backup) |
| **Lookup** | |
| `ev.idf('e42')` | Look up record by ID string (E42, A3, 42, etc.) |
| **Queue** | |
| `ev.queue('task', inputL, outputL, query)` | Add task to execution queue → returns queue ID |
| **I/O** | |
| `ev.save(wipe=False)` | Save to JSON files (one per top-level key) |
| `ev.backup(wipe=False)` | Zip backup of database directory |
| `ev.readFromDisk()` | Walk evidence directory, return raw file list |
| `ev.load_xtras()` | Load extra data from DATAPATH |
| **Utilities** | |
| `ev.fileAttr(ls)` | Build file attribute dicts (mime, hash, size, etc.) |
| `ev.mkid('E', 42)` | Format an ID string (E0042) |
| **Properties** | |
| `ev.evidence_path` | Path to evidence files root |
| `ev.data_path` | Path to data/database directory |

---

## Quick reference

| Command | Description |
|---------|-------------|
| **CLI** | |
| `evidence-setup` | Manage profiles (add, edit, remove, select, list) |
| `evidence-build-database` | Build/rebuild database (scan, rescan, load) |
| `evidence-show` | View database contents (files, structure) |
| `idf e10` | Resolve Evidence ID → full pathname |
| `fid file.mp4` | Resolve filename → Evidence ID |
| `fname test_5` | Resolve any input → full pathname |
| **Python** | |
| `Evidence(mode=Mode.SCAN)` | Full scan with metadata extraction |
| `Evidence(mode=Mode.LOAD)` | Load existing database |
| `Evidence(mode=Mode.RESCAN)` | Add new files to database |
| `Evidence(mode=Mode.ZERO)` | Empty database (with backup) |
| `ev.idf('e42')` | Look up by ID |
| `ev.filter(type='E')` | Filter evidence items (inherited from Dlist) |
| `ev.save()` | Save database to JSON files |
| **Queue** | |
| `ev.queue('exif', ...)` | Queue a task for background processing |
| `evidence-queue launch` | Run pending queue jobs (parallel) |
| `evidence-queue status` | Show running + pending commands |
| `evidence-queue log` | Show full queue history |
| `evidence-queue cancel` | Stop after current jobs |
| `evidence-obsidian` | Generate Obsidian vault with thumbnails and notes |

---

## Project structure

```
evidence/
├── src/evidence/
│   ├── __init__.py        # Package exports (lazy imports)
│   ├── build.py           # evidence-build-database CLI
│   ├── config.py          # TOML config + Rich/Typer CLI
│   ├── main.py            # Evidence class (Dlist-based)
│   ├── descriptors.py     # File metadata extraction
│   ├── fid.py             # Filename → ID resolver (DuckDB)
│   ├── filename.py        # Filename abstraction
│   ├── fname.py           # Universal filename resolver (DuckDB)
│   ├── formatting.py      # Display formatting
│   ├── idf.py             # ID resolver (DuckDB)
│   ├── obsidian.py        # Obsidian vault generator
│   ├── show.py            # evidence-show CLI
│   ├── tasks.py           # Task registry (@task decorator)
│   ├── queue.py           # Queue engine (jobs, runner, merge)
│   └── queue_cli.py       # evidence-queue CLI
├── docs/
│   ├── queue.md           # Queue system documentation
│   └── windows-setup.md   # Windows installation guide
├── tests/
│   ├── test_evidence.py   # Core test suite
│   └── test_queue.py      # Queue system tests
└── pyproject.toml
```

## License

MIT
