Metadata-Version: 2.4
Name: vd3
Version: 0.1.0
Summary: Content database library and CLI for VisData 3
Project-URL: Homepage, https://github.com/jmuncaster/vd3
Project-URL: Repository, https://github.com/jmuncaster/vd3
Project-URL: Issues, https://github.com/jmuncaster/vd3/issues
Author-email: Justin Muncaster <justin@muncasterconsulting.com>
License-Expression: MIT
Keywords: computer-vision,dataset,dvc,foxglove,mcap,video
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Requires-Dist: duckdb>=1.1.0
Requires-Dist: dvc-azure>=3.0
Requires-Dist: dvc-gs>=3.0
Requires-Dist: dvc-s3>=3.0
Requires-Dist: dvc>=3.50.0
Requires-Dist: ffmpeg-python>=0.2.0
Requires-Dist: foxglove-schemas-protobuf>=0.3.0
Requires-Dist: mcap-protobuf-support>=0.5.4
Requires-Dist: mcap>=1.1.0
Requires-Dist: orjson>=3.10.0
Requires-Dist: pillow>=10.0.0
Requires-Dist: protobuf>=5.0
Requires-Dist: pydantic>=2.0
Requires-Dist: rich>=13.0.0
Requires-Dist: typer>=0.12.0
Description-Content-Type: text/markdown

# VD3Storage

Content database library and CLI for VisData 3. Manages video assets, annotations, and workspaces backed by MCAP files and CSV-based metadata.

## Installation

```bash
uv sync
```

To use as a dependency in a content database project:

```toml
# pyproject.toml
[project]
dependencies = ["vd3storage"]

[tool.uv.sources]
vd3storage = { path = "./vd3storage", editable = true }
```

## Quick Start

```bash
# Initialize a new content database
vd3 init /path/to/mydb

# Add a video
vd3 add video clip.mp4 -c my-collection -p /path/to/mydb

# Add multiple videos with a glob
vd3 add video '*.mp4' -c my-collection -p /path/to/mydb

# List assets
vd3 list assets -p /path/to/mydb

# Check status
vd3 status -p /path/to/mydb
```

## Remote Storage

VD3Storage uses [DVC](https://dvc.org) under the hood to push and pull large media files to/from remote storage.

### Google Cloud Storage

1. Authenticate with GCS:

   ```bash
   gcloud auth application-default login
   ```

2. Add a GCS remote:

   ```bash
   vd3 remote add gcs gs://my-bucket/vd3-data -p /path/to/mydb
   ```

3. Push media to remote:

   ```bash
   vd3 push --all -p /path/to/mydb
   ```

4. Pull media from remote (e.g. on another machine):

   ```bash
   vd3 pull --all -p /path/to/mydb
   ```

### Amazon S3

```bash
# Ensure AWS credentials are configured (aws configure, env vars, etc.)
vd3 remote add s3remote s3://my-bucket/vd3-data -p /path/to/mydb
```

### Local / NAS

```bash
vd3 remote add backup /mnt/nas/vd3-backup -p /path/to/mydb
```

### Listing Remotes

```bash
vd3 remote list -p /path/to/mydb
```

## Workspaces

Workspaces let you organize subsets of assets into named groups with optional packages (folders).

```bash
# Create a workspace
vd3 workspace create "My Experiment" -p /path/to/mydb

# Add assets by name, ID, or media file glob
vd3 workspace add-asset my-experiment 'db/media/videos/fc/*.mcap' -p /path/to/mydb

# List workspaces
vd3 workspace list -p /path/to/mydb

# Show workspace details
vd3 workspace show my-experiment -p /path/to/mydb
```

## Adding Videos

```bash
# Single file
vd3 add video clip.mp4 -c dashcam

# Glob pattern (quote to prevent shell expansion)
vd3 add video '*.mp4' -c dashcam

# Recursive glob
vd3 add video 'rawdata/**/*.mp4' -c dashcam

# Force re-import of a duplicate
vd3 add video clip.mp4 -c dashcam --force

# Add and assign to a workspace
vd3 add video clip.mp4 -c dashcam -w my-workspace -k batch1
```

Duplicate detection uses SHA-256 hashing of the source file. Use `--force` to overwrite an existing asset with the same content.

## Adding Results

Import VD3 JSON detection/track results into an asset's MCAP:

```bash
vd3 add result results.json -a clip -p /path/to/mydb
```

## Viewing in Foxglove

MCAP files are written with `foxglove.CompressedImage` protobuf messages on the `/video/frames` topic, compatible with [Foxglove Studio](https://foxglove.dev) and Lichtblick.

Open any `.mcap` file under `db/media/videos/` directly in Foxglove to view the video and any annotation overlays.

## CLI Reference

```
vd3 --help              Top-level help
vd3 <command> --help    Help for a specific command
```

| Command | Description |
|---|---|
| `init` | Initialize a new content database |
| `info` | Show database information |
| `add video` | Import video files |
| `add result` | Import detection/track results |
| `list assets` | List assets |
| `list collections` | List collections |
| `show` | Show asset details |
| `query` | Run raw SQL against the database |
| `remove` | Delete an asset |
| `workspace create` | Create a workspace |
| `workspace add-asset` | Add assets to a workspace |
| `workspace remove-asset` | Remove an asset from a workspace |
| `workspace list` | List workspaces |
| `workspace show` | Show workspace details |
| `workspace delete` | Delete a workspace |
| `status` | Show media availability (use `-l` to list all) |
| `push` | Push media to remote storage |
| `pull` | Pull media from remote storage |
| `remote add` | Add a remote storage backend |
| `remote list` | List configured remotes |
| `export` | Export data from the database |
