Metadata-Version: 2.1
Name: postgres-backup-plugin
Version: 1.0.1
Summary: Framework-agnostic PostgreSQL backup library with filtering support
Author-email: Võ Thiên Hòa <vothienhoa@gmail.com>
Maintainer-email: Võ Thiên Hòa <vothienhoa@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/vohoa/postgres_backup_plugin
Project-URL: Documentation, https://github.com/vohoa/postgres_backup_plugin#readme
Project-URL: Repository, https://github.com/vohoa/postgres_backup_plugin
Project-URL: Source Code, https://github.com/vohoa/postgres_backup_plugin
Project-URL: Bug Tracker, https://github.com/vohoa/postgres_backup_plugin/issues
Project-URL: Changelog, https://github.com/vohoa/postgres_backup_plugin/blob/main/CHANGELOG.md
Keywords: postgresql,postgres,backup,database,sql,copy,django,filter,export,s3,restore,dump
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Topic :: Database
Classifier: Topic :: Database :: Database Engines/Servers
Classifier: Topic :: System :: Archiving :: Backup
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Classifier: Framework :: Django
Classifier: Framework :: Django :: 3.0
Classifier: Framework :: Django :: 4.0
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: psycopg2-binary>=2.8.0
Provides-Extra: all
Requires-Dist: boto3>=1.20.0; extra == "all"
Requires-Dist: django>=3.0; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=7.0.0; extra == "dev"
Requires-Dist: pytest-cov>=3.0.0; extra == "dev"
Requires-Dist: pytest-mock>=3.10.0; extra == "dev"
Requires-Dist: black>=22.0.0; extra == "dev"
Requires-Dist: flake8>=4.0.0; extra == "dev"
Requires-Dist: mypy>=0.990; extra == "dev"
Requires-Dist: isort>=5.10.0; extra == "dev"
Provides-Extra: django
Requires-Dist: django>=3.0; extra == "django"
Provides-Extra: s3
Requires-Dist: boto3>=1.20.0; extra == "s3"

# PostgreSQL Backup Plugin

Framework-agnostic PostgreSQL backup library with advanced filtering support.

## Features

- ✅ **Direct Streaming**: No temp files, minimal RAM usage (works with TB-sized tables)
- ✅ **COPY Format**: Fast backup and restore using PostgreSQL COPY
- ✅ **Flexible Filtering**: Filter data per table with reusable filter classes
- ✅ **Multiple Exporters**: Local file, S3, or custom destinations
- ✅ **Framework-Agnostic**: Works with Django, Flask, FastAPI, or pure Python
- ✅ **Production Ready**: Error handling, logging, validation

## Installation

```bash
# Basic installation
pip install psycopg2-binary

# For S3 support
pip install psycopg2-binary boto3

# For Django integration
pip install psycopg2-binary django
```

## Quick Start

### Basic Usage (Pure Python)

```python
from postgres_backup_plugin import PostgresBackupEngine, DatabaseConfig

# Configure database
db_config = DatabaseConfig(
    host='localhost',
    port=5432,
    user='postgres',
    password='secret',
    database='mydb'
)

# Create backup engine
engine = PostgresBackupEngine(db_config)

# Create backup
result = engine.backup('/tmp/backup.sql')

if result.success:
    print(f"Backup created: {result.file_path}")
    print(f"Size: {result.size_bytes} bytes")
    print(f"Tables: {result.tables_count}")
    print(f"Duration: {result.duration_seconds}s")
else:
    print(f"Backup failed: {result.error_message}")
```

### Django Integration

```python
from postgres_backup_plugin import PostgresBackupEngine

# Automatically use Django database settings
engine = PostgresBackupEngine.from_django_settings(
    excluded_tables=['django_session', 'django_admin_log']
)

result = engine.backup('/tmp/backup.sql')
```

### With Filtering

```python
from postgres_backup_plugin import (
    PostgresBackupEngine,
    DateRangeFilter,
    ForeignKeyFilter,
    StatusFilter
)

engine = PostgresBackupEngine.from_django_settings()

# Define filters per table
filters = {
    # Filter orders by customer IDs
    'orders': ForeignKeyFilter('customer_id', [123, 456, 789]),

    # Filter invoices by date range
    'invoices': DateRangeFilter('created_at', '2024-01-01', '2024-12-31'),

    # Filter users by status
    'users': StatusFilter('status', allowed_statuses=['active', 'pending']),

    # Raw SQL also works
    'products': 'SELECT * FROM products WHERE price > 100'
}

result = engine.backup(
    output_path='/tmp/filtered_backup.sql',
    filters=filters,
    schema_name='my_backup',
    metadata={'purpose': 'customer_migration', 'ticket': 'PROJ-123'}
)
```

### With S3 Export

```python
from postgres_backup_plugin import PostgresBackupEngine, S3Exporter

engine = PostgresBackupEngine.from_django_settings()

# Create backup
result = engine.backup('/tmp/backup.sql')

if result.success:
    # Export to S3
    exporter = S3Exporter(
        bucket='my-backups',
        prefix='postgres/deliveries/',
        delete_local=True  # Delete local file after upload
    )

    s3_url = exporter.export(result.file_path, metadata=result.metadata)
    print(f"Uploaded to: {s3_url}")
```

## Advanced Usage

### Custom Filters

Create reusable filter classes:

```python
from postgres_backup_plugin.filters import FilterQuery

class AgeRangeFilter(FilterQuery):
    def __init__(self, min_age, max_age):
        self.min_age = min_age
        self.max_age = max_age

    def build(self, table_name, **params):
        return f"""
            SELECT * FROM {table_name}
            WHERE age BETWEEN {self.min_age} AND {self.max_age}
        """

# Use it
filters = {
    'users': AgeRangeFilter(18, 65)
}
```

### Composite Filters

Combine multiple filters:

```python
from postgres_backup_plugin.filters import CompositeFilter

date_filter = DateRangeFilter('created_at', '2024-01-01', '2024-12-31')
status_filter = StatusFilter('status', ['active'])

# Combine with AND
composite = CompositeFilter(date_filter, status_filter, operator='AND')

filters = {
    'orders': composite
}
```

### Validate Filters Before Backup

```python
engine = PostgresBackupEngine.from_django_settings()

filters = {
    'orders': 'SELECT * FROM orders WHERE invalid_column = 123'
}

try:
    engine.validate_filters(filters)
except FilterValidationError as e:
    print(f"Invalid filter: {e}")
```

### Estimate Backup Size

```python
engine = PostgresBackupEngine.from_django_settings()

estimates = engine.estimate_size(filters)

for table, row_count in estimates.items():
    print(f"{table}: {row_count} rows")
```

## Configuration

### Database Config

```python
from postgres_backup_plugin import DatabaseConfig

db_config = DatabaseConfig(
    host='localhost',
    port=5432,
    user='postgres',
    password='secret',
    database='mydb'
)
```

### Backup Config

```python
from postgres_backup_plugin import BackupConfig

backup_config = BackupConfig(
    excluded_tables=['temp_table', 'cache_table'],
    buffer_size=8192,
    timeout=3600,  # seconds
    encoding='utf-8',
    disable_triggers=True,  # Faster restore
    disable_fsync=True,  # Faster restore
    include_header=True,
    verbose_logging=True
)

engine = PostgresBackupEngine(db_config, backup_config)
```

## Restore Backup

To restore a backup created by this plugin:

```bash
# Simple restore
psql -U postgres -d mydb < backup.sql

# With specific schema
psql -U postgres -d mydb -f backup.sql

# Remote database
psql -h remote-host -U postgres -d mydb < backup.sql
```

## Performance

### Benchmark (1GB table, 10 million rows)

| Method | Backup Time | RAM Usage | Disk I/O |
|--------|-------------|-----------|----------|
| **This Plugin** | ~30-35s | ~50MB | 1GB |
| INSERT statements | ~120s | ~50MB | 1GB |
| pg_dump (custom) | ~40s | ~50MB | 1GB |
| Temp CSV method | ~45s | ~50MB | 2GB |

### Why Fast?

- Direct streaming from PostgreSQL to file
- COPY format (native PostgreSQL bulk export)
- No intermediate files
- Minimal memory usage

## Architecture

```
postgres_backup_plugin/
├── core/
│   ├── backup_engine.py      # Main backup logic
│   ├── stream_wrapper.py     # Direct streaming
│   └── query_builder.py      # SQL utilities
├── filters/
│   ├── base.py               # Filter interface
│   └── common_filters.py     # Pre-built filters
├── exporters/
│   ├── base.py               # Exporter interface
│   ├── file_exporter.py      # Local file
│   └── s3_exporter.py        # AWS S3
├── config.py                 # Configuration classes
└── exceptions.py             # Custom exceptions
```

## Requirements

- Python 3.7+
- PostgreSQL 9.5+
- psycopg2 or psycopg2-binary
- boto3 (optional, for S3 export)

## License

MIT License

## Contributing

Contributions welcome! Please open an issue or PR.
