Metadata-Version: 2.4
Name: mdb-engine
Version: 0.11.5
Summary: MongoDB Engine
Home-page: https://github.com/ranfysvalle02/mdb-engine
Author: Fabian Valle
Author-email: Fabian Valle <oblivio.company@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/ranfysvalle02/mdb-engine
Project-URL: Documentation, https://github.com/ranfysvalle02/mdb-engine#readme
Project-URL: Repository, https://github.com/ranfysvalle02/mdb-engine
Project-URL: Issues, https://github.com/ranfysvalle02/mdb-engine/issues
Keywords: mongodb,runtime,engine,database,scoping
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: motor<4.0.0,>=3.0.0
Requires-Dist: pymongo<5.0.0,>=4.0.0
Requires-Dist: fastapi<1.0.0,>=0.100.0
Requires-Dist: pydantic<3.0.0,>=2.0.0
Requires-Dist: pyjwt>=2.8.0
Requires-Dist: jsonschema>=4.0.0
Requires-Dist: bcrypt>=4.0.0
Requires-Dist: cryptography>=41.0.0
Requires-Dist: casbin>=1.0.0
Requires-Dist: casbin-motor-adapter>=0.1.0
Provides-Extra: ai
Requires-Dist: openai<3.0.0,>=1.0.0; extra == "ai"
Requires-Dist: google-genai>=1.0.0; extra == "ai"
Requires-Dist: voyageai>=0.3.0; extra == "ai"
Requires-Dist: semantic-text-splitter>=0.9.0; extra == "ai"
Provides-Extra: tokens
Requires-Dist: tiktoken>=0.5.0; extra == "tokens"
Provides-Extra: osi
Requires-Dist: pyyaml>=6.0.0; extra == "osi"
Requires-Dist: jinja2>=3.1.0; extra == "osi"
Provides-Extra: oauth
Requires-Dist: authlib>=1.3.0; extra == "oauth"
Requires-Dist: httpx>=0.27.0; extra == "oauth"
Provides-Extra: otel
Requires-Dist: opentelemetry-api>=1.20.0; extra == "otel"
Requires-Dist: opentelemetry-sdk>=1.20.0; extra == "otel"
Requires-Dist: opentelemetry-exporter-otlp>=1.20.0; extra == "otel"
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.41b0; extra == "otel"
Requires-Dist: opentelemetry-instrumentation-pymongo>=0.41b0; extra == "otel"
Provides-Extra: casbin
Provides-Extra: oso
Requires-Dist: oso-cloud>=0.1.0; extra == "oso"
Provides-Extra: og-image
Requires-Dist: Pillow>=10.0.0; extra == "og-image"
Provides-Extra: test
Requires-Dist: pytest>=7.4.0; extra == "test"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "test"
Requires-Dist: pytest-cov>=4.1.0; extra == "test"
Requires-Dist: pytest-mock>=3.11.0; extra == "test"
Requires-Dist: pytest-timeout>=2.1.0; extra == "test"
Requires-Dist: pytest-xdist>=3.3.0; extra == "test"
Requires-Dist: testcontainers>=3.7.0; extra == "test"
Requires-Dist: nest-asyncio>=1.5.0; extra == "test"
Requires-Dist: python-dateutil>=2.8.2; extra == "test"
Requires-Dist: pyyaml>=6.0.0; extra == "test"
Provides-Extra: dev
Requires-Dist: ruff<0.6.0,>=0.4.0; extra == "dev"
Requires-Dist: semgrep>=1.50.0; extra == "dev"
Requires-Dist: mypy>=1.10.0; extra == "dev"
Requires-Dist: bandit>=1.7.0; extra == "dev"
Requires-Dist: pip-audit>=2.6.0; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.6.0; extra == "docs"
Requires-Dist: mkdocs-material>=9.5.0; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.25.0; extra == "docs"
Provides-Extra: all
Requires-Dist: openai<3.0.0,>=1.0.0; extra == "all"
Requires-Dist: google-genai>=1.0.0; extra == "all"
Requires-Dist: voyageai>=0.3.0; extra == "all"
Requires-Dist: semantic-text-splitter>=0.9.0; extra == "all"
Requires-Dist: casbin>=1.0.0; extra == "all"
Requires-Dist: casbin-motor-adapter>=0.1.0; extra == "all"
Requires-Dist: oso-cloud>=0.1.0; extra == "all"
Requires-Dist: authlib>=1.3.0; extra == "all"
Requires-Dist: httpx>=0.27.0; extra == "all"
Requires-Dist: opentelemetry-api>=1.20.0; extra == "all"
Requires-Dist: opentelemetry-sdk>=1.20.0; extra == "all"
Requires-Dist: opentelemetry-exporter-otlp>=1.20.0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-fastapi>=0.41b0; extra == "all"
Requires-Dist: opentelemetry-instrumentation-pymongo>=0.41b0; extra == "all"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# mdb-engine

**The MongoDB Engine for Python Apps** — Auto-sandboxing, index management, and AI services in one package.

[![PyPI](https://img.shields.io/pypi/v/mdb-engine)](https://pypi.org/project/mdb-engine/)
[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## Get Running in 60 Seconds

### Step 1: Start MongoDB

You need a running MongoDB instance. Pick one:

```bash
# Local (Docker) — full Atlas features including Vector Search
docker run -d --name mongodb -p 27017:27017 mongodb/mongodb-atlas-local:latest
```

Or use [MongoDB Atlas](https://www.mongodb.com/cloud/atlas) (free tier):

```bash
export MONGODB_URI="mongodb+srv://user:pass@cluster.mongodb.net/"
```

### Step 2: Install and run

```bash
pip install mdb-engine fastapi uvicorn
```

Create `web.py`:

```python
from mdb_engine import quickstart
from mdb_engine.dependencies import get_scoped_db
from fastapi import Depends

app = quickstart("my_app")

@app.get("/items")
async def list_items(db=Depends(get_scoped_db)):
    return await db.items.find({}).to_list(10)
```

```bash
uvicorn web:app --reload
```

Open [http://localhost:8000/items](http://localhost:8000/items) -- you're live.

> **Using AI features?** You'll also need an API key:
> `export OPENAI_API_KEY=sk-...` (or `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`).
> See [Environment Variables](#environment-variables) for the full list.

**What you get automatically:**

- **Data isolation** — every query is scoped by `app_id`; you cannot accidentally leak data across apps
- **Collection prefixing** — `db.items` transparently becomes `my_app_items`
- **Lifecycle management** — engine startup/shutdown handled for you
- **Dependency injection** — `get_scoped_db`, `get_memory_service`, etc. ready to use

---

## Installation

```bash
pip install mdb-engine
```

---

## Feature Layers

mdb-engine is designed for progressive adoption. Start with Layer 0 and add features as you need them.

| Layer | What it gives you | How to enable |
|-------|-------------------|---------------|
| **0: Scoped DB + Indexes** | Auto-sandboxed collections, declarative indexes | `quickstart("slug")` or minimal manifest |
| **1: Auth + GDPR** | JWT, RBAC (Casbin/OSO), SSO, data export/deletion | Add `auth` section to manifest |
| **2: LLM + Embeddings + Memory** | Persistent AI memory, semantic search, fact extraction | Add `llm_config` + `memory_config` to manifest |
| **3: GraphRAG + ChatEngine** | Knowledge graphs, conversation orchestration with STM + LTM | Add `graph_config`, use `ChatEngine` |

---

## Three Ways to Create an App

### 1. Zero-config (quickstart)

No manifest file, no explicit connection string. Best for getting started.

```python
from mdb_engine import quickstart

app = quickstart("my_app")
```

### 2. Inline manifest (dict)

Pass configuration directly in Python. Good for programmatic setups.

```python
from mdb_engine import MongoDBEngine

engine = MongoDBEngine()
app = engine.create_app(
    slug="my_app",
    manifest={
        "schema_version": "2.0",
        "slug": "my_app",
        "name": "My App",
        "managed_indexes": {
            "tasks": [{"type": "regular", "keys": {"status": 1}, "name": "status_idx"}]
        },
    },
)
```

### 3. File-based manifest (recommended for production)

The full-featured approach. A single `manifest.json` defines your app's identity, indexes, auth, AI services, and more.

```python
from pathlib import Path
from mdb_engine import MongoDBEngine

engine = MongoDBEngine(
    mongo_uri="mongodb+srv://...",  # or set MONGODB_URI env var
    db_name="production"
)
app = engine.create_app(slug="my_app", manifest=Path("manifest.json"))
```

**Minimal manifest.json (3 fields):**

```json
{
  "schema_version": "2.0",
  "slug": "my_app",
  "name": "My App"
}
```

**Learn more**: [Manifest Reference](docs/MANIFEST_REFERENCE.md) | [Quick Start Guide](docs/QUICK_START.md)

### 4. Zero-Code Manifest (no Python at all)

A single `manifest.json` defines a production-grade REST API — auth, CRUD, hooks,
relations, computed fields, unique constraints, TTL — without writing a line of Python.

```bash
mdb-engine serve manifest.json
```

Thirteen declarative collection primitives:

| Primitive | What it does |
|-----------|-------------|
| `schema` | JSON Schema validation on every write |
| `defaults` | Auto-populate fields on create (`{{user.*}}`, `$$NOW`) |
| `owner_field` | Auto-inject creator ID, enforce ownership, admin bypass |
| `immutable_fields` | Silently strip protected fields from updates |
| `hooks` | Fire-and-forget side effects (`after_create`, `after_update`, `after_delete`) |
| `relations` | Declarative joins via `?populate=name` |
| `computed` | Virtual fields via `?computed=name` (aggregation at read time) |
| `scopes` | Named MQL filters via `?scope=name` (with optional auth gates) |
| `pipelines` | Named aggregation endpoints at `/_agg/{name}` |
| `policy` | Document-level read/write/delete access policies |
| `x-unique` | Schema-driven unique indexes (409 on duplicates) |
| `ttl` | Auto-expiring documents (`"expire_after": "90d"`) |
| `auth` | Role-based access for reads, creates, and mutations |

**Real-world example** — a blog post collection with audit trail, computed comment
counts, and ownership enforcement:

```json
{
  "posts": {
    "auto_crud": true,
    "soft_delete": true,
    "owner_field": "author_id",
    "immutable_fields": ["author_id"],
    "auth": { "write_roles": ["admin"] },
    "defaults": { "status": "draft", "author": "{{user.email}}" },
    "scopes": { "published": { "status": "published" } },
    "hooks": {
      "after_create": [{ "action": "insert", "collection": "audit_log", "document": {
        "event": "post_created", "entity_id": "{{doc._id}}", "actor": "{{user.email}}", "timestamp": "$$NOW"
      }}]
    },
    "computed": {
      "comment_count": { "pipeline": [
        { "$lookup": { "from": "comments", "let": { "pid": { "$toString": "$_id" } },
          "pipeline": [{ "$match": { "$expr": { "$eq": ["$post_id", "$$pid"] }}}], "as": "_c" }},
        { "$addFields": { "comment_count": { "$size": "$_c" } } },
        { "$project": { "_c": 0 } }
      ]}
    }
  }
}
```

**See it in action**: [zero_code_api](examples/basic/zero_code_api/) — a complete blog with auth, comments, categories, and audit log.

---

## Examples Ladder

Start simple, add complexity when you need it.

| Example | What it shows | Lines of Python |
|---------|---------------|:---:|
| [zero_code_api](examples/basic/zero_code_api/) | Blog with auth, hooks, relations, computed fields, unique, TTL — **all manifest** | **0** |
| [hello_world](examples/basic/hello_world/) | Zero-config CRUD, no manifest | ~15 |
| [memory_quickstart](examples/basic/memory_quickstart/) | AI memory with semantic search | ~25 |
| [chit_chat](examples/basic/chit_chat/) | Full AI chat with ChatEngine, auth, WebSockets | ~2400 |
| [interactive_rag](examples/basic/interactive_rag/) | RAG with vector search | — |
| [simple_app](examples/advanced/simple_app/) | Task management with `create_app()` pattern | — |
| [sso-multi-app](examples/advanced/sso-multi-app/) | SSO with shared user pool across apps | — |

---

## CRUD Operations (Auto-Scoped)

All database operations are automatically scoped to your app:

```python
from mdb_engine.dependencies import get_scoped_db

@app.post("/tasks")
async def create_task(task: dict, db=Depends(get_scoped_db)):
    result = await db.tasks.insert_one(task)
    return {"id": str(result.inserted_id)}

@app.get("/tasks")
async def list_tasks(db=Depends(get_scoped_db)):
    return await db.tasks.find({"status": "pending"}).to_list(length=10)
```

**Under the hood:**

```python
# You write:
await db.tasks.find({}).to_list(length=10)

# Engine executes:
# Collection: my_app_tasks
# Query: {"app_id": "my_app"}
```

---

## AI Memory (MemoryService)

Add persistent, searchable AI memory to any app.

```python
from mdb_engine.dependencies import get_memory_service

@app.post("/remember")
async def remember(text: str, memory=Depends(get_memory_service)):
    result = await memory.add(messages=text, user_id="user1")
    return {"stored": result}

@app.get("/recall")
async def recall(q: str, memory=Depends(get_memory_service)):
    results = await memory.search(query=q, user_id="user1")
    return {"results": results}
```

> **Note**: All memory operations are async. Use `await` directly in your routes.

Enable in your manifest:

```json
{
  "llm_config": {"enabled": true, "default_model": "openai/gpt-4o-mini"},
  "embedding_config": {"enabled": true, "default_embedding_model": "text-embedding-3-small"},
  "memory_config": {"enabled": true, "provider": "cognitive", "infer": true}
}
```

Optional cognitive features (add to `memory_config`):

- **Importance scoring**: AI evaluates memory significance
- **Memory reinforcement**: Similar memories strengthen each other
- **Memory decay**: Less relevant memories fade over time
- **Memory merging**: Related memories combined intelligently

---

## ChatEngine (Conversation Orchestration)

`ChatEngine` (formerly `CognitiveEngine`) orchestrates full conversations with short-term memory (chat history) + long-term memory (semantic search):

```python
from mdb_engine.memory import ChatEngine

cognitive = ChatEngine(
    app_slug="my_app",
    memory_service=memory_service,
    chat_history_collection=db.chat_history,
    llm_provider=llm_provider,
)

result = await cognitive.chat(
    user_id="user123",
    session_id="session456",
    user_query="What did we discuss about the project?",
    system_prompt="You are a helpful assistant.",
    extract_facts=True,
)
```

---

## RequestContext (All Services in One Place)

```python
from mdb_engine import RequestContext, get_request_context

@app.post("/ai-chat")
async def chat(query: str, ctx: RequestContext = Depends(get_request_context)):
    user = ctx.require_user()       # 401 if not logged in
    ctx.require_role("user")        # 403 if missing role

    # ctx.db, ctx.memory, ctx.llm, ctx.embedding_service — all available
    if ctx.llm:
        response = ctx.llm.chat.completions.create(
            model=ctx.llm_model,
            messages=[{"role": "user", "content": query}]
        )
        return {"response": response.choices[0].message.content}
```

---

## Index Management

Define indexes in `manifest.json` — they're auto-created on startup:

```json
{
  "managed_indexes": {
    "tasks": [
      {"type": "regular", "keys": {"status": 1, "created_at": -1}, "name": "status_sort"},
      {"type": "regular", "keys": {"email": 1}, "name": "email_unique", "unique": true}
    ]
  }
}
```

Supported index types: `regular`, `text`, `vector`, `ttl`, `compound`.

---

## Advanced Features

| Feature | Description | Learn More |
|---------|-------------|------------|
| **Authentication** | JWT + Casbin/OSO RBAC | [Bible: Auth](MDB_ENGINE_BIBLE.md#authentication-and-authorization) |
| **Vector Search** | Atlas Vector Search + embeddings | [RAG Example](examples/basic/interactive_rag) |
| **MemoryService** | Persistent AI memory with cognitive features | [Memory Docs](docs/MEMORY_SERVICE.md) |
| **GraphService** | Knowledge graph with `$graphLookup` traversal | [Graph Docs](docs/GRAPH_SERVICE.md) |
| **ChatEngine** | Full RAG pipeline with STM + LTM | [Chat Example](examples/basic/chit_chat) |
| **WebSockets** | Real-time updates from manifest config | [Bible: WebSockets](MDB_ENGINE_BIBLE.md#websocket-system) |
| **Multi-App** | Secure cross-app data access | [SSO Example](examples/advanced/sso-multi-app) |
| **SSO** | Shared auth across apps | [SSO Example](examples/advanced/sso-multi-app) |
| **GDPR** | Data discovery, export, deletion, rectification | [Bible: GDPR](MDB_ENGINE_BIBLE.md#gdpr-compliance) |

---

## MongoDB Connection Reference

mdb-engine connects to `mongodb://localhost:27017` by default. Override via env var or constructor:

| Setup | Connection String |
|-------|-------------------|
| Local / Docker | `mongodb://localhost:27017` (default, no config needed) |
| Atlas (free tier) | `mongodb+srv://user:password@cluster.mongodb.net/dbname` |

```bash
# Option A: environment variable
export MONGODB_URI="mongodb+srv://user:password@cluster.mongodb.net/"

# Option B: in code
engine = MongoDBEngine(mongo_uri="mongodb+srv://...")
```

---

## Environment Variables

| Variable | Required | Purpose |
|----------|----------|---------|
| `MONGODB_URI` | No | MongoDB connection string (default: `localhost:27017`) |
| `MDB_DB_NAME` | No | Database name (default: `mdb_engine`) |
| `OPENAI_API_KEY` | For AI features | OpenAI API key for LLM/embeddings |
| `ANTHROPIC_API_KEY` | For AI features | Anthropic API key (alternative to OpenAI) |
| `GEMINI_API_KEY` | For AI features | Google Gemini API key (alternative to OpenAI) |
| `MDB_JWT_SECRET` | For auth | JWT signing secret for shared auth mode |

---

## Why mdb-engine?

- **Zero to running in 3 lines** — `quickstart("my_app")` and go
- **Data isolation built in** — multi-tenant ready with automatic app sandboxing
- **manifest.json is everything** — single source of truth for your app config
- **Incremental adoption** — start minimal, add features as needed
- **No lock-in** — standard Motor/PyMongo underneath; export with `mongodump --query='{"app_id":"my_app"}'`

---

## Links

- [GitHub Repository](https://github.com/ranfysvalle02/mdb-engine)
- [Documentation](docs/)
- [All Examples](examples/)
- [Quick Start Guide](docs/QUICK_START.md) — **Start here!**
- [Manifest Reference](docs/MANIFEST_REFERENCE.md)
- [Contributing](CONTRIBUTING.md)

---

**Stop building scaffolding. Start building features.**
