Metadata-Version: 2.4
Name: cemaf
Version: 0.2.1
Summary: Context Engineering Multi-Agent Framework - A pluggable, modular framework for building AI agent systems
Project-URL: Homepage, https://github.com/drchinca/cemaf
Project-URL: Documentation, https://github.com/drchinca/cemaf/blob/main/docs/README.md
Project-URL: Repository, https://github.com/drchinca/cemaf
Project-URL: Bug Tracker, https://github.com/drchinca/cemaf/issues
Project-URL: Discussions, https://github.com/drchinca/cemaf/discussions
Project-URL: Discord Community, https://discord.gg/C8ZXAbD8
Project-URL: Changelog, https://github.com/drchinca/cemaf/blob/main/CHANGELOG.md
Project-URL: Source Code, https://github.com/drchinca/cemaf
Author-email: Hikuri Bado Chinca <chincadr@gmail.com>
License: MIT
License-File: LICENSE
Keywords: agents,ai,autogen,context-engineering,context-management,crewai,dag,framework-agnostic,langchain,langgraph,llm,multi-agent,orchestration,provenance,replay,token-budgeting
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.14
Requires-Dist: aiosqlite>=0.22.1
Requires-Dist: pydantic>=2.0
Requires-Dist: structlog>=25.5.0
Requires-Dist: typing-extensions>=4.9
Provides-Extra: all
Requires-Dist: aiosqlite>=0.20; extra == 'all'
Requires-Dist: anthropic>=0.20; extra == 'all'
Requires-Dist: fastapi>=0.115; extra == 'all'
Requires-Dist: openai>=1.0; extra == 'all'
Requires-Dist: prometheus-client>=0.20; extra == 'all'
Requires-Dist: sse-starlette>=2.0; extra == 'all'
Requires-Dist: tiktoken>=0.5; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.34; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.20; extra == 'anthropic'
Provides-Extra: dev
Requires-Dist: bandit[toml]>=1.7; extra == 'dev'
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pre-commit>=4.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=4.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.3; extra == 'dev'
Provides-Extra: openai
Requires-Dist: openai>=1.0; extra == 'openai'
Requires-Dist: tiktoken>=0.5; extra == 'openai'
Provides-Extra: pinecone
Requires-Dist: pinecone-client>=3.0; extra == 'pinecone'
Provides-Extra: profiling
Requires-Dist: line-profiler>=4.2.0; extra == 'profiling'
Requires-Dist: memory-profiler>=0.61.0; extra == 'profiling'
Requires-Dist: pytest-benchmark>=5.1.0; extra == 'profiling'
Provides-Extra: prometheus
Requires-Dist: prometheus-client>=0.20; extra == 'prometheus'
Provides-Extra: sqlite
Requires-Dist: aiosqlite>=0.20; extra == 'sqlite'
Provides-Extra: tiktoken
Requires-Dist: tiktoken>=0.5; extra == 'tiktoken'
Provides-Extra: web
Requires-Dist: fastapi>=0.115; extra == 'web'
Requires-Dist: sse-starlette>=2.0; extra == 'web'
Requires-Dist: uvicorn[standard]>=0.34; extra == 'web'
Description-Content-Type: text/markdown

# CEMAF

**Context Engineering Multi-Agent Framework**

[![Open Source](https://img.shields.io/badge/Open%20Source-%E2%9D%A4-red?style=flat-square)](https://opensource.org)
[![Project Status: Alpha](https://img.shields.io/badge/Status-Alpha-yellow?style=flat-square)](https://github.com/drchinca/cemaf)
[![Discord](https://img.shields.io/badge/Discord-Join_Community-5865F2?style=flat-square&logo=discord&logoColor=white)](https://discord.gg/C8ZXAbD8)
[![Python](https://img.shields.io/badge/Python-3.11+-3776AB?style=flat-square&logo=python&logoColor=white)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/License-MIT-green?style=flat-square)](https://opensource.org/licenses/MIT)
[![Tests](https://img.shields.io/badge/Tests-2301_Passing-success?style=flat-square&logo=pytest&logoColor=white)](.)
[![Coverage](https://img.shields.io/badge/Coverage-80%25-brightgreen?style=flat-square)](.)
[![CI](https://img.shields.io/github/actions/workflow/status/drchinca/cemaf/ci.yml?branch=main&style=flat-square&logo=github&label=CI)](https://github.com/drchinca/cemaf/actions/workflows/ci.yml)
[![Ruff](https://img.shields.io/badge/Code_Style-Ruff-FCC21B?style=flat-square&logo=ruff&logoColor=black)](https://github.com/astral-sh/ruff)
[![MyPy](https://img.shields.io/badge/Typed-MyPy-blue?style=flat-square)](http://mypy-lang.org/)
[![Stars](https://img.shields.io/github/stars/drchinca/cemaf?style=flat-square&logo=github)](https://github.com/drchinca/cemaf)
[![Issues](https://img.shields.io/github/issues/drchinca/cemaf?style=flat-square&logo=github)](https://github.com/drchinca/cemaf/issues)
[![Open Startup](https://img.shields.io/badge/Open-Startup-00ADD8?style=flat-square)](OPEN.md)

**Open source** context engineering infrastructure that solves the hard problems in AI agent systems. CEMAF can be used standalone or plugged into existing frameworks like LangGraph, AutoGen, and CrewAI.

---

## Table of Contents

- [Overview](#overview)
- [The Hard Problems We Solve](#the-hard-problems-we-solve)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Integration Modes](#integration-modes)
- [Key Features](#key-features)
- [Documentation](#documentation)
- [Configuration](#configuration)
- [Testing](#testing)
- [Contributing](#contributing)
- [Getting Help](#getting-help)
- [Philosophy & Open Startup](#philosophy--open-startup)
- [License](#license)

---

## Overview

CEMAF is a protocol-first framework designed for **context engineering** in multi-agent AI systems. It provides:

- Token budgeting and automatic context optimization
- Deterministic run recording and replay capabilities
- Full provenance tracking for every context change
- Memory management with strict scoping and TTL
- Zero-config defaults with environment-based customization

**Philosophy**: Own the hard infrastructure problems while remaining framework-agnostic.

**Design**: Protocol-based architecture where modules work standalone. Use our defaults or replace them with your own implementations. See [Protocol Guide](docs/protocol_guide.md) for details.

---

## The Hard Problems We Solve

| Problem | What Happens | CEMAF Solution |
|---------|--------------|----------------|
| **Context Growth** | Token limits blow up | Token budgeting + automatic summarization |
| **Reliability** | Non-deterministic behavior | Patch-based provenance tracking |
| **Cost** | Wasteful token usage | Smart context compilation |
| **Reproducibility** | Can't replay/debug runs | Run recording + deterministic replay |
| **Memory Leaks** | State bleeds between scopes | Strict memory boundaries with TTL |
| **Content Safety** | Harmful outputs slip through | Pre/post-flight moderation gates + PII detection |
| **Quality Drift** | Output quality degrades silently | Online eval pipeline with rolling monitors and halt gates |
| **Prompt Engineering** | Inconsistent LLM outputs | Semantic blueprints for structured content generation |

---

## Installation

```bash
# Core installation (minimal dependencies)
pip install cemaf

# With optional integrations
pip install "cemaf[openai]"        # OpenAI + tiktoken
pip install "cemaf[anthropic]"     # Anthropic
pip install "cemaf[tiktoken]"      # Accurate token counting only
pip install "cemaf[prometheus]"    # Prometheus metrics export
pip install "cemaf[all]"           # All optional dependencies

# Development installation
git clone https://github.com/drchinca/cemaf.git
cd cemaf
pip install -e ".[dev]"
```

**Requirements**: Python 3.14+

---

## Quick Start

```python
from cemaf.context import Context, ContextPatch
from cemaf.observability import InMemoryRunLogger
from cemaf.replay import Replayer
from cemaf.blueprint import Blueprint, SceneGoal, StyleGuide
from cemaf.moderation import ModerationPipeline, PreFlightGate, KeywordRule, PIIRule

# Create context with provenance tracking
ctx = Context()
patch = ContextPatch.from_tool("search", "results", {"items": [...]})
ctx = ctx.apply(patch)

# Record runs for replay
logger = InMemoryRunLogger()
logger.start_run("run-123", initial_context=ctx)
logger.record_patch(patch)
record = logger.end_run(final_context=ctx)

# Replay deterministically
replayer = Replayer(record)
result = await replayer.replay()
assert result.final_context == record.final_context  # Deterministic!

# Use semantic blueprints for structured content generation
blueprint = Blueprint(
    id="social-post",
    name="Social Media Post",
    scene_goal=SceneGoal(objective="Create engaging social media content"),
    style_guide=StyleGuide(tone="casual", length_hint="concise")
)
prompt = blueprint.to_prompt()  # Convert to LLM-ready prompt

# Add content safety with moderation guardrails
moderation = ModerationPipeline(
    pre_flight=PreFlightGate([KeywordRule(["spam", "scam"]), PIIRule()])
)
result = await moderation.check_input(user_message)
if not result.allowed:
    raise ValueError(f"Content blocked: {result.violations}")
```

See the [Quick Start Guide](docs/quickstart.md) for more detailed examples.

---

## Integration Modes

### Mode A: CEMAF Orchestrates

CEMAF owns execution, external frameworks are "engines":

```python
from cemaf.orchestration import DAGExecutor
from cemaf.observability import InMemoryRunLogger

executor = DAGExecutor(
    node_executor=LangGraphNodeExecutor(langgraph_app),
    run_logger=InMemoryRunLogger(),
)
result = await executor.run(dag, context)

# Replay later for debugging
replayer = Replayer(run_logger.get_record("run-123"))
await replayer.replay()
```

### Mode B: CEMAF as Library

External frameworks orchestrate, CEMAF provides infrastructure:

```python
from cemaf.context import Context, ContextPatch
from cemaf.observability import InMemoryRunLogger

@langgraph_node
def my_node(state):
    ctx = Context.from_dict(state)

    # Track provenance of every change
    patch = ContextPatch.from_tool("search", "results", search_results)
    ctx = ctx.apply(patch)
    run_logger.record_patch(patch)

    # Compile within budget
    compiled = compiler.compile(ctx, budget)
    return compiled.to_dict()
```

See the [Integration Guide](docs/integration.md) for detailed patterns.

---

## Key Features

### Context Engineering
- **Context Patches**: Track every context change with full provenance
- **Token Budgeting**: Stay within limits with smart compilation (greedy, knapsack, optimal algorithms)
- **Deterministic Replay**: Record and replay runs for debugging
- **Glass Box Audit**: Full provenance chain linking every LLM call to its context sources, citations, and costs
- **Context Type Classification**: RESOURCE/MEMORY/SKILL behavioral semantics with per-type compaction rules
- **Semantic Blueprints**: Structured content generation with Denis Rothman's blueprint pattern
- **Recursive LLM**: Parallel divide-and-conquer querying for 1M+ token contexts

### Memory System
- **Strict Scoping**: Memory boundaries with TTL prevent state leaks
- **Three-Tier Progressive Loading**: L0 abstract / L1 overview / L2 full content for token-efficient retrieval
- **Semantic Deduplication**: Exact key + embedding similarity detection with merge/skip resolution
- **Post-Session Extraction**: Automatic promotion of session learnings to long-term memory (patterns, corrections, facts)
- **Hierarchical Scope Propagation**: Parent-to-child score propagation for scope-aware retrieval
- **SQLite Persistence**: Production-ready persistent memory store via aiosqlite

### Online Evaluation
- **Hierarchical Judge**: Three-tier evaluation -- fast deterministic checks, semantic similarity, LLM judge (escalates only when needed)
- **Online Eval Pipeline**: Subscribe to execution events and run evaluators on node outputs in real-time
- **Quality Police**: Rolling window quality monitor with anomaly detection and automatic halt gates
- **Eval Tools & Agents**: RunEvalTool, CheckQualityTool, RecordScoreTool, QualityGuardAgent -- dogfooding the eval system as CEMAF tools

### Production Backends
- **Resilient LLM Client**: Retry with exponential backoff + circuit breaker + rate limiter composing around any LLMClient
- **OpenAI Embeddings**: Production embedding provider using text-embedding-3-small with batch support
- **Structured Logging**: JSON-lines logger with context fields for production observability
- **Prometheus Metrics**: Counter/gauge/histogram/timing export with lazy metric registration

### Orchestration
- **DAG Executor**: Topological sort, parallel execution, conditional routing, loop nodes, cooperative cancellation
- **Node Type Handlers**: Extracted router, conditional, loop, parallel handlers for clean separation
- **RuntimeServices**: Frozen dataclass bundling 15+ optional dependencies for composition root
- **Bootstrap**: Single `create_executor()` entry point wiring registry, services, and subscriptions
- **Context Agents**: Built-in Librarian, Researcher, Summarizer, Writer agents with dynamic registry
- **Budget Guard**: Configurable cost and token limits with warning/critical/halt thresholds

### Infrastructure
- **Protocol-Based**: Plug into any framework -- modules work standalone, extend with your own implementations
- **Extensible Registries**: ProviderRegistry pattern for pluggable LLM, compiler, and retrieval backends
- **Instrumented LLM**: Transparent LLM call recording for automatic glass box audit
- **Moderation & Guardrails**: Pre/post-flight content safety with PII detection and compliance rules
- **Configuration-Driven**: Zero-config defaults with .env customization
- **Resilience**: Retry, circuit breaker, rate limiting as composable decorators

### Self-Hosting Engine (v0.2.0)
CEMAF is its own first client — three opt-in modules where the framework uses its own primitives to introspect, audit, and extend itself. Fully decoupled from the base framework.

- **Audit Trail**: `EventBusAuditLog` subscribes to EventBus, converts events into queryable `AuditEntry` records with quality trend analysis and z-score anomaly detection
- **Knowledge Graph**: `MemoryBackedKnowledgeGraph` — entities and relations backed by MemoryManager with semantic search and neighbor traversal
- **Meta-Agents**: `MetaArchitect` (DAG design), `MetaSynthesizer` (code gen), `MetaAuditor` (trace analysis), `MetaKnowledgeGraph` (KG operations)
- **Pre-built DAGs**: `create_self_audit_dag()`, `create_feature_synthesis_dag()`, `create_knowledge_refresh_dag()`
- **Entry point**: `create_meta_executor()` wraps `create_executor()`, auto-wires audit + KG from RuntimeServices

---

## Documentation

**[Full Documentation →](docs/README.md)**

### Getting Started
- [Quick Start Guide](docs/quickstart.md) - Get running in 5 minutes
- [Protocol Guide](docs/protocol_guide.md) - Understanding CEMAF's protocol-based architecture
- [Extension Patterns](docs/extension_patterns.md) - How to extend CEMAF with your own implementations
- [Standalone Usage](docs/standalone_usage.md) - Using modules independently

### Core Guides
- [Architecture Overview](docs/architecture.md) - System design and principles
- [Context Management](docs/context.md) - Patches, provenance, budgeting
- [Replay & Recording](docs/replay.md) - Deterministic replay
- [Tools, Skills, Agents](docs/tools.md) - Execution layer
- [Integration Guide](docs/integration.md) - Framework integration patterns

### Module References
- [LLM Integration](docs/llm.md)
- [Caching](docs/cache.md)
- [Persistence](docs/persistence.md)
- [Observability](docs/observability.md)
- [Citation Tracking](docs/citation.md) - Source attribution
- [MCP Integration](docs/mcp.md) - Model Context Protocol
- [Blueprint](docs/blueprint.md) - Semantic blueprints for content generation
- [Moderation](docs/moderation.md) - Guardrails and content safety
- [Retrieval](docs/retrieval.md) - Vector stores and search

---

## Configuration

CEMAF is designed for zero-config startup with production-ready defaults. Customize via environment variables:

```bash
# Copy example configuration
cp .env.example .env

# Configure your setup
CEMAF_LLM_PROVIDER=openai
CEMAF_LLM_API_KEY=your-key
CEMAF_CACHE_BACKEND=redis
CEMAF_CACHE_MAX_SIZE=10000
```

Use factory functions for automatic configuration loading:

```python
from cemaf.llm import create_llm_client_from_config
from cemaf.cache import create_cache_from_config

# Automatically loads from .env or environment
client = create_llm_client_from_config()
cache = create_cache_from_config()
```

See the [Configuration Guide](docs/config.md) for all available settings.

---

## Testing

```bash
# Run all tests
pytest tests/

# Unit tests only
pytest tests/unit/

# Skip slow tests
pytest tests/ -m "not slow"

# With coverage
pytest tests/ --cov=cemaf

# Pre-commit checks
pre-commit run --all-files
```

**Project Stats**: 2301+ tests | 100% passing | TDD from day one

---

## Contributing

We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

Development setup:

```bash
# Fork and clone the repo
git clone https://github.com/YOUR_USERNAME/cemaf.git
cd cemaf

# Install dependencies with uv
uv venv
uv sync

# Install pre-commit hooks
uv run pre-commit install
```

See [HOW_TO_USE.md](HOW_TO_USE.md) for detailed usage examples.

---

## Getting Help

We're here to help! Here are the best ways to get support:

### Documentation

- [Full Documentation](docs/README.md) - Comprehensive guides for all features
- [Quick Start Guide](docs/quickstart.md) - Get started in minutes
- [HOW_TO_USE.md](HOW_TO_USE.md) - Detailed usage patterns
- [Architecture Guide](docs/architecture.md) - Understand CEMAF's design

### Community

- [Discord Server](https://discord.gg/C8ZXAbD8) - Join our community for real-time help
- [GitHub Discussions](https://github.com/drchinca/cemaf/discussions) - Ask questions and share ideas
- [GitHub Issues](https://github.com/drchinca/cemaf/issues) - Report bugs or request features

### Contributing

Want to contribute? Check out our [Contributing Guide](CONTRIBUTING.md) to get started!

We're in **Alpha** and actively seeking feedback!

---

## Philosophy & Open Startup

CEMAF operates as an **open startup** - we believe in radical transparency, community collaboration, and building in public.

### Our Principles

- **Community First:** We serve developers building AI agents
- **Transparent:** All decisions, metrics, and roadmap are public
- **Bias Toward Action:** Show > tell. Open PRs, not long debates
- **Anyone Can Help:** Contribution > credentials
- **Learn in Public:** We share wins AND mistakes

### Resources

- **[Philosophy Guide](docs/philosophy.md)** - Our 10 core principles and values
- **[Open Metrics](OPEN.md)** - Transparent metrics, roadmap, and financials
- **[Decision Log](docs/decisions/)** - All major decisions documented
- **[Weekly Updates](https://github.com/drchinca/cemaf/discussions)** - Progress, learnings, and challenges

**We're building CEMAF together. Your voice matters.**

---

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

---

## Authors

**Hikuri Bado Chinca** ([@drchinca](https://github.com/drchinca))
Email: chincadr@gmail.com

Copyright (c) 2026 | Published on 1.1.2026 🎉

---

## Links

- **Documentation**: [docs/README.md](docs/README.md)
- **Issues**: [GitHub Issues](https://github.com/drchinca/cemaf/issues)
- **Contributing**: [CONTRIBUTING.md](CONTRIBUTING.md)
