Metadata-Version: 2.4
Name: framelog
Version: 0.1.1
Summary: A framework for making agents more aware of their environment through explicit modeling of pre and post effects
Project-URL: Homepage, https://github.com/NaSirMiller/agent-log
License: MIT
Requires-Python: >=3.11
Requires-Dist: numpy
Requires-Dist: pydantic>=2.0
Requires-Dist: scikit-learn
Requires-Dist: sentence-transformers
Description-Content-Type: text/markdown

# Frame-log

## Overview

Frame-log is a framework aiming to make agents more aware of their environment through the explicit modeling of pre and post effects.
The approach is inspired by logic-based AI theory such as the ["AI Frame Problem"](https://plato.stanford.edu/entries/frame-problem/).

## Repository Overview

This repository implements the Frame-Log framework, providing utility methods that are usable across agentic workflows. Specifically, the following are included:

- Prompt templates to elicit change-logs
- Pre-defined change-log schema
  - Currently supports JSON. Future work can include XML or git diff formats.
- Extraction methods to decompose a set of change-logs from an agent response into structured objects
- Evaluation and sanity check methods for change-logs, including:
  - Schema validation
  - Action verification
  - Semantic similarity-based effect grounding via sentence embeddings

## Setup

```bash
pip install framelog
```

## Quick Start

```python
from framelog.extract_log import to_dict
from framelog.evaluator import ChangeLogEvaluator, ChangeLogFailureType

evaluator = ChangeLogEvaluator()

# Parse a change log from an agent response
change_log = to_dict(response, evaluator=evaluator)

# Advance the evaluator timestep
evaluator.step()

# Inspect failure rates
print(evaluator.no_json_rate())
print(evaluator.invalid_action_rate())
print(evaluator.failure_breakdown())
```

## Supported Benchmarks

- [ALFWorld](https://github.com/alfworld/alfworld)
- [OfficeBench](https://github.com/zlwang-cs/OfficeBench)

## License

MIT

---

> Do you spot any bugs? Inefficiencies? Or have any general suggestions? Please create an issue or contact me via [miller.j.nasir@gmail.com](mailto:miller.j.nasir@gmail.com)