Metadata-Version: 2.1
Name: athina
Version: 1.5.20
Summary: Python SDK to configure and run evaluations for your LLM-based application
Author: Shiv Sakhuja
Author-email: shiv@athina.ai
Requires-Python: >=3.9,<3.13
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: RestrictedPython (>=7.1,<8.0)
Requires-Dist: bandit (>=1.7.8,<2.0.0)
Requires-Dist: colorlog (>=6.7.0,<7.0.0)
Requires-Dist: datasets (>=2.16.0,<3.0.0)
Requires-Dist: hyphenator (>=0.6.0,<0.7.0)
Requires-Dist: jinja2 (>=3.1.4,<4.0.0)
Requires-Dist: jsonpath-ng (>=1.6.0,<2.0.0)
Requires-Dist: jsonschema (==4.22.0)
Requires-Dist: langchain (>=0.0.350)
Requires-Dist: langchain-openai (>=0.0.8,<0.0.9)
Requires-Dist: litellm (==1.48.0)
Requires-Dist: llama-index (>=0.9.40,<0.10.0)
Requires-Dist: marvin (>=2.3.4,<3.0.0)
Requires-Dist: openai (>=1.3.4,<2.0.0)
Requires-Dist: pandas
Requires-Dist: pinecone-client (>=4.1.0,<5.0.0)
Requires-Dist: pydantic (==2.6.3)
Requires-Dist: pydantic-settings (==2.2.1)
Requires-Dist: pydantic_core (==2.16.3)
Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
Requires-Dist: pyyaml (>=6.0.1,<7.0.0)
Requires-Dist: qdrant-client (>=1.9.1,<2.0.0)
Requires-Dist: ragas (>=0.0.22,<0.0.23)
Requires-Dist: requests (>=2.31.0,<3.0.0)
Requires-Dist: retrying (>=1.3.4,<2.0.0)
Requires-Dist: textatistic (>=0.0.1,<0.0.2)
Requires-Dist: timeout-decorator (>=0.5.0,<0.6.0)
Requires-Dist: tokenizers (>=0.19,<0.20)
Requires-Dist: weaviate-client (>=4.7.1,<5.0.0)
Description-Content-Type: text/markdown

# Overview
Athina is an Observability and Experimentation platform for AI teams.

This SDK is an open-source repository of [50+ preset evals](https://docs.athina.ai/evals/preset-evals/overview). You can also use [custom evals](https://docs.athina.ai/evals/custom-evals/overview).

This SDK also serves as a companion to [Athina IDE](https://athina.ai/develop) where you can prototype pipelines, run experiments and evaluations, and compare datasets.

---

### Quick Start
Follow [this notebook](https://github.com/athina-ai/athina-evals/blob/main/examples/run_eval_suite.ipynb) for a quick start guide.

To get an Athina API key, sign up at https://app.athina.ai

---

### Run Evals

These evals can be run [programmatically](https://athina.ai/videos/run-evals-programmatically.mp4), or [via the UI](https://docs.athina.ai/ide/run-eval) on Athina IDE.

<img width="1530" alt="image" src="https://github.com/athina-ai/athina-evals/assets/7515552/98494736-31b6-458f-bd0a-a5b2cbca9d70">

---

### Compare datasets side-by-side ([Docs](https://docs.athina.ai/ide/compare-datasets))

Once a dataset is logged to Athina IDE, you can also compare it against another dataset.

![image](https://github.com/athina-ai/athina-evals/assets/7515552/90640acc-495e-45e0-b590-d6ddee8c5727)


Once you run evals using Athina, they will be visible in [Athina IDE](https://athina.ai/develop) where you can run experiments, evals, and compare datasets side-by-side.

---

### Preset Evals

