Metadata-Version: 2.1
Name: dat1-cli
Version: 0.0.4
License: MIT
Description-Content-Type: text/markdown
Requires-Dist: typer>=0.9.0
Requires-Dist: inquirer>=3.1.3
Requires-Dist: boto3>=1.35.43
Requires-Dist: pyyaml>=6.0.2
Requires-Dist: yaspin>=3.1.0

# dat1-cli

A command line interface for the [dat1](dat1.co) platform.

## Installation

```bash
pip install dat1-cli
```


## Usage

Initialize with your API key:

```bash
dat1 login
```

To initialize a new model project, run in the root directory of your project:

```bash
dat1 init
```

To upload your model to the platform:

```bash
dat1 deploy
```

A good starting point for your model is using the [example model](https://github.com/dat1-co/dat1-docker/tree/main/example).

Otherwise, the platform expects a `handler.py` file in the root directory of your project that contains a FastAPI app with two endpoints: GET `/` for healthchecks and POST `/infer` for inference.
An example handler is shown below:

```python
from fastapi import Request, FastAPI
from vllm import LLM, SamplingParams
import os

llm = LLM(model=os.path.expanduser('./'), load_format="safetensors", enforce_eager=True)

app = FastAPI()

@app.get("/")
async def root():
    return "OK"

@app.post("/infer")
async def infer(request: Request):
    request = await request.json()
    prompts = request["prompt"]
    sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
    outputs = llm.generate(prompts, sampling_params)
    return { "response" : outputs[0].outputs[0].text }
```

## License

MIT
