Metadata-Version: 2.1
Name: dstack
Version: 0.12.0a2
Summary: dstack is an open-source framework for orchestration GPU workloads and development of generative AI models across multiple clouds.
Home-page: https://dstack.ai
Author: Andrey Cheptsov
Author-email: andrey@dstack.ai
License: UNKNOWN
Project-URL: Source, https://github.com/dstackai/dstack
Platform: UNKNOWN
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: pyyaml
Requires-Dist: requests
Requires-Dist: typing-extensions >=4.0.0
Requires-Dist: cryptography
Requires-Dist: packaging
Requires-Dist: python-dateutil
Requires-Dist: gitpython
Requires-Dist: jsonschema
Requires-Dist: paramiko
Requires-Dist: git-url-parse
Requires-Dist: cursor
Requires-Dist: rich
Requires-Dist: rich-argparse
Requires-Dist: tqdm
Requires-Dist: simple-term-menu
Requires-Dist: fastapi
Requires-Dist: starlette >=0.26.0
Requires-Dist: uvicorn
Requires-Dist: pydantic <=1.10.10
Requires-Dist: sqlalchemy[asyncio] >=2.0.0
Requires-Dist: sqlalchemy-utils >=0.40.0
Requires-Dist: alembic >=1.10.2
Requires-Dist: apscheduler
Requires-Dist: aiosqlite
Requires-Dist: aiohttp
Requires-Dist: websocket-client
Requires-Dist: watchfiles
Requires-Dist: python-multipart
Requires-Dist: filelock
Requires-Dist: docker >=6.0.0
Requires-Dist: dnspython
Requires-Dist: grpcio >=1.50
Provides-Extra: all
Requires-Dist: boto3 ; extra == 'all'
Requires-Dist: botocore ; extra == 'all'
Requires-Dist: azure-identity >=1.12.0 ; extra == 'all'
Requires-Dist: azure-keyvault-secrets >=4.6.0 ; extra == 'all'
Requires-Dist: azure-storage-blob >=12.15.0 ; extra == 'all'
Requires-Dist: azure-monitor-query >=1.2.0 ; extra == 'all'
Requires-Dist: azure-mgmt-subscription >=3.1.1 ; extra == 'all'
Requires-Dist: azure-mgmt-compute >=29.1.0 ; extra == 'all'
Requires-Dist: azure-mgmt-network ==23.0.0b2 ; extra == 'all'
Requires-Dist: azure-mgmt-resource >=22.0.0 ; extra == 'all'
Requires-Dist: azure-mgmt-authorization >=3.0.0 ; extra == 'all'
Requires-Dist: azure-mgmt-storage >=21.0.0 ; extra == 'all'
Requires-Dist: azure-mgmt-keyvault >=10.1.0 ; extra == 'all'
Requires-Dist: azure-mgmt-loganalytics ==13.0.0b6 ; extra == 'all'
Requires-Dist: azure-mgmt-msi ; extra == 'all'
Requires-Dist: azure-mgmt-monitor ; extra == 'all'
Requires-Dist: azure-graphrbac ; extra == 'all'
Requires-Dist: google-auth >=2.3.0 ; extra == 'all'
Requires-Dist: google-cloud-storage >=2.0.0 ; extra == 'all'
Requires-Dist: google-cloud-compute >=1.5.0 ; extra == 'all'
Requires-Dist: google-cloud-secret-manager >=2.0.0 ; extra == 'all'
Requires-Dist: google-cloud-logging >=2.0.0 ; extra == 'all'
Requires-Dist: google-api-python-client >=2.80.0 ; extra == 'all'
Requires-Dist: google-cloud-billing >=1.11.0 ; extra == 'all'
Provides-Extra: aws
Requires-Dist: boto3 ; extra == 'aws'
Requires-Dist: botocore ; extra == 'aws'
Provides-Extra: azure
Requires-Dist: azure-identity >=1.12.0 ; extra == 'azure'
Requires-Dist: azure-keyvault-secrets >=4.6.0 ; extra == 'azure'
Requires-Dist: azure-storage-blob >=12.15.0 ; extra == 'azure'
Requires-Dist: azure-monitor-query >=1.2.0 ; extra == 'azure'
Requires-Dist: azure-mgmt-subscription >=3.1.1 ; extra == 'azure'
Requires-Dist: azure-mgmt-compute >=29.1.0 ; extra == 'azure'
Requires-Dist: azure-mgmt-network ==23.0.0b2 ; extra == 'azure'
Requires-Dist: azure-mgmt-resource >=22.0.0 ; extra == 'azure'
Requires-Dist: azure-mgmt-authorization >=3.0.0 ; extra == 'azure'
Requires-Dist: azure-mgmt-storage >=21.0.0 ; extra == 'azure'
Requires-Dist: azure-mgmt-keyvault >=10.1.0 ; extra == 'azure'
Requires-Dist: azure-mgmt-loganalytics ==13.0.0b6 ; extra == 'azure'
Requires-Dist: azure-mgmt-msi ; extra == 'azure'
Requires-Dist: azure-mgmt-monitor ; extra == 'azure'
Requires-Dist: azure-graphrbac ; extra == 'azure'
Provides-Extra: gcp
Requires-Dist: google-auth >=2.3.0 ; extra == 'gcp'
Requires-Dist: google-cloud-storage >=2.0.0 ; extra == 'gcp'
Requires-Dist: google-cloud-compute >=1.5.0 ; extra == 'gcp'
Requires-Dist: google-cloud-secret-manager >=2.0.0 ; extra == 'gcp'
Requires-Dist: google-cloud-logging >=2.0.0 ; extra == 'gcp'
Requires-Dist: google-api-python-client >=2.80.0 ; extra == 'gcp'
Requires-Dist: google-cloud-billing >=1.11.0 ; extra == 'gcp'
Provides-Extra: lambda
Requires-Dist: boto3 ; extra == 'lambda'
Requires-Dist: botocore ; extra == 'lambda'

<div align="center">
<h1 align="center">
  <a target="_blank" href="https://dstack.ai">
    <img alt="dstack" src="https://raw.githubusercontent.com/dstackai/dstack/master/docs/assets/images/dstack-logo.svg" width="350px"/>
  </a>
</h1>

<h3 align="center">
Orchestrate GPU workloads across clouds
</h3>

<p align="center">
<a href="https://dstack.ai/docs" target="_blank"><b>Docs</b></a> •
<a href="https://dstack.ai/examples" target="_blank"><b>Examples</b></a> •
<a href="https://dstack.ai/blog" target="_blank"><b>Blog</b></a> •
<a href="https://discord.gg/u8SmfwPpMd" target="_blank"><b>Discord</b></a>
</p>

[![Last commit](https://img.shields.io/github/last-commit/dstackai/dstack?style=flat-square)](https://github.com/dstackai/dstack/commits/)
[![PyPI - License](https://img.shields.io/pypi/l/dstack?style=flat-square&color=blue)](https://github.com/dstackai/dstack/blob/master/LICENSE.md)
</div>

`dstack` is an open-source framework for orchestrating GPU workloads
across multiple cloud GPU providers. It provides a simple cloud-agnostic interface for 
development and deployment of generative AI models.

## Latest news ✨

- [2023/09] [Deploying LLMs using Python API](https://dstack.ai/examples/python-api) (Example)
- [2023/09] [Managed gateways](https://dstack.ai/blog/2023/09/01/managed-gateways) (Release)
- [2023/08] [Fine-tuning Llama 2 using QLoRA](https://dstack.ai/examples/finetuning-llama-2) (Example)
- [2023/08] [Deploying Stable Diffusion using FastAPI](https://dstack.ai/examples/stable-diffusion-xl) (Example)
- [2023/07] [Deploying LLMS using TGI](https://dstack.ai/examples/text-generation-inference) (Example)
- [2023/07] [Deploying LLMS using vLLM](https://dstack.ai/examples/vllm) (Example)

## Installation

To use `dstack`, install it with `pip`, and start the server.

```shell
pip install "dstack[all]" -U
dstack start
```
## Configure clouds

Upon startup, the server sets up the default project called `main`.
Prior to using `dstack`, make sure to [configure clouds](https://dstack.ai/docs/guides/clouds#configure-backends).

Once the server is up, you can orchestrate GPU workloads using
either the CLI or Python API.

## Using CLI

### Define a configuration

The CLI allows you to define what you want to run as a YAML file and
run it via the `dstack run` CLI command.

Configurations can be of three types: `dev-environment`, `task`, and `service`.

#### Dev environments

A dev environment is a virtual machine with a pre-configured IDE.

```yaml
type: dev-environment

python: "3.11" # (Optional) If not specified, your local version is used

setup: # (Optional) Executed once at the first startup
  - pip install -r requirements.txt

ide: vscode
```

#### Tasks

A task can be either a batch job, such as training or fine-tuning a model, or a web application.

```yaml
type: task

python: "3.11" # (Optional) If not specified, your local version is used

ports:
  - 7860

commands:
  - pip install -r requirements.txt
  - python app.py
```

While the task is running in the cloud, the CLI forwards its ports traffic to `localhost`
for convenient access.

#### Services

A service is an application that is accessible through a public endpoint.

```yaml
type: service

port: 7860

commands:
  - pip install -r requirements.txt
  - python app.py
```

Once the service is up, `dstack` makes it accessible from the Internet through
the [gateway](https://dstack.ai/docs/guides/clouds#configure-gateways).

### Run a configuration

To run a configuration, use the [`dstack run`](https://dstack.ai/docs/reference/cli/run.md) command followed by 
working directory and the path to the configuration file.

```shell
dstack run . -f text-generation-inference/serve.dstack.yml --gpu 80GB -y

 RUN           BACKEND  INSTANCE              SPOT  PRICE STATUS    SUBMITTED
 tasty-zebra-1 lambda   200GB, 1xA100 (80GB)  no    $1.1  Submitted now
 
Privisioning...

Serving on https://tasty-zebra-1.mydomain.com
```

## Using API

As an alternative to the CLI, you can run tasks and services programmatically 
via [Python API](https://dstack.ai/docs/reference/api/python/).

```python
import sys

import dstack

task = dstack.Task(
    image="ghcr.io/huggingface/text-generation-inference:latest",
    env={"MODEL_ID": "TheBloke/Llama-2-13B-chat-GPTQ"},
    commands=[
        "text-generation-launcher --trust-remote-code --quantize gptq",
    ],
    ports=["8080:80"],
)
resources = dstack.Resources(gpu=dstack.GPU(memory="20GB"))

if __name__ == "__main__":
    print("Initializing the client...")
    client = dstack.Client.from_config(repo_dir="~/dstack-examples")

    print("Submitting the run...")
    run = client.runs.submit(configuration=task, resources=resources)

    print(f"Run {run.name}: " + run.status())

    print("Attaching to the run...")
    run.attach()

    # After the endpoint is up, http://127.0.0.1:8080/health will return 200 (OK).

    try:
        for log in run.logs():
            sys.stdout.buffer.write(log)
            sys.stdout.buffer.flush()

    except KeyboardInterrupt:
        print("Aborting the run...")
        run.stop(abort=True)
    finally:
        run.detach()
```

## More information

For additional information and examples, see the following links:

- [Docs](https://dstack.ai/docs)
- [Examples](https://dstack.ai/examples)
- [Blog](https://dstack.ai/blog)
- [Discord](https://discord.gg/u8SmfwPpMd)

## Licence

[Mozilla Public License 2.0](LICENSE.md)


