Metadata-Version: 2.4
Name: snowflake-cli-nextflow
Version: 0.1.2
Summary: A Snowflake CLI plugin that enables running Nextflow workflows directly in Snowpark Container Services
Author-email: Haowei Yu <haowei.yu@snowflake.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/Snowflake-Labs/snowflake-cli-nextflow
Project-URL: Repository, https://github.com/Snowflake-Labs/snowflake-cli-nextflow
Project-URL: Documentation, https://github.com/Snowflake-Labs/snowflake-cli-nextflow
Project-URL: Issues, https://github.com/Snowflake-Labs/snowflake-cli-nextflow/issues
Keywords: snowflake,cli,plugin,nextflow,workflow,bioinformatics,snowpark,container-services
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Database
Classifier: Topic :: System :: Distributed Computing
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: snowflake-cli>=3.0.0
Requires-Dist: pyyaml
Requires-Dist: websockets
Requires-Dist: docker
Requires-Dist: groovy-parser
Dynamic: license-file

# Snowflake CLI Nextflow Plugin

A Snowflake CLI plugin that enables running [Nextflow](https://www.nextflow.io/) workflows directly in [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview).

## Overview

This plugin extends the Snowflake CLI with Nextflow workflow capabilities, allowing you to:

- **Run Nextflow workflows at scale** using Snowflake's compute resources
- **Stream live execution logs** through secure WebSocket connections
- **Manage workflow execution** directly from your command line
- **Leverage Snowflake's security and governance** for bioinformatics and data pipelines

### Key Features

- 🚀 **Seamless Integration**: Run Nextflow workflows directly in Snowpark Container Services
- 📊 **Real-time Monitoring**: Stream live logs and execution status via WebSocket
- 🔒 **Enterprise Security**: Built-in Snowflake authentication and authorization
- 🎯 **Simple CLI Interface**: Familiar command-line experience for workflow management
- 📦 **Flexible Configuration**: Support for Nextflow profiles and custom configurations

## Getting Started

### Prerequisites

- All prerequisites of running snowflake-cli mentioned [here](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation#requirements)
- Access to a Snowflake account with Snowpark Container Services enabled
- A Snowflake compute pool configured for container workloads

### Installation

1. **Install Snowflake CLI**
   ```bash
   pip install snowflake-cli
   ```

2. **Install the Nextflow plugin**
   ```bash
   pip install snowflake-cli-nextflow
   ```

3. **Configure Snowflake connection**
   ```bash
   snow connection add
   ```
   Follow the prompts to configure your Snowflake connection

4. **Enable the Nextflow plugin**
   ```bash
   snow plugin enable nextflow
   ```
   The `nextflow` command should now be available in the CLI.

### Quick Start

1. **Prepare your Nextflow project**
   
   Create a `nextflow.config` file in your project directory:
   ```groovy
   // nextflow.config
   snowflake {
       computePool = 'YOUR_COMPUTE_POOL'
       workDirStage = 'WORKDIR_STAGE'
       stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'
       driverImage = '/DB/SCHEMA/REPO/nf-snowflake:0.8.0'
   }
   ```

2. **Upload nf-snowflake image into snowflake image repository**
  
   ```bash
   snow nextflow image push --source ghcr.io/snowflake-labs/nf-snowflake:0.8.0 --target nf_repo
   ```

3. **Run your workflow**
   ```bash
   snow nextflow run /path/to/your/nextflow-project --profile snowflake
   ```

4. **Monitor execution**
   
   The plugin will automatically:
   - Upload your project to Snowflake
   - Submit the workflow to Snowpark Container Services
   - Stream live logs to your terminal

## Configuration

### Snowflake-specific Configuration

Your `nextflow.config` must include the following Snowflake-specific settings:

```groovy
plugins { id 'nf-snowflake@0.8.0' }

snowflake {
    computePool = 'YOUR_COMPUTE_POOL'
    workDirStage = 'WORKDIR_STAGE'
    stageMounts = 'INPUT:/mnt/input,OUTPUT:/mnt/output'
    driverImage = '/DB/SCHEMA/REPO/nf-snowflake:0.8.0'
}
```

### Configuration Parameters

- **`computePool`** (Required): The name of your Snowflake compute pool where the Nextflow containers will run. This compute pool must be configured with appropriate resources and permissions for container workloads.

- **`workDirStage`** (Required): The Snowflake stage name where the plugin will upload your workflow files and store execution artifacts. This stage serves as the working directory for your Nextflow execution.

- **`driverImage`** (Required): The full path to the nf-snowflake image pushed to Snowflake registry. The version of the image should match the version specified in plugins section.

- **`stageMounts`** (Optional): A comma-separated list of stage mounts in the format `STAGE_NAME:/mount/path`. Each mount makes a Snowflake stage available inside the container at the specified path. Use this to provide input data and collect output results.

## Usage Examples

### Basic Workflow Execution

```bash
# Run with default profile
snow nextflow run ./my-workflow

# Run with specific profile
snow nextflow run ./my-workflow --profile snowflake

# Check available commands
snow nextflow --help
```

## License

This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.

## Acknowledgments

- [Nextflow](https://www.nextflow.io/) - The workflow management system
- [Snowflake](https://www.snowflake.com/) - The cloud data platform
- [Snowpark Container Services](https://docs.snowflake.com/en/developer-guide/snowpark-container-services/overview) - Container orchestration platform
