Metadata-Version: 2.1
Name: shub-workflow
Version: 1.14.5.1
Summary: Workflow manager for Zyte ScrapyCloud tasks.
Home-page: https://github.com/scrapinghub/shub-workflow
Maintainer: Scrapinghub
License: BSD
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml>=3.12
Requires-Dist: scrapinghub[msgpack]>=2.3.1
Requires-Dist: jinja2>=2.7.3
Requires-Dist: sqlitedict==1.6.0
Requires-Dist: boto3>=1.9.92
Requires-Dist: bloom-filter2==2.0.0
Requires-Dist: collection-scanner
Requires-Dist: tenacity
Requires-Dist: typing-extensions
Requires-Dist: scrapy
Provides-Extra: with-s3-tools
Requires-Dist: s3fs>=0.4.0; extra == "with-s3-tools"
Provides-Extra: with-gcs-tools
Requires-Dist: google-cloud-storage>=1.38.0; extra == "with-gcs-tools"

A set of tools for controlling processing workflow with spiders and script running in scrapinghub ScrapyCloud.

# Installation

```
pip install shub-workflow
```

If you want to support s3 tools:

```
pip install shub-workflow[with-s3-tools]
```

For google cloud storage tools support:

```
pip install shub-workflow[with-gcs-tools]
```

# Usage

Check [Project Wiki](https://github.com/scrapinghub/shub-workflow/wiki) for documentation. You can also see code tests for lots of examples of usage.

# Note

The requirements for this library are defined in setup.py as usual. The Pipfile files in the repository don't define dependencies. It is only used
for setting up a development environment for shub-workflow library development and testing.
