Metadata-Version: 2.1
Name: pyncette
Version: 0.10.1
Summary: A reliable distributed scheduler with pluggable storage backends
Home-page: https://github.com/tibordp/pyncette
Author: Tibor Djurica Potpara
Author-email: tibor.djurica@ojdip.net
License: MIT
Project-URL: Documentation, https://pyncette.readthedocs.io/
Project-URL: Changelog, https://pyncette.readthedocs.io/en/latest/changelog.html
Project-URL: Issue Tracker, https://github.com/tibordp/pyncette/issues
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: Unix
Classifier: Operating System :: POSIX
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Topic :: Utilities
Requires-Python: >=3.8
License-File: LICENSE
License-File: AUTHORS.rst
Requires-Dist: croniter (>=1.3.14)
Requires-Dist: aiosqlite (>=0.19.0)
Requires-Dist: aiohttp (>=3.8.4)
Requires-Dist: python-dateutil (>=2.8.2)
Requires-Dist: coloredlogs
Provides-Extra: all
Requires-Dist: redis (>=4.5.4) ; extra == 'all'
Requires-Dist: prometheus-client (>=0.16.0) ; extra == 'all'
Requires-Dist: asyncpg (>=0.27.0) ; extra == 'all'
Requires-Dist: aioboto3 (>=11.1.0) ; extra == 'all'
Requires-Dist: aiomysql (>=0.1.1) ; extra == 'all'
Requires-Dist: cryptography (>=40.0.2) ; extra == 'all'
Requires-Dist: uvloop (>=0.16.0) ; extra == 'all'
Provides-Extra: dynamodb
Requires-Dist: aioboto3 (>=11.1.0) ; extra == 'dynamodb'
Provides-Extra: mysql
Requires-Dist: aiomysql (>=0.1.1) ; extra == 'mysql'
Requires-Dist: cryptography (>=40.0.2) ; extra == 'mysql'
Provides-Extra: postgres
Requires-Dist: asyncpg (>=0.27.0) ; extra == 'postgres'
Provides-Extra: prometheus
Requires-Dist: prometheus-client (>=0.16.0) ; extra == 'prometheus'
Provides-Extra: redis
Requires-Dist: redis (>=4.5.4) ; extra == 'redis'
Provides-Extra: uvloop
Requires-Dist: uvloop (>=0.16.0) ; extra == 'uvloop'

========
Overview
========



A reliable distributed scheduler with pluggable storage backends for Async Python.

* Free software: MIT license

Installation
============

Minimal installation (just SQLite persistence):

::

    pip install pyncette

Full installation (all the backends and Prometheus metrics exporter):

::

    pip install pyncette[all]

You can also install the in-development version with::

    pip install https://github.com/tibordp/pyncette/archive/master.zip


Documentation
=============


https://pyncette.readthedocs.io


Usage example
=============

Simple in-memory scheduler (does not persist state)

.. code:: python

    from pyncette import Pyncette, Context

    app = Pyncette()

    @app.task(schedule='* * * * *')
    async def foo(context: Context):
        print('This will run every minute')

    if __name__ == '__main__':
        app.main()

Persistent distributed cron using Redis (coordinates execution with parallel instances and survives restarts)

.. code:: python

    from pyncette import Pyncette, Context
    from pyncette.redis import redis_repository

    app = Pyncette(repository_factory=redis_repository, redis_url='redis://localhost')

    @app.task(schedule='* * * * * */10')
    async def foo(context: Context):
        print('This will run every 10 seconds')

    if __name__ == '__main__':
        app.main()


See the `examples` directory for more examples of usage.

Use cases
=========

Pyncette is designed for reliable (at-least-once or at-most-once) execution of recurring tasks (think cronjobs) whose
lifecycles are managed dynamically, but can work effectively for non-reccuring tasks too.

Example use cases:

- You want to perform a database backup every day at noon
- You want a report to be generated daily for your 10M users at the time of their choosing
- You want currency conversion rates to be refreshed every 10 seconds
- You want to allow your users to schedule non-recurring emails to be sent at an arbitrary time in the future

Pyncette might not be a good fit if:

- You want your tasks to be scheduled to run (ideally) once as soon as possible. It is doable, but you will be better served by a general purpose reliable queue like RabbitMQ or Amazon SQS.
- You need tasks to execute at sub-second intervals with low jitter. Pyncette coordinates execution on a per task-instance basis and this corrdination can add overhead and jitter.


Supported backends
==================

Pyncette comes with an implementation for the following backends (used for persistence and coordination) out-of-the-box:

- SQLite (included)
- Redis (``pip install pyncette[redis]``)
- PostgreSQL (``pip install pyncette[postgres]``)
- MySQL 8.0+ (``pip install pyncette[mysql]``)
- Amazon DynamoDB (``pip install pyncette[dynamodb]``)

Pyncette imposes few requirements on the underlying datastores, so it can be extended to support other databases or
custom storage formats / integrations with existing systems. For best results, the backend needs to provide:

- Some sort of serialization mechanism, e.g. traditional transactions, atomic stored procedures or compare-and-swap
- Efficient range queries over a secondary index, which can be eventually consistent


Development
===========

To run integration tests you will need Redis, PostgreSQL, MySQL and Localstack (for DynamoDB) running locally.

To run the all tests run::

    tox

Alternatively, there is a Docker Compose environment that will set up all the backends so that integration tests can run seamlessly::

    docker-compose up -d
    docker-compose run --rm shell
    tox

To run just the unit tests (excluding integration tests)::

    tox -e py310  # or your Python version of choice

Note, to combine the coverage data from all the tox environments run:

.. list-table::
    :widths: 10 90
    :stub-columns: 1

    - - Windows
      - ::

            set PYTEST_ADDOPTS=--cov-append
            tox

    - - Other
      - ::

            PYTEST_ADDOPTS=--cov-append tox


Changelog
=========

0.10.1 (2023-05-09)
-------------------
* Include missing lua files in the built wheel


0.10.0 (2023-05-08)
-------------------

* Drop support for Python 3.7
* Add support for Python 3.11
* Modernize Python package structure and linters
* Fix a few bugs and type annotations


0.8.1 (2021-04-08)
------------------

* Improve performance for calculation of the next execution time
* Add ability for repositories to pass a pagination token
* Add `add_to_context()` to inject static data to context
* Clean up documentation and add additional examples


0.8.0 (2021-04-05)
------------------

* Added Amazon DynamoDB backend
* Added MySQL backend
* Added support for partitioned dynamic tasks


0.7.0 (2021-03-31)
------------------

* Added support for automatic and cooperative lease heartbeating
* PostgreSQL backend can now skip automatic table creation
* Improved signal handling
* CI: Add Codecov integration
* Devenv: Run integration tests in Docker Compose


0.6.1 (2020-04-02)
------------------

* Optimize the task querying on Postgres backend
* Fix: ensure that there are no name colissions between concrete instances of different dynamic tasks
* Improve fairness of polling tasks under high contention.


0.6.0 (2020-03-31)
------------------

* Added PostgreSQL backend
* Added Sqlite backend and made it the default (replacing `InMemoryRepository`)
* Refactored test suite to cover all conformance/integration tests on all backends
* Refactored Redis backend, simplifying the Lua scripts and improving exceptional case handling (e.g. tasks disappearing between query and poll)
* Main loop only sleeps for the rest of remaining `poll_interval` before next tick instead of the full amount
* General bug fixes, documentation changes, clean up


0.5.0 (2020-03-27)
------------------

* Fixes bug where a locked dynamic task could be executed again on next tick.
* poll_task is now reentrant with regards to locking. If the lease passed in matches the lease on the task, it behaves as though it were unlocked.


0.4.0 (2020-02-16)
------------------

* Middleware support and optional metrics via Prometheus
* Improved the graceful shutdown behavior
* Task instance and application context are now available in the task context
* Breaking change: dynamic task parameters are now accessed via `context.args['name']` instead of `context.name`
* Improved examples, documentation and packaging


0.2.0 (2020-01-08)
------------------

* Timezone support
* More efficient poling when Redis backend is used


0.1.1 (2020-01-08)
------------------

* First release that actually works.


0.0.0 (2019-12-31)
------------------

* First release on PyPI.
