Metadata-Version: 2.0
Name: plumb
Version: 1.0.6
Summary: Connect systems via Kafka, Redis, AWS SQS and SNS
Home-page: UNKNOWN
Author: Elvio Toccalino
Author-email: elvio@spect.ro
License: UNKNOWN
Description-Content-Type: UNKNOWN
Keywords: redis,AWS,queues,distributed
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Utilities
Classifier: Topic :: System :: Networking
Classifier: Topic :: System :: Distributed Computing
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Object Brokering
Requires-Dist: boto3
Requires-Dist: confluent-kafka (==0.11.0)
Requires-Dist: redis

=================
The Plumb Library
=================

:Author:   Elvio Toccalino
:Date:     2018-03-02
:Version:  $Revision: 1.0.6 $

Provides higher level interfaces to work with data producers and consumers. ``plumb`` supports four backends: Redis, Amazon SQS and SNS services, Kafka, and a memory implementation (designed to be a testing artifact).

The focus is on **python 3** and **boto3** (for AWS support).

-------
The API
-------

``plumb`` uses the concepts of *Source* and *Sink* to abstract the backend. Data is transfered encoded in JSON, and by default compressed using zlib.

To fetch raw packages use a "Package Source" object, which handles a backend connection and its configuration and exposes a "get()" method. Similarly, a "Package Sink" object exposes a "put(pkg)" method.

-----
Tests
-----

The library is provided with unit tests and integration tests for Redis and AWS. To run the unit tests::

  python setup.py test

or using ``nose``::

  nosetests tests/unit

The integration tests can be run using nose::

  nosetests tests/integration

**Keep in mind** that ``boto3`` will fetch your AWS credentials. It currently tries the environment variables ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY``, then tries the ``~/.aws`` directory. For details, see `Boto3 Credentials Configuration <http://boto3.readthedocs.io/en/latest/guide/configuration.html#configuring-credentials>`__.


