Metadata-Version: 2.4
Name: node-blue
Version: 0.0.0
Summary: Node-RED without a mouse / Node-RED as a library / Write Node-RED flows in YAML / Efficient test harnesses for Node-RED / Headless Node-RED / Improved development cycle times / Now it all makes sense
Author-email: Andreas Motl <andreas.motl@panodata.org>
License-Expression: Apache-2.0
Project-URL: changelog, https://github.com/daq-tools/node-blue/blob/main/CHANGES.md
Project-URL: documentation, https://github.com/daq-tools/node-blue
Project-URL: homepage, https://github.com/daq-tools/node-blue
Project-URL: repository, https://github.com/daq-tools/node-blue
Keywords: automation,flow-programming,javascript,json,JSPyBridge,node-blue,node-red,yaml
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Customer Service
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Manufacturing
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Telecommunications Industry
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Unix
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Adaptive Technologies
Classifier: Topic :: Communications
Classifier: Topic :: Database
Classifier: Topic :: Documentation
Classifier: Topic :: Education
Classifier: Topic :: Internet
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Office/Business
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Software Development :: Embedded Systems
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Object Brokering
Classifier: Topic :: Software Development :: Pre-processors
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Version Control
Classifier: Topic :: System :: Archiving
Classifier: Topic :: System :: Benchmark
Classifier: Topic :: System :: Clustering
Classifier: Topic :: System :: Distributed Computing
Classifier: Topic :: System :: Hardware
Classifier: Topic :: System :: Logging
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: System :: Networking
Classifier: Topic :: System :: Systems Administration
Classifier: Topic :: Text Processing
Classifier: Topic :: Utilities
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click<9
Requires-Dist: colorama<1
Requires-Dist: colorlog<7
Requires-Dist: funcy<3
Requires-Dist: javascript==1!1.2.6
Requires-Dist: json5<1
Requires-Dist: pygments<3
Requires-Dist: pyyaml<7
Requires-Dist: requests<3
Requires-Dist: runs<2
Requires-Dist: tabulate<1
Provides-Extra: develop
Requires-Dist: munch<5; extra == "develop"
Requires-Dist: mypy<1.20; extra == "develop"
Requires-Dist: poethepoet<1; extra == "develop"
Requires-Dist: pyproject-fmt<3; extra == "develop"
Requires-Dist: ruff<0.16; extra == "develop"
Requires-Dist: types-tabulate; extra == "develop"
Provides-Extra: release
Requires-Dist: build<2; extra == "release"
Requires-Dist: twine<7; extra == "release"
Provides-Extra: test
Requires-Dist: coverage<8; extra == "test"
Requires-Dist: pytest<9; extra == "test"
Requires-Dist: pytest-asyncio; extra == "test"
Requires-Dist: pytest-mqtt; extra == "test"
Requires-Dist: pytest-repeat; extra == "test"
Dynamic: license-file

# Node-BLUE

[Node-BLUE] is a friendly wrapper around [Node-RED].

_Node-RED without a mouse / Node-RED as a library / Write Node-RED flows in YAML /
Write Node-RED user-defined functions in Python / Efficient test harnesses for Node-RED /
True headless Node-RED / Improved development iteration times / Node-RED without needing
to take the red pill / Node-BLUE is Node-RED on rails / Now it all makes sense_

## About

The idea is to embed [Node-RED] into Python programs, in order to leverage it
for a number of use cases, like system automation, software testing, parallel
execution, etc.

The project has similar intentions like [node-red-embedded-start], offering to
interact programmatically with the Node-RED API, with a few bells and whistles.

The other idea is to extend the Node-RED ecosystem by leveraging other programming
languages and their ecosystems natively, beyond what JavaScript/NPM can do. We made
a start with [Python], by using the excellent [JSPyBridge] package.

## Synopsis

### Command-line use

Install [Node-BLUE] and [HTTPie], and their prerequisites.
```shell
pip install httpie node-blue
```

There is currently a little manual post-setup procedure.
```shell
node-blue setup
wget https://github.com/daq-tools/node-blue/raw/refs/heads/main/package.json
wget https://github.com/daq-tools/node-blue/raw/refs/heads/main/node.importmap
npm install
```

Start Node-BLUE with a Node-RED flow defining an HTTP/HTML endpoint/responder.
```shell
# Launch Node-BLUE/Node-RED.
node-blue launch --flow=examples/flows/http-html-templating.json
node-blue launch --flow=https://github.com/daq-tools/node-blue/raw/main/examples/flows/http-html-templating.json

# Run an example HTTP request.
http http://localhost:1880/hello-form name=Hotzenplotz
```

Launch a flow which defines a pipeline to converge data from [MQTT] to [CrateDB].
```shell
# Launch Node-BLUE/Node-RED.
node-blue launch --flow=examples/flows/mqtt-to-cratedb.yaml
```

### Library use

```python
import asyncio
from node_blue.core import NodeBlue

async def launch_blue():
    """
    Launch a Node-BLUE instance.
    """
    
    # Configure Node-BLUE instance with Node-RED flow.
    blue = NodeBlue(flow="https://github.com/daq-tools/node-blue/raw/main/examples/flows/http-html-templating.json")

    # Start Node-RED instance.
    blue.start().wait_started()

    # Wait until termination.
    await blue.forever()

if __name__ == "__main__":
    asyncio.run(launch_blue())
```

## Examples

### Introduction

Unless otherwise noted, all subsequent examples can be exercised using MQTT. You will
start the [Mosquitto] MQTT broker, subscribe to a topic, and publish a JSON message,
which will be processed by a Node-RED flow definition.

All flow definitions implement the same rule, and as such, can be triggered by using
the same MQTT messages, outlined below. For demonstration purposes, the transformation
rule is very simple: It applies a unit conversion formula to a numeric input parameter,
effectively converting value units, from Fahrenheit to Celsius. In order to not mix
things up, the outcome will be published to a different MQTT topic.

This effectively demonstrates message routing and republishing capabilities in both
terms of content transformation and topic rewriting. The procedures can be applied
to any message broker system, we just used MQTT here, because it is so convenient
to operate.

```shell
docker run --name=mosquitto -it --rm --publish=1883:1883 eclipse-mosquitto:2.0 \
  mosquitto -c /mosquitto-no-auth.conf
```

Subscribe to the broker to observe the outcome.
```shell
mosquitto_sub -t 'testdrive/#' -v
```

Run an example MQTT publish.
```shell
echo '{"temperature": 42.42}' | mosquitto_pub -t 'testdrive/imperial' -l
```

### JSON5 flow format

Start Node-BLUE with a Node-RED flow defining an MQTT topic rewrite and message transformation
step. Note that this example uses the [JSON5] format, which has a few advantages over regular
JSON, like inline comments, and multi-line strings.
```shell
wget https://github.com/daq-tools/node-blue/raw/main/examples/flows/mqtt-unit-rewriting.json5
node-blue launch --flow=mqtt-unit-rewriting.json5
```
This flow snippet demonstrates JSON5 capabilities on behalf of a sensible example.
```json5
// The transformation node element uses a function written in JavaScript to convert from
// Fahrenheit to Celsius, and to rewrite the MQTT topic from `imperial` to `metric`.
// The outcome will be the amended MQTT message object, which can be passed to the `mqtt out`
// data sink node without further ado.
{
  "id": "transformation.55460a",
  "type": "function",
  "func": "\
    if (msg.topic.endsWith('imperial')) { \
      msg.payload.temperature = Number.parseFloat(((Number.parseFloat(msg.payload.temperature) - 32) * 5 / 9).toFixed(2)); \
      msg.topic = msg.topic.replace('imperial', 'metric'); \
      return msg; \
    } \
  ",
  "wires": [
    [
      "sink.3539af",
    ]
  ]
}
```

### YAML flow format

You can also use the [YAML] format for writing flow recipes. The authors believe it offers
the best conciseness and convenience, with even better multi-line code blocks, without needing
any sorts of line-continuation characters. YAML is already used by the
[node-red-contrib-flow-manager] and [node-red-contrib-yaml-storage] plugins.
```shell
node-blue launch --flow=https://github.com/daq-tools/node-blue/raw/main/examples/flows/mqtt-unit-rewriting.yaml
```
With this example flow snippet, you will immediately recognize how convenient it is,
especially for user-defined functions.
```yaml
# The transformation node element uses a function written in JavaScript to convert from
# Fahrenheit to Celsius, and to rewrite the MQTT topic from `imperial` to `metric`.
# The outcome will be the amended MQTT message object, which can be passed to the `mqtt out`
# data sink node without further ado.
- id: transformation.55460a
  type: function
  func: |-
    if (msg.topic.endsWith('imperial')) {
      msg.payload.temperature = Number.parseFloat(((Number.parseFloat(msg.payload.temperature) - 32) * 5 / 9).toFixed(2));
      msg.topic = msg.topic.replace('imperial', 'metric');
      return msg;
    }
  wires: [
    [
      "sink.3539af",
    ]
  ]
```

### Python user-defined functions

This time, let's use the [Python] language, to define a user-defined function within the
flow step.
```shell
wget https://github.com/daq-tools/node-blue/raw/main/examples/flows/mqtt-routing-python.yaml
node-blue launch --flow=examples/flows/mqtt-routing-python.yaml
```

```yaml
# The transformation node element uses a function written in Python to convert from
# Fahrenheit to Celsius, and to rewrite the MQTT topic from `imperial` to `metric`.
# The outcome will be the amended MQTT message object, which can be passed to the `mqtt out`
# data sink node without further ado.
- id: transformation.66571b
  type: python-function
  func: |-
    if msg.topic.endswith("imperial"):
      msg.payload.temperature = round((float(msg.payload.temperature) - 32) * 5 / 9, 2)
      msg.topic = msg.topic.replace("imperial", "metric")
      send(msg)
    done()
  wires: [
    [
      "sink.3539af",
    ]
  ]
```

## References

See also the IBM Data management article about [Implementing ETL flows with
Node-RED] by [Ondrej Lehota].

## Etymology

To use the name »Node-BLUE« for this project was obvious. However, we discovered
that there has been another project called Node-BLUE, referenced below. On the
other hand, because it has been archived two years ago already, we think there
will not be much harm to reuse that name now.

- https://www.npmjs.com/package/node-blue
- https://github.com/node-blue/node-blue

## Acknowledgements

This project bundles a few significant pieces of software and technologies,
which a few bright minds invented the other day, and countless authors
contributed to. We are only listing inventors of the major application-level
components, remember there may be several layers of operating systems beneath,
and silicon either.

- [Andy Stanford-Clark], [Arlen Nipper], and co-authors for [MQTT].
- [Brendan Eich] and contributors for [JavaScript].
- [@extremeheat] and contributors for [JSPyBridge].
- [Guido van Rossum] and contributors for [Python].
- [Lars Bak] and contributors for [V8].
- [Nick O'Leary] and contributors for [Node-RED]. 
- [Roger Light]  and contributors for [Mosquitto].
- [Ryan Dahl] and contributors for [Node.js] and [Deno].

Thank you!


[Andy Stanford-Clark]: https://stanford-clark.com/
[Arlen Nipper]: https://github.com/anipper
[Brendan Eich]: https://en.wikipedia.org/wiki/Brendan_Eich
[Deno]: https://github.com/denoland
[@extremeheat]: https://github.com/extremeheat
[Guido van Rossum]: https://github.com/gvanrossum
[HTTPie]: https://httpie.io/
[Implementing ETL flows with Node-RED]: https://developer.ibm.com/articles/implementing-etl-flows-with-node-red/
[JavaScript]: https://en.wikipedia.org/wiki/JavaScript
[JSON5]: https://json5.org/
[JSPyBridge]: https://github.com/extremeheat/JSPyBridge
[Lars Bak]: https://en.wikipedia.org/wiki/Lars_Bak_(computer_programmer)
[Mosquitto]: https://mosquitto.org/
[MQTT]: https://en.wikipedia.org/wiki/MQTT
[Nick O'Leary]: https://github.com/knolleary
[Node.js]: https://github.com/nodejs
[Node-BLUE]: https://github.com/daq-tools/node-blue
[Node-RED]: https://github.com/node-red/node-red
[node-red-contrib-flow-manager]: https://flows.nodered.org/node/node-red-contrib-flow-manager
[node-red-contrib-yaml-storage]: https://flows.nodered.org/node/node-red-contrib-yaml-storage
[node-red-embedded-start]: https://www.npmjs.com/package/node-red-embedded-start
[Ondrej Lehota]: https://github.com/barshociaj
[Python]: https://en.wikipedia.org/wiki/Python_(programming_language)
[Roger Light]: https://github.com/ralight
[Ryan Dahl]: https://github.com/ry
[V8]: https://en.wikipedia.org/wiki/V8_(JavaScript_engine)
[YAML]: https://en.wikipedia.org/wiki/YAML
