Metadata-Version: 2.1
Name: validata-table
Version: 0.10.2
Summary: Package used in validation tabular data
Home-page: https://gitlab.com/validata-table/validata-table
License: GPL-3.0-or-later
Author: Amélie Rondot
Author-email: amelie.rondot@multi.coop
Requires-Python: >=3.9,<4.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU Affero General Public License v3
Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: cachetools (>=5.3.1,<6.0.0)
Requires-Dist: commonmark (>=0.9.1,<0.10.0)
Requires-Dist: flasgger (>=0.9.7.1,<0.10.0.0)
Requires-Dist: flask (==2.1.2)
Requires-Dist: flask-cors (>=4.0.0,<5.0.0)
Requires-Dist: flask-matomo (>=1.2.0,<2.0.0)
Requires-Dist: frictionless[excel] (==4.38)
Requires-Dist: gunicorn (>=21.2.0,<22.0.0)
Requires-Dist: importlib-metadata (>=6.8.0,<7.0.0)
Requires-Dist: jinja2 (==3.0.3)
Requires-Dist: opendataschema (>=0.6.0,<0.7.0)
Requires-Dist: opening-hours-py (>=0.6.17,<0.7.0)
Requires-Dist: phonenumbers (>=8.13.22,<9.0.0)
Requires-Dist: pydantic (==1.10.9)
Requires-Dist: pyright (>=1.1.336,<2.0.0)
Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
Requires-Dist: python-stdnum (>=1.19,<2.0)
Requires-Dist: requests-cache (>=1.1.0,<2.0.0)
Requires-Dist: sentry-sdk (>=1.31.0,<2.0.0)
Requires-Dist: shapely (>=2.0.1,<3.0.0)
Requires-Dist: toml (>=0.10.2,<0.11.0)
Requires-Dist: toolz (>=0.12.0,<0.13.0)
Requires-Dist: ujson (>=5.8.0,<6.0.0)
Requires-Dist: werkzeug (==2.0.2)
Project-URL: Repository, https://gitlab.com/validata-table/validata-table
Description-Content-Type: text/markdown

# Validata Table

[Validata Table](https://gitlab.com/validata-table/validata-table)  is python package used as a tabular data validation service.

It includes four subpackages, where you can find their respective documentations :
- [`validata_core`](src/validata_core/README.md)
- [`validata_ui`](src/validata_ui/README.md)
- [`validata_api`](src/validata_api/README.md)
- `tests` used for testing the project (in development environment or in continuous integration)

It offers also a command line tool `validata` used to validate tabular data. 
See [validata_core/README.md](src/validata_core/README.md) for more details.

# Using `validata-table` package

You can use locally this package `validata-table`, doing:
```commandline
pip install validata-table 
```
This allows you to use `validata` command line tool to validate tabular data:
```commandline
validata --help
```

# Development

This project is based on [Docker](https://docs.docker.com/get-started/overview/) to use a local environement developement.

This project includes a Makefile, which allows you to run predefined actions 
by running specific commands.

Dependency management tool used : [Poetry version 1.6.1](https://python-poetry.org/docs/)

### Requirements

First install [Docker Engine](https://docs.docker.com/engine/install/) and [docker-compose >= version 2](https://docs.docker.com/compose/install/) 
on your machine if not already done.

Then you may clone source code in your deployment directory:
```commandline
git clone https://gitlab.com/validata-table/validata-table.git
cd validata-table
```

### Run on development local environment

Configuration is done by editing environment variables in `.env`, 
(see `.env.example` file to set `.env` file)

Warning: Docker env files do not support using quotes around variable values!

Launch the development local environment, thanks to the makefile command:
```bash
# in validata-table/
make serve_dev
```
This launches two docker containers:
- validata-table-api-dev
- validata-table-ui-dev

### Validata Table API (using `validata-table-api-dev` docker container)
To access to the API of Validata Table click on http://localhost:5000/

[Try Validata Table API](src/validata_api/README.md)

### Validata Table UI (using `validata-table-ui-dev` docker container)
To access to the API of Validata Table click on http://localhost:5001/

### Validata Table command line tool (using `validata-table-api-dev` docker container)
To use validata command line tool in the docker development environment, run:
  ```
  docker exec -it validata-table-api-dev bash
  validata --help
  ```

### Test
To launch tests in the development environment, run:
  ```make test```

### Linting
Some code linting tools are configured for this project:
- [black](https://black.readthedocs.io/en/stable/): to format code, run `make black`
- [isort](https://pycqa.github.io/isort/): to format import, run `make isort`
- [flake8](https://flake8.pycqa.org/en/latest/): to enforce style coding, run `make flake8`
- [pyright](https://microsoft.github.io/pyright/#/): to check static types.
  `pyright` will be executed in local virtual environment with `poetry`:
  
  First you need to create a virtual environment .venv at the root of the project
and configure it:
  ```commandline
  # At /validata-table/ root 
  python3.9 -m venv .venv  # install virtual environement locally
  poetry config virtualenvs.in-project true
  poetry config --list # Check if correctly configured
  >>>
  ...
  virtualenvs.in-project = true
  ...
  
  poetry install  # install project dependencies
  ```
  Then execute locally `pyright` with `poetry`:
  ```commandline
  poetry run pyright .
  ```


# Continuous Integration

The continuous integration is configured in `.gitlab-ci.yml` file

## Release a new version

On master branch :
- Update version in [pyproject.toml](pyproject.toml) and [CHANGELOG.md](CHANGELOG.md) files
- Commit changes using `Release` as commit message
- Create git tag (starting with "v" for the release) `git tag -a`
- Git push: `git push && git push --tags`
- Check that pypi package is created and container images for validata_ui and validata_api are well-built 
([validata-table pipelines](https://gitlab.com/validata-table/validata-table/-/pipelines))

Creating and pushing a new release will trigger the pipeline in order to automatically publish 
a new version of `validata-table` package and build a new container image.

This pipeline runs when a new tag under the format 'vX.X.X' is pushed.


# Deploy to production
This project uses Docker to deploy it in a production environment.

First you may clone source code in your deployment directory:
```commandline
git clone https://gitlab.com/validata-table/validata-table.git
cd validata-table
```
Configuration is done by editing environment variables in `.env`, 
(see `.env.example` file to set `.env` file).

Warning: Docker env files do not support using quotes around variable values !

Launch the docker production environment with makefile:
```commandline
make serve_prod
```
OR launch the docker production environment with `docker compose` command:
```commandline
docker compose -f docker-compose.yml up --build -d
```
Then: 
- To access to the API of Validata Table click on http://localhost:<PORT_NUMBER_API>/
  (replacing PORT_NUMBER_API with the value you choose)
- To access to the UI of Validata Table click on http://localhost:<PORT_NUMBER_UI>/ 
  (replacing PORT_NUMBER_UI with the value you choose)
- To access to the `validata` command lines tool:
  ```
  docker exec -it validata-table-api bash
  validata --help
  ```
  NB: if, for any reasons to deploy to production, you use another Dockerfile using 
  the definition of a new user thanks to `USER` with some specific permissions,
  you could not use the `validata` command lines tool.

# History
To keep track of the project's history, [Validata Table](https://gitlab.com/validata-table/validata-table) 
comes from the merge of four gitlab repositories :
- [Validata core](https://gitlab.com/validata-table/validata-core)
- [Validata UI](https://gitlab.com/validata-table/validata-ui)
- [Validata API](https://gitlab.com/validata-table/validata-api)
- [Validata Docker](https://gitlab.com/validata-table/validata-docker)
