Metadata-Version: 2.1
Name: bytebridge
Version: 0.0.1
Summary: A data tool designed to move data seamlessly between various sources and destinations.
Author-email: Rafael Vargas <contact@rafaelvargas.dev>
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Requires-Dist: psycopg[binary] >=3.0.9
Requires-Dist: pyarrow >=7.0.0
Requires-Dist: mysql-connector-python >=8.0.19
Requires-Dist: ruff ==0.1.5 ; extra == "dev"
Requires-Dist: pre-commit >=2.17.0 ; extra == "dev"
Provides-Extra: dev




<p align="center" style="padding: 20px">
  <img src="static/branding/logo.svg" style="width:450px" alt="bytebridge">
</p>

<p align="center" style="padding-bottom: 20px">
    <em>A data tool designed to move data seamlessly between various sources and destinations.</em>
</p>

## CLI

Bytebridge aims to have a CLI that can be used to easily transfer data from multiple sources. Some examples are shown below:

### Parquet to PostgreSQL

```bash
bytebridge transfer \
    --connections connections.json \
    --source parquet_conn \
    --source-object data.parquet \
    --target postgresql_conn \
    --target-object bytebridge.public.data
```
### PostgreSQL to Parquet

```bash
bytebridge transfer \
    --connections connections.json \
    --source postgresql_conn \
    --source-object bytebridge.public.data \
    --target postgresql_conn \
    --target-object data.parquet
```

In both cases, the connections metadata are defined in the `connections.json` file. An example of the file definition is:

```json
{
    "parquet_conn": {
        "type": "parquet"
    },
    "postgresql_conn": {
        "type": "postgresql",
        "parameters": {
            "host": "[hostname_of_the_connection]",
            "user": "[postgresql_username_to_be_used]",
            "password": "[postgresql_password_to_be_used]"
        }
    }
}
```


## API

Coming soon.

## Data Connectors

### Currently Supported

| Name             | Type          | Client                                                                         |
|----------------- | --------------|--------------------------------------------------------------------------------|
| PostgreSQL       | Database      | [psycopg](https://pypi.org/project/psycopg/)                                   |
| MySQL            | Database      | [mysql-connector-python](https://pypi.org/project/mysql-connector-python/)     |
| Parquet          | File          | [pyarrow](https://pypi.org/project/pyarrow/)                                   |



### Planned

| Name             | Type          |
|----------------- | --------------|
| SQLite           | Database      |
| SQL Server       | Database      |
| Oracle           | Database      |
| Clickhouse       | Database      |
| CSV              | File          |
| ORC              | File          |
| Avro             | File          |
| Excel (XLSX)     | File          |



## Contributing

Feel free to contribute to this project. See the contribution guidelines in [here](CONTRIBUTING.md).

## License

This project is licensed under the terms of the [Apache 2.0 license](LICENSE).
