# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['dora_parser', 'dora_parser.dialects']

package_data = \
{'': ['*']}

install_requires = \
['moz-sql-parser>=4.40.21126,<5.0.0',
 'pandas>=1.3.0,<2.0.0',
 'seaborn>=0.11.1,<0.12.0']

setup_kwargs = {
    'name': 'dora-parser',
    'version': '0.1.1',
    'description': 'SQL Parser ans Transpiler',
    'long_description': '# Parser\n\nDora\'s [parser](https://en.wikipedia.org/wiki/Parsing) and [transpiler](https://en.wikipedia.org/wiki/Source-to-source_compiler) tool for some big data **SQL** dialects, based on [Mozilla SQL Parser](https://github.com/mozilla/moz-sql-parser)  project.\n\n[![PyPI](https://img.shields.io/pypi/v/dora-parser)](https://pypi.org/project/dora-parser/) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/dora-parser)\n\n## Getting Started\n\nAn application that will translate source code from a given language (**Impala**, **Spark**, **Hive**, **Presto** and **Athena**) and produce equivalent code in another language that has a similar level of abstraction.\n\n## Installation\n\n```bash\npip install dora-parser\n```\n\n## Usage\n\n### Translate **Query**\n\nI. Import Modules\n\n```python\nfrom dora_parser.parser import Parser\nfrom dora_parser.transpiler import Transpiler\n```\n\nII. Generate the parse `tree` from `query`\n\n```python\ntree = Parser(query)\n```\n\nWhere `query` have the following value:\n\n```sql\nWITH t1 as (select now() att_day)\nSELECT DECODE(DAYOFWEEK(att_day)\n            , 1 , \'Monday\'\n            , 2 , \'Tuesday\'\n            , 3 , \'Wednesday\'\n            , 4 , \'Thursday\'\n            , 5 , \'Friday\'\n            , 6 , \'Saturday\'\n            , 7 , \'Sunday\'\n            , \'Unknown day\') "Day of week"\n  , TRUNC(MONTHS_ADD(att_day,2),\'SYEAR\') as "Trunc SYEAR"\n  , TRUNC(MONTHS_ADD(att_day,2),\'YEAR\') as  "Trunc YEAR"\n  FROM t1;\n```\n\ncreates the tree object as shown below:\n\n```json\n{\n    "select": [\n        {\n          "value": {"decode": [\n            {"dayofweek": "att_day"},1,\n              {"literal": "Monday"},2,\n              {"literal": "Tuesday"},3,\n              {"literal": "Wednesday"},4,\n              {"literal": "Thursday"},5,\n              {"literal": "Friday"},6,\n              {"literal": "Saturday"},7,\n              {"literal": "Sunday"},\n              {"literal": "Unknown day"}]},\n          "name": "Day of week"},\n        {\n          "value": {"trunc": [{"months_add": ["att_day",2]},{"literal": "SYEAR"}]},\n          "name": "Trunc SYEAR"\n        },\n        {\n          "value": {"trunc": [{"months_add": ["att_day",2]},{"literal": "YEAR"}]},\n          "name": "Trunc YEAR"\n        }\n    ],\n    "from": "t1",\n    "with": {\n        "name": "t1",\n        "value": {\n          "select": {\n            "value": {"now": {}},\n            "name": "att_day"}}\n    }\n}\n```\n\nIII. Translate the `tree` object from your original SQL Dialect (*impala*) to the new one (*spark*)\n\n```python\ntranspiler = Transpiler(from_dialect=\'impala\', to_dialect=\'spark\')\nresult, errors = transpiler.translate(tree)\n```\n\nthe `result` value will be like\n\n```sql\nWITH t1 AS (SELECT NOW() AS att_day) \nSELECT CASE\n        WHEN DAYOFWEEK(att_day) = 1 THEN \'Monday\' \n        WHEN DAYOFWEEK(att_day) = 2 THEN \'Tuesday\' \n        WHEN DAYOFWEEK(att_day) = 3 THEN \'Wednesday\'\n        WHEN DAYOFWEEK(att_day) = 4 THEN \'Thursday\' \n        WHEN DAYOFWEEK(att_day) = 5 THEN \'Friday\' \n        WHEN DAYOFWEEK(att_day) = 6 THEN \'Saturday\' \n        WHEN DAYOFWEEK(att_day) = 7 THEN \'Sunday\' \n        ELSE \'Unknown day\' \n      END AS `Day of week`\n  , TRUNC(att_day + INTERVAL 2 MONTHS, \'SYEAR\') AS `Trunc SYEAR`\n  , TRUNC(att_day + INTERVAL 2 MONTHS, \'YEAR\') AS `Trunc YEAR`\nFROM t1\n```\n\nYou can also have access to a list with information about any `errors`, as well as where they occur.\nIn this example, The `TRUNC` function in **Spark** only works with a few data formats, so you can not use it with "*SYEAR*"\n\n```json\n[\n  {\n    "trunc": "MEDIUM:20:[{\'add\': [\'att_day\', {\'interval\': [2,\'MONTHS\']}]}, {\'literal\': \'SYEAR\'}]"\n  }\n] \n```\n\nInformation about the errors will also appear in the output log.\n\n```log\ndora_parser 2021-08-12 17:24:24,650 WARNING _TRUNC_ Spark \ndata formats:[\'YEAR\', \'YYYY\', \'YY\', \'QUARTER\', \'MONTH\', \'MM\', \'MON\', \'WEEK\']\ndora_parser 2021-08-12 17:24:24,650 WARNING resolve trunc NotImplemented: \n--TRANSPILER:MEDIUM:LEVEL20:\'trunc\'\n```\n\n#### Translate **Script**\n\nAre considered an *script* any type of *string* with **multiple SQL statements**\n\nI. Import Module\n\n``` py\nfrom dora_parser.reader import Reader\n```\n\nII. Translate\n\n```python\nscript="""\nINSERT INTO t.customer SELECT DCEIL(p_sale) FROM Customers;\n\nCOMPUTE STATS customer;\n\nSELECT staff_id, staff_name, CHAR_LENGTH(staff_name) AS lengthofname, COUNT(*) order_count  \nFROM sales.orders \nWHERE YEAR(order_date) = 2021 \nGROUP BY staff_id;\n"""\nreader = Reader(from_dialect=\'impala\',to_dialect=\'presto\')\nresult, errors, n_queries = reader.translate_script(script)\n```\n\nGive you as `result`\n\n```sql\nINSERT INTO t.customer SELECT CEIL(p_sale) FROM Customers;\n\n/* STATEMENT ERRORS:COMPUTE STATS*/\nCOMPUTE STATS customer;\n\nSELECT staff_id, staff_name, LENGTH(staff_name) AS lengthofname, \nCOUNT(*) AS order_count \nFROM sales.orders \nWHERE YEAR(order_date) = 2021 \nGROUP BY staff_id;\n```\n\nIII. Generate a summary list (*optional*)\n\n```python\nsummary = reader.create_summary(errors, n_queries)\n```\n\nvalue for `summary` variable:\n\n```json\n[\n  {"N_queries": 3}, \n  {"Success": 2}, \n  {"Failed": {"HARD": 1}}, \n  {"Er_types": ["compute stats"]}\n]\n```\n\n#### Translate multiple **Files**\n\nI. Import Module\n\n``` python\nfrom dora_parser.reader import Reader\n```\n\nII. Translate\n\n``` py\ndir_impala = \'scrpits/impala/\'\ndir_spark = \'scripts/spark\'\nreader = Reader(from_dialect=\'impala\', to_dialect=\'spark\', input_dir =dir_impala,output_dir=dir_spark)\nreader.translate_files()\n```\n\nThe translated files will be saved to folders in the output directory according to the result of the translation.\nIf you don\'t specify the output directory, the resulting folders will be in the input directory.\n\nIII. Generate a migration report (*optional*)\n\nYou can also have access to a report in HTML with an overview of the result of migration process.\nTo do this, set the **migration_report** argument equal to True".\n\n``` python\nreader = Reader(from_dialect=\'impala\', to_dialect=\'spark\', input_dir =dir_impala,output_dir=dir_spark, migration_report=True)\nreader.translate_files()\n```\n\nIV. Generate a summary (optional)\n\nIf you want to access a summary dictionary of the migration process, set the **summary_dict** argument equal to True, as in the example below:\n\n```python\nreader = Reader(from_dialect=\'impala\', to_dialect=\'spark\', input_dir =dir_impala,output_dir=dir_spark)\nreader.translate_files(summary_dict=True)\n```\n\n```json\n{\n    "Input_dir": "/scripts/impala",\n    "From_dialect": "impala",\n    "To_dialect": "spark",\n    "Sucess_files": 3,\n    "Failed_files": 1,\n    "Files": {\n        "row_benchmark.sql": [\n            {"N_queries": 7},\n            {"Success": 7},\n            {"Failed": {}},\n            {"Er_types": []}],\n        "customer_facts.sql": [\n            {"N_queries": 12},\n            {"Success": 12},\n            {"Failed": {}},\n            {"Er_types": []}],\n        "document.sh": [\n            {"N_queries": 1},\n            {"Success": 1},\n            {"Failed": {}},\n            {"Er_types": []}],\n        "orches.sql": [\n            {"N_queries": 9}, \n            {"Success": 5}, \n            {"Failed": {"HARD": 3}}, \n            {"Er_types": ["create role", "appx_median", "parser"]}]\n    }\n}\n```\n\n## Error Types\n\n+ `Parser`: Errors when generating the tree structure.\n+ `Transpiler`: Unimplemented functions and their levels of complexity.\n+ `Statement`: Commands that are not capable of being reproduced, since there are no equivalent in the target language.\n\n## Supported Languages\n\n+ Apache Impala 2.12\n+ Hive 3.12\n+ Spark 3.1\n+ Presto 0.217\n\n## Getting Help\n\nWe use GitHub [issues](https://github.com/doraproject/parser/issues) for tracking [bugs](https://github.com/doraproject/parser/labels/bug), [questions](https://github.com/doraproject/parser/labels/question) and [feature requests](https://github.com/doraproject/parser/labels/enhancement).\n\n## Contributing\n\nPlease read through this [contributing](.github/CONTRIBUTING.md) document to get start and before submitting any issues or pull requests to ensure we have all the necessary information to effectively respond to your contribution.\n\n---\n\n[Dora Project](https://github.com/doraproject) is a recent open-source project based on technology developed at [Compasso UOL](https://compassouol.com/)\n',
    'author': 'DataLabs',
    'author_email': 'time.dataanalytics.datalabs@compasso.com.br',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://github.com/doraproject/parser',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'python_requires': '>=3.7.1,<4.0.0',
}


setup(**setup_kwargs)
