# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['formsite_util', 'formsite_util.legacy']

package_data = \
{'': ['*']}

install_requires = \
['aiohttp>=3.8.1,<4.0.0',
 'colorama>=0.4.4,<0.5.0',
 'pandas>=1.3,<2.0',
 'pytz>=2021.3,<2022.0',
 'requests>=2.27.1,<3.0.0',
 'tqdm>=4.62.3,<5.0.0']

entry_points = \
{'console_scripts': ['getform = formsite_util.cli:main']}

setup_kwargs = {
    'name': 'formsite-util',
    'version': '2.0.2',
    'description': 'Python library and CLI tool for interacing with the FormSite API',
    'long_description': '<!-- HEADER -->\n<br />\n<p align="center">\n  <a>\n    <img src="https://www.formsite.com/wp-content/themes/formsite-theme/assets/favicon/favicon-96x96.png" alt="Logo" width="80" height="80">\n  </a>\n\n  <h3 align="center">Formsite Utility</h3>\n</p>\n\nPython library and CLI tool for interacing with the FormSite API\n\n[![Codacy Badge](https://app.codacy.com/project/badge/Grade/31145f1b97934080981a53783803701f)](https://www.codacy.com/gh/strny0/formsite-utility/dashboard?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=strny0/formsite-utility&amp;utm_campaign=Badge_Grade)\n[![PyPI version](https://badge.fury.io/py/formsite-util.svg)](https://badge.fury.io/py/formsite-util)\n![PyPI - Downloads](https://img.shields.io/pypi/dm/formsite-util)\n\n## Quickstart\n\n### Install\n\n```bash\n$ pip install formsite-util\n```\n\n### CLI\n\nTOKEN: Formsite API access token, acquired from account settings\n\nLook at the formsite URL when accessing one of your forms\n\nhttps:/<span></span>/`SERVER`.formsite.com/`DIRECTORY`/`FORM_ID`\n\nExport a form:\n\n```bash\n$ getform -t TOKEN -s SERVER -d DIRECTORY -f FORM_ID -o ./my_export.csv\n```\n\nDownload files uploaded to a form:\n\n```bash\n$ getform -t TOKEN -s SERVER -d DIRECTORY -f FORM_ID -D ./download_dir/\n```\n\n### Module\n\n```python\nfrom formsite_util import FormsiteForm, FormsiteFormsList\n\n\n# Formsite Form\nform = FormsiteForm(FORM_ID, TOKEN, SERVER, DIRECTORY)\nform.fetch()\n\ndf = form.data # data with columns as column IDs\ndf = form.data_labels # data with columns as the actual labels\n\n# work with df\n...\n\n# Formsite Forms List\nformlist = FormsiteFormsList(TOKEN, SERVER, DIRECTORY)\nformlist.fetch()\n\nforms = formlist.data\n# work with forms\n...\n```\n\n## Overview\n\nThis program performs an export of a specified formsite form with parameters\nA faster alternative to a manual export from the formsite website. It uses the formsite API v2 to perform exports of results. You can specify parametrs in commandline to filter these results. Allow advanced features like link extraction and even download of files to a specified directory.\n\nSupported python versions: | `3.8` | `3.9` | `3.10+` |\n\n### Dependencies\n\n```txt\naiohttp\nrequests\ntqdm\npytz\npandas\npyarrow # parquet and feather support\nopenpyxl # excel support\ntables # hdf support\n```\n\n## Usage\n\nYou can invoke the module with `getform [args]` (if it\'s in your [PATH](https://datatofish.com/add-python-to-windows-path/))\n\nor `py -m formsite_util.cli`\n\nor `python3 -m formsite_util.cli`\n\nYou can access the help page with `getform -h`\n\n## **CLI Documentation:**\n\n### **Authorization arguments:**\n\nAuthorization arguments are required for all operations.\n\n```bash\ngetform -t \'TOKEN\' -s \'SERVER\' -d \'DIRECTORY\'\n```\n\n**Token**: Your Formsite API token.\n\n**Server**: Your Formsite server. A part of the url. https:/<span></span>/`fsX`.forms… <- the \'fsX\' part. For example \'fs4\'.\n\n**Directory**: Can be found under [Share > Links > Directory] It is the highlighted part of the url: https:/<span></span>/fsXX.formsite.com/`directory`/hfjfdyjyf\n\n### **Results parameters:**\n\nResults parameters are used to set filters upon the results you are retreiving from a specific form.\n\n**Form**: Can be found under [Share > Links > Directory] It is the highlighted part of the url: https:/<span></span>/fsXX.formsite.com/gnosdH/`form_id` or by running `getform -l` to list all forms and their IDs\n\n#### **Filter by Reference #**\n\nYou can provide arguments `--afterref *int* and --beforeref *int*` to specify an interval of which reference numbers to get from your export. It is the same as setting a filter based on reference number when doing an export from the formsite website.\n\nExample:\n\n```bash\n$ getform -t \'token\' -d \'directory\' -s \'server\' -f \'form_id\'  \\\n--afterref 14856178 --beforeref 15063325 \\\n-o\n```\n\nThis will output results in between reference numbers `14856178` and `15063325`.\n\n#### **Filter by Date**\n\nYou can provide arguments `--afterdate *date* and --beforedate *date*` to specify an interval of which dates to get from your export. It is the same as setting a filter based on date number when doing an export from the formsite website.\n\nValid datetime formats are `ISO 8601`, `yyyy-mm-dd` or `yyyy-mm-dd HH:MM:SS`\n\n```txt\nISO 8601 example\nyyyy-mm-ddTHH:MM:SSZ\n2021-01-20T12:00:00Z is 20th January, 2021 at noon in UTC timezone\n```\n\nExample:\n\n```bash\n$ getform -t \'token\' -d \'directory\' -s \'server\' -f \'form_id\'  \\\n--afterdate \'2021-01-01T00:00:00Z\' ---beforedate \'2021-02-01T00:00:00Z\' \\\n-o\n```\n\nThis will retrieve all results in for the month of January 2021 in UTC.\n\n#### **Timezone:**\n\nThe dates provided are in UTC. This can become a problem if your organizations formsite account is set to a particular timezone you are not in, especially when deleting results/attachments using date filters.\n\nYou can set a manual offset with the `-T` or `--timezone` arguments.\nValid input is a timezone database name such as `America/Chicago`. This will shift formsite statistics dates *(columns Date | Start time | Finish time)* and your before/after date argument to your input timezone to the same UTC time but in the target timezone.\n\nExample usage: I want to get results for a certain day in non-UTC timezone. Passing `-T \'America/Chicago\'`, my input args `--afterdate 2021-04-10 --beforedate 2021-4-11` will become `2021-04-09 18:00:00` and `2021-04-10 18:00:00`. Additionally, columns Date, Start time, Finish time will also be shifted.\n\nExample:\n\n```bash\n$ getform -t \'token\' -d \'directory\' -s \'server\' -f \'form_id\' --afterdate \'2021-02-01T00:00:00Z\' --beforedate \'2021-02-01T06:30:00Z\' -T \'America/New_York\' -o\n```\n\n#### **Sorting:**\n\nInvoked with `--sort` [asc|desc] ex: `--sort desc`\n\nSorts rows based on Reference # column. Defaults to descending.\n\n#### **Results labels and views:**\n\nResults labels are the names of columns of items you put on your form. The default option is to use the default question labels you would put on your form, such as "What is your email address?" or "Your email:" or simply "E-mail:" but for actually working with the data, it is easier and clearer to use a feature formsite supports called Results labels, where you can set your own names of these columns for exports.\n\nYou can find the ID of your labels under `[Form Settings > Integrations > Formsite API]`\n\nYou can set them with the `--resultslabels *id*` argument. Defaults to 10 which are typically the first results label you add.\n\nYou can also set results view with `--resultsview *id*` argument. Defaults to 11 which is all items + statistics.\n\n### **Outputing to a file:**\n\nYou can use the `-o` flag to output your export to a file. If you don\'t specify this flag, no results will be outputted. For reasons as to why you wouldn\'t include this flag, please see ***File downloads:*** below.\n\nYou can leave the `-o` flag by itself or you can specify a path, either relative or absolute to a file you want to output to. If you don\'t include a path, it will default to its current directory, the file will be a csv with the name in the following format:\n\n```txt\nexport_formID_date.csv | example: export_formID_2021-04-08--20-00-18.csv\n```\n\nThe output file changes completely based on what extension you give it. Supported filetypes are:\n\n | `.csv`\n | `.xlsx`\n | `.parquet`\n | `.feather`\n | `.pkl | .pickle`\n | `.hdf`\n\nExample:\n\n```bash\n$ getform -t \'token\' -d \'directory\' -s \'server\' -f \'form_id\'  \\\n-o \'./exports/export_filename.csv\'\n```\n\n```bash\n$ getform -t \'token\' -d \'directory\' -s \'server\' -f \'form_id\'  \\\n-o \'./output.xlsx\'\n```\n\n#### **CSV Encoding:**\n\nSpecify encoding of the output file (if output format supports it). Defaults to \'utf-8-sig\'.\n\nInvoked with `--encoding utf-8`\n\n#### **CSV Delimiter/Separator:**\n\nSpecify separator of the output file (if output format supports it). Defaults to \',\' (comma)\n\nInvoked with `--separator \',\'`\n\n#### **CSV Line ending:**\n\nSpecify line terminator of the output file (if output format supports it).\n\nCan be one of: { LF, CR, CRLF, os_default }\n\nInvoked with: `--line_terminator os_default`\n\n#### **CSV Quoting:**\n\nSpecify quoting level of the output file (if output format supports it). Defaults to \'MINIMAL\'\n\nMore info about the quoting levels: <https://docs.python.org/3/library/csv.html>\n\nCan be one of: { QUOTE_ALL, QUOTE_MINIMAL, QUOTE_NONNUMERIC, QUOTE_NONE }\n\nInvoked with: `--quoting QUOTE_MINIMAL`\n\n#### **Date format:**\n\nInvoked with `--date_format format` where format is a string of python datetime directives.\n\nDefaults to `\'%Y-%m-%d %H:%M:%S\'` which is `yyyy-mm-dd HH:MM:SS`\n\nYou can find the possible format directives here: <https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior>\n\nSets the date format of all formsite statistics dates such as the Date column. Only applies to CSV output files as excel handles it by itself.\n\n### **File downloads:**\n\nYou can use the `-x` flag to extract all links that begin with the formsite base url into a text file, such as:\n\n```txt\nhttps://fsXX.formsite.com/directory/files/f-XXX-XXX-reference#_filename.ext\n```\n\nThe links file defaults to the same directory as the `getform.py` with the file name `links_{formID}_{timestamp}.txt`\n\nYou can use the `-D` option to download all links to a directory you specify, just like with the `-o` argument. The file will be saved with the filename in the link `reference#_filename.ext`\n\nInvoked with: `-D` or `--download` \'path/to/folder\' | `-x` or `--extract` path/to/file\n\nExample:\n\n```bash\n$ getform -t \'token\' -d \'directory\' -s \'server\' -f \'form_id\'  \\\n-D \'./download_03/\' -xre \'\\.jpg$\'\n```\n\n*^Example (will create a directory download_03 in the folder where you run it and save all files that end with .jpg uploaded to the form)*\n\n#### **Links regex:**\n\nYou can also specify a regex to filter links. You will only get links that contain a match of a regex you provide with the `-xre` or `--extract_regex` option:\n\n```txt\nex: \\.json$   - will only return files that end with .json\n```\n\n#### **Max concurrent downloads:**\n\nYou can specify how many concurrent downloads can take place at any given moment.\n\nInvoked with: `-c x` or `--concurrent_downloads x`\n\nDefaults to `10`\n\n#### **Stop file overwrite:**\n\nIf you include this flag, files with the same filenames as you are downloading in the target folder will not be overwritten and re-downloaded.\n\nInvoked with: `-n` or `--dont_overwrite_downloads`\n\n#### **Timeout:**\n\nTimeout in seconds for each individual file download.\n\nIf download time exceeds it, it will throw a timeout error and retry up until retries.\n\nInvoked with: `--timeout x`\n\nDefaults to `160` seconds\n\n#### **Retries:**\n\nNumber of times to retry downloading files if the download fails.\n\nInvoked with: `--retries x`\n\nDefaults to `3`\n\n#### **Strip prefix:**\n\nIf you enable this option, filenames of downloaded files will not have f-xxx-xxx- prefix.\n\nInvoked with: `--strip_prefix`\n\n#### **Filename regex:**\n\nIf you include this argument, filenames of the files you download from formsite servers will remove characters that dont match the regex from their filename.\n\nInvoked with: `-Dre` or `--download_regex`\n\nExample: `-Dre \'[^\\w\\_\\-]+\'` will only keep alphanumeric chars, underscores and dashes in filename.\n\n*^(in case of filename colissions, appends _number)*\n\n### **Other arguments:**\n\n#### **Special options:**\n\n`-h --help` - shows a help message and exits\n\n`--disable_progressbars` or `-P` If you use this flag, program will not display progressbars to console\n\n`-l --list_forms` - prints all forms, can be saved as csv if you provide a path to a file (`-l list.csv`).\n\n*^You can pair this with `getform (...) -l | grep form name` to find form ID easily.*\n\n*Or pipe it into less `getform (...) -l | less` to browse it*\n\n\n`-V --version` - displays the current version and exits\n\n`-v --verbose` - displays logger *(level DEBUG)* information to stdout, disables progress bars\n\n## **Module Examples:**\n\n### formsite_util\n\nThe formsite-util package provides several interfaces for common tasks.\n\n### High level interfaces\n\nFormsiteForm: Represents the form data and session\n\nFormsiteFormsList: Represents the list of all forms for the specified account\n\nFormsiteParameters: Represents parameters for results/items requests\n\n### Low level interfaces\n\nFormFetcher: Result/Item fetching operations *(useful)*\n\nFormParser: Result/Item parsing operations *(useful)*\n\nFormData: Represents the base form data class *(not very useful)*\n\nFormsiteLogger: Custom logger you may connect to your own logging *(not very useful)*\n\n## Module example usage\n\n### FormsiteForm example usage\n\n```python\nfrom formsite_util import FormsiteForm, FormsiteParameters\n\ntoken = "efwfjwi0fj0W4JG340G343G" # not real token\nserver = "fs1"\ndirectory = "aqWfcw"\n\n# Get all data for the month of January 2022\nparams = FormsiteParameters(\n    after_date="2022-01-01T00:00:00Z", \n    before_date="2021-02-01T00:00:00Z",\n    )\n\nform = FormsiteForm(form_id, token, server, directory)\nform.fetch(params=params)\nform.data_labels.to_csv(\'./2022_jan.csv\')\n```\n\n### FormsiteFormsList example usage\n```python\nfrom formsite_util import FormsiteFormsList, FormsiteForm\n\n# Iterate through all forms for the account and fetch them\nflist = FormsiteFormsList(token, server, directory)\nflist.fetch()\n\nfor form_id in flist.data[\'form_id\'].to_list():\n    form = FormsiteForm(form_id, token, server, directory)\n    form.fetch()\n    form.to_csv(f\'./expors/{form_id}_data.csv\')\n```\n\n### Caching example\n\nWhen fetching a form, it is wasteful to fetch everything again.\n\nWe wish to spare the API calls due to the rate limit.\n\nFor this reason, you may use caching of the items and results.\n\n```python\nfrom formsite_util import FormsiteForm\n\nform = FormsiteForm(form_id, token, server, directory)\n\n\n# By adding these parameters to the fetch request...\nform.fetch(\n    cache_items_path=f\'./cache_dir/{form_id}_items.json\',\n    cache_results_path=f\'./cache_dir/{form_id}_data.parquet\',\n)\n\n# For Items:\n#   - You are only re-fetching them if the results labels column IDs don\'t match the results column IDs (ie. only when necessary)\n\n\n# For Results:\n#   - You are only fetching results since the latest \'id\' (aka Reference #) and merging them back\n```\n\n### Connecting logging\n\n```python\nimport logging\n\n# Create your desired handler object\n# Example Stream handler\nhandler = logging.StreamHandler(sys.stdout)\nfmt = logging.Formatter("%(message)s")\nhandler.setLevel(logging.DEBUG)\nhandler.setFormatter(fmt)\n\n# Add it to the form.logger object using the addHandler method\nform.logger.addHandler(handler)\n\n# NOTE: You only need to do this once for all form objects.\n#       FormsiteLogger is a singleton, only 1 instance may exist at any time.\n#       To do this once at the start of your program, you can do this:\n\nfrom formsite_util.logger import FormsiteLogger\nl = FormsiteLogger()\nl.addHandler(...)\n\n```\n\n### FormsiteForm class\n\n```python\nfrom formsite_util import FormsiteForm, FormsiteParameters\n\nform = FormsiteForm(...)\nparams = FormsiteParameters(...)\n\nform.fetch(fetch_results=True, # Populates form.data\n           fetch_items=True,   # Populates form.items, form.labels\n           fetch_delay=3.0,    # delay between each API call in seconds\n           params=params,      # Results parameters\n           result_labels_id=0, # Items labels ID found in [Form settings -> Integrations -> Formsite API]\n           )\n\ndf1 = form.data \n# DataFrame with columns as column IDs used by formsite\n# returns data if results were fetched\n\ndf2 = form.data_labels \n# DataFrame with columns as labels of (result_labels_id)\n# data_labels only returns data, if both results and items were fetched\n\nlabels_dict = form.labels \n# The mapping of column ID -> label\n# returns data if items were fetched\n\nitems_dict = form.items \n# The original, unparsed formsite items object\n# returns data if items were fetched\n\n\nform.extract_urls(...)\n# Extracts URLs to all files uploaded to the form using bult in controls\n\nform.downloader(...)\n# Launches a syncrhonous downloader of the files uploaded to the form\n\nform.async_downloader(...)\n# Launches an async downloader of the files uplaoded to the form\n\nform.to_csv(...)\n# Saves the form data or data_labels to a CSV file with the following default settings:\n#   - by default save data_labels\n#   - encoded as utf-8-sig (utf-8 with BOM)\n#   - index=False\n#   - %Y-%m-%d %H:%M:%S datetime format\n\nform.to_excel(...)\n# Saves the form data or data_labels to an Excel file with the following default settings:\n#   - by default save with data_labels\n#   - index=False\n\n```\n\n### FormsiteFormsList class\n\n```python\nfrom formsite_util import FormsiteFormsList\n\nflist = FormsiteFormsList(...)\n\nflist.fetch()\n\nflist.data\n# DataFrame in the format:\n# form_id | name | state | results_count | files_size | files_size_human | url\n# ...\n```\n\n## Notes\n\nMore info can be found at Formsite API v2 help page\n\n**<https://support.formsite.com/hc/en-us/articles/360000288594-API>**\n\nYou can find API related information of your specific form under:\n\n```txt\n[Form Settings > Integrations > Formsite API] on the Formstie website\n```\n\n**API response error codes table:**\n\n| code | description                                 |\n|------|---------------------------------------------|\n| 401  | Authentication info is missing or invalid.  |\n| 403  | Forbidden.                                  |\n| 404  | Path or object not found.                   |\n| 422  | Invalid parameter.                          |\n| 429  | Too many requests or too busy.              |\n| 5xx  | Unexpected internal error.                  |\n\n## License\n\n© 2022 Jakub Strnad\n\nMIT License\nPlease see LICENSE.<span></span>md for more details.\n',
    'author': 'Jakub Strnad',
    'author_email': 'jakub.strnad@protonmail.com',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://github.com/strny0/formsite-utility/',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'entry_points': entry_points,
    'python_requires': '>=3.8,<4.0',
}


setup(**setup_kwargs)
