Metadata-Version: 2.1
Name: esneft-tools
Version: 0.1.1
Summary: Utilities to analyse population and deprivation data.
Author: Stephen Richer
Author-email: stephen.richer@nhs.net
License: MIT License
        
        Copyright (c) 2022 NHS England and Improvement
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: repository, https://github.com/nhsx/p24-pvt-diabetes-inequal
Keywords: esneft_tools
Classifier: Natural Language :: English
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Healthcare Industry
Classifier: Development Status :: 1 - Planning
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Provides-Extra: geo
License-File: LICENSE

# ESNEFT Analysis - Diabetes Inequalities

[![status: experimental](https://github.com/GIScience/badges/raw/master/status/experimental.svg)](https://github.com/GIScience/badges#experimental)

## Table of contents

  * [Installation](#installation)
  * [Setup](#setup)
  * [Retrieve Data](#retrieve-public-data)
    * [Download](#download)
    * [Process](#process)
      * [Aggregate By Practice Level](#aggregate-by-practice-level)
      * [Aggregate By LSOA Level](#aggregate-by-lsoa-Level)
  * [Visualise](#visualise)
    * [Practice Map](#practice-map)
    * [LSOA Map](#lsoa-map)
  * [Further Documentation](#additional-documentation)
  * [Contributing](#contributing)
  * [License](#license)
  * [Contact](#contact)


## Installation
Installation is possible via `pip` as shown below.

Unix/macOS
```bash
python3 -m pip install esneft_tools
```

Windows
```bash
py -m pip install esneft_tools
```

#### Install within a Virtual Environment (optional)
<details>
<summary><strong>Unix/macOS</strong></summary>

```bash
python -m venv esneft_tools
source esneft_tools/bin/activate
python3 -m pip install esneft_tools
```
</details>

<details>
<summary><strong>Windows</strong></summary>

```bash
py -m venv esneft_tools
esneft_tools/Scripts/Activate.ps1
py -m pip install esneft_tools
```

If running scripts is disabled on your system then run the following command before activating your environment.

```bash
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
```
</details>


#### Geospatial Dependencies (optional)
Additional geospatial utilities may be optionally installed as below.
Note these packages have non-trivial dependencies and automatic installation may not work on all systems.

An additional optional dependency, OSMnx, must be installed by the user.
Please refer to the [OSMnx documentation](https://osmnx.readthedocs.io/en/stable/) for further installation instructions.

<details>
<summary><strong>Clear here for code</strong></summary>

Unix/macOS
```bash
python3 -m pip install git+https://github.com/nhsx/p24-pvt-diabetes-inequal#egg=esneft_tools[geo]
```

Windows
```bash
py -m pip install git+https://github.com/nhsx/p24-pvt-diabetes-inequal#egg=esneft_tools[geo]
```

</details>


## Setup
The logging level of `esneft_tools` can be set via the `setVerbosity()` function.

```python
import logging
from esneft_tools.utils import setVerbosity
from esneft_tools import download, process, visualise

setVerbosity(logging.INFO)
```

## Retrieve Public Data

### Download

#### From Host
Each of the `esneft_tools.download.getData().fromHost()` functions retrieve a static copy of a particular data set from GitHub.
A local copy of these tables is saved to `./.data-cache/` by default.
Each can be obtained individually but it is recommended to retrieve all data, as below.

```python
# Instantiate data download class.
getData = download.getData(cache='./.data-cache')

# Retrieve all data as dictionary (recommended)
data = getData.fromHost('all')
```

* `all` **(default)**
  * Retrieve all of the below data in dictionary format (**recommended**).
*  `postcodeLSOA`
  * Postcode -> LSOA (2011) lookup Table from [ArcGIS](https://hub.arcgis.com/datasets/6a46e14a6c2441e3ab08c7b277335558/about)
*  `imdLSOA`
  * Indices of Multiple Deprivation by LSOA in England from [National Statistics (.gov.uk)](https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/845345/File_7_-_All_IoD2019_Scores__Ranks__Deciles_and_Population_Denominators_3.csv/preview)
*  `populationLSOA`
  * LSOA population estimates, by age and sex, from [ONS](https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/datasets/lowersuperoutputareamidyearpopulationestimates)
* `ethnicityLSOA`
  * Proportion of ethnicity minorities (non-white) by LSOA from [NOMIS](https://www.nomisweb.co.uk/query/select/getdatasetbytheme.asp?opt=3&theme=&subgrp=)
*  `areaLSOA`
  * Land hectare measures by LSOA from [ArcGIS](https://hub.arcgis.com/datasets/ons::standard-area-measurements-2011-for-2011-census-areas-in-england-and-wales/about)
* `gpRegistration`
  * GP registration by LSOA [NHS Digital](https://digital.nhs.uk/data-and-information/publications/statistical/patients-registered-at-a-gp-practice/metadata)
* `gpPractice`
  * GP Practice information from [NHS Digital](https://digital.nhs.uk/services/organisation-data-service/file-downloads/gp-and-gp-practice-related-data)
* `gpStaff`
  * GPs by GP Practices from [NHS Digital](https://digital.nhs.uk/services/organisation-data-service/file-downloads/gp-and-gp-practice-related-data)
* `qof`
  * Quality and Outcomes Framework, 2021-22 from [NHS Digital](https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data/2021-22)
* `geoLSOA`
  * LSOA GeoJSON from [UK Data Service](https://statistics.ukdataservice.ac.uk/dataset/2011-census-geography-boundaries-lower-layer-super-output-areas-and-data-zones)
* `esneftLSOA`
  * List of LSOAs within ESNEFT trust.
* `esneftOSM`
  * OpenStreetMap (OSM) data for ESNEFT area from [Geofabrik](https://download.geofabrik.de/europe/great-britain/england.html)

#### From Source
Alternatively, the `.fromSource()` method can be used to retrieve each dataset directly from its public source.
By default, `esneft_tools` will retrieve the source data that matches the `fromHost()` methods.
Each source URL can be updated to newer versions by providing a YAML file to `download.getData()` as shown below.
An example of the YAML file is shown [here](./README_files/sourceURL.yaml).

```python
# Instantiate data download class.
getData = download.getData(sourceURL='sourceURL.yaml', cache='./.data-cache')

# Retrieve a specific dataset from host.
imdLSOA = getData.fromSource('imdLSOA')
```


### Processing

#### Aggregate by Practice Level
The `getGPsummary` function aggregates the downloaded data to practice level statistics.

```python
GPsummary = process.getGPsummary(**data, iod_cols='IMD')
```

| Field               | Description                                               |
| ---                 | ---                                                       |
| *OrganisationCode*  | Practice Service Code                                     |
| IMD                 | Mean Index of Multiple Deprivation of Registered Patients |
| Patient             | Total Registered Patients                                 |
| QOF-DM              | QOF achievement for Diabetes mellitus (max 76)            |
| DM-prevalance       | Prevalence of Diabetes mellitus                           |
| PCDS                | Postcode                                                  |
| OpenDate            | Opening Date                                              |
| CloseDate           | Closing Date                                              |
| Status              | Service Status (e.g. Active)                              |
| PrescribingSetting  | Service Type (e.g. GP Practice)                           |
| currentStaff        | Current Practitioners                                     |
| departedStaff       | Total Departed Practitioners                              |
| meanStaff           | Average Working Practitioners                             |
| annualStaffTurnover | Mean Annual % Practitioner Turnover                       |
| Lat                 | Latitude of Site                                          |
| Long                | Longitude of Site                                         |
| patientPerGP        | Total Registered Patient per Average Practitioner Count   |
| ESNEFT              | Boolean Flag of Practices within ESNEFT                   |
| Node                | Closest OSM Map Node to Site                              |


#### Aggregate by LSOA Level
The `getLSOAsummary` function aggregates the downloaded data LSOA level statistics.

```python
LSOAsummary = process.getLSOAsummary(**data, iod_cols='IMD')
```

| Field         | Description                               |
| ---           | ---                                       |
| *LSOA11CD*    | LSOA (2011) Code                          |
| LSOA11NM      | LSOA (2011) Name                          |
| Age (median)  | Median Age of Population                  |
| Population    | Population Estimate (2011 Census)         |
| LandHectare   | Land Area (Hectares)                      |
| Patient       | Total Registered GP Patients              |
| IMD           | Index of Multiple Deprivation             |
| IMD (q5)      | Index of Multiple Deprivation (quintiles) |
| DM-prevalance | Prevalence of Diabetes mellitus           |
| Density       | Population Density                        |
| ESNEFT        | Boolean Flag of LSOAs within ESNEFT       |


### Visualise

### Practice Map

```python
fig = visualise.scatterGP(GPsummary[GPsummary['Status'] == 'Active'], minCount=250)
```

![gp-loc](./README_files/GP-locations.png)
 <br> *Map of Practice Distributions within ESNEFT (Plotly Interactive)*


### LSOA Map

```python
fig = visualise.choroplethLSOA(LSOAsummary, data['geoLSOA'], colour='IMD')
```

![gp-loc](./README_files/LSOA-choropleth.png)
 <br> *Choropleth Map of LSOA Domains within ESNEFT coloured by IMD (Plotly Interactive)*


## Further Documentation
Refer to the [additional documentation](./README_files/docs.md) for further examples of functionality.


## Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**.

1. Fork the Project
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the Branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request


## License
Distributed under the MIT License. _See [LICENSE](./LICENSE) for more information._


## Contact
If you have any other questions please contact the author, [Stephen Richer](mailto:stephen.richer@proton.me?subject=[GitHub]%20esneft_tools).
