{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to the FluxEngine tutorials\n",
    "These tutorials are written using Jupyter, which allows the creation of notebooks that combine text embedded Python and/or command line code in a format which can be viewed from a web browser. This allows code to appear alongside their explanations and you can run code and modify examples live from within the web browser.\n",
    "\n",
    "Embedded Python code looks like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#this is a python example\n",
    "print(\"Hello Jupyter!\")"
   ]
  },
  {
   "attachments": {
    "run_button.png": {
     "image/png": "[encoded data, 14120 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run this example, click in the cell (box) containing the example code and then click the \"Run\" button on the tool bar at the top.\n",
    "\n",
    "![run_button.png](attachment:run_button.png)\n",
    "\n",
    "Any code in the selected cell will run and any output will appear below it - in this case the words \"Hello Jupyter!\" These code cells are editable allowing you to modify parts of the code and test your changes by re-running them. Try modifying the code above to print a different message and run it again.\n",
    "\n",
    "FluxEngine has several command line tools that provide different functionality. We'll be running these tools through Jupyter notebooks in order to keep everything together in a single document, but usually you would use Command Prompt (Windows) or Terminal (MacOS and Linux) to run them. In these notebooks, command line commands are distinguished from Python code by an exclamation mark (`!`) prefix before the command. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!echo \"Hello command line!\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running this will invoke the `echo` program from the command line, which is the equivalent to the `print` function in Python.\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Delayed output?</b> - On some systems you may not see the output from commands that you run until the command has finished running. This happens because your system is buffering the output and only displays it after the script finished executing. This won't happen if you run the tools directly from the command line, but it can be frustrating not knowing if some code you run in the Jupter notebook is still running. When you run a code cell you'll see the `In [ ]:` to the left of the cell change to `In [*]:` indicating it is actively running. When it finishes the asterisk will go away.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Setting the working directory\n",
    "When you first open a Jupyter notebook the working directory is set to the directory containing the Jupyter notebook. This will be inside a sub-directory of wherever you've installed your FluxEngine Python package. We want to create a new directory to keep all the tutorial files together (configuration files, FluxEngine output etc.) The script below will create the a `FluxEngineTutorials/tutorial_01` directory in your home directory. If you want to want to change this you can do so by modifying the line of the script starting with `workingDirectory = `. We will refer back to this directory as the tutorial's working directory so you should make a note of it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This code changes the working directory from the tutorial folder in your FluxEngine installation\n",
    "# to a newly created folder in your home drive\n",
    "import os;\n",
    "import matplotlib.pyplot as plt; #Used later for plotting\n",
    "\n",
    "homeDirectory = os.path.expanduser(\"~\");\n",
    "workingDirectory = os.path.join(homeDirectory, \"FluxEngineTutorials\", \"tutorial_01\"); #Change this line to change th default location for files you create in this tutorial.\n",
    "\n",
    "if os.path.exists(workingDirectory) == False: #If the new directory doesn't already exist, create it\n",
    "    os.makedirs(workingDirectory);\n",
    "os.chdir(workingDirectory);\n",
    "print(\"Working directory is now:\", os.getcwd());"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting setup with FluxEngine\n",
    "Hopefully you have already downloaded and installed FluxEngine prior to this workshop. It you haven't you should do this now. To download the latest version go to the git hub repository [here](https://github.com/oceanflux-ghg/FluxEngine/).\n",
    "\n",
    "FluxEngine uses several third party libraries and tools. A script is included in the download to automatically install these dependencies for MacOS and Linux, and installation instructions are included for Windows in the instructions (see section 3 of [the instructions](https://github.com/oceanflux-ghg/FluxEngine/blob/master/FluxEngineV3_instructions.pdf)). If you haven't done this already you should do this now."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Verifying FluxEngine has been installed correctly\n",
    "It is important to verify that FluxEngine, and its dependencies, have been installed correctly. This will prevent problems arising later, and gives us confidence that the flux calculation is being performed correctly by running a known scenario and comparing it to previously published data. This is all handled automatically using a verification script. Do this by running the command below. It will take about ten minutes to run and on most systems it will provide real-time output showing the script's progress. On some systems this real-time output won't appear and you'll instead get all the output displayed once the script finishes. You can continue reading the rest of the tutorial while you wait but don't run any code until the verification has finished running. Once it has finished check to make sure a message saying \"Validation successful! All values are within threshold limits\" is displayed.\n",
    "\n",
    "Note: Even if you've already ran the verification script, if you want to create the plot at the end of this tutorial you'll need to run it again. This is because the tutorial assumes there's a copy of the verification data in the current working directory. If you don't want to wait for the verification script to run again (about ten minutes), that's no problem but the final plot won't display."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Run the verification script.\n",
    "!fe_verify_socatv4.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Command line help</b> - If you've never used the command line to run something before, don't worry, it's easy. Command Prompt (on Windows, Terminal on MacOS and Linux) provides a way to type commands to your operating system instead of using the graphical user interface. This means you can run tools which do not have graphical user interfaces and also gives you a way to automate and document the analysis steps you have done (e.g. by saving the commands to a file). Lots of tools (especially free/open source scientific ones) don't have graphical user interfaces (including FluxEngine, for now at least) simply because they can be very time consuming to build and maintain.\n",
    "    \n",
    "The anatomy of a command line command looks like this:\n",
    "```\n",
    "program_to_run options\n",
    "```\n",
    "Where `program_to_run` is the name or path to the program or tool you want to run, and `options` is a list of options (sometimes prefixed with a dash `-` or double-dash `--`) containing the parameters you are using with the command. In the case of the command above, `!python fe_verify_socatv4.py`, the `!` character tells the Jupyter notebook that what follows is a command line instruction. `python` is the name of the program to run (the Python interpreter), and we specify a single option `fe_verify_socatv4.py`. Here, the option is the path of the script we want to run. The path is relative to our working directory, which we have already set to be folder we made to keep all the our tutorial files together. The whole command therefore runs FluxEngine's verification script using the Python interpreter. All the command line tools which come with FluxEngine are prefixed with `fe_` to help to identify them.\n",
    "</div> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating a custom configuration file\n",
    "For the remainder of this tutorial, we will create a new run configuration based on the verification run but with a number of changes and then visualise the output. To do this, you'll first need to understand FluxEngine configuration files.\n",
    "\n",
    "FluxEngine uses plain text configuration files to specify input data and to select options that change how the flux calculation is performed. We'll be using the configuration file from the verification run as the basis for our new FluxEngine run, but it's important not to modify the original file otherwise future verification runs may not work. You could copy the configuration file with your file browser, but here's some Python code to do that for you:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fluxengine.core.fe_setup_tools import get_fluxengine_root;\n",
    "from os import path, getcwd;\n",
    "import shutil;\n",
    "try:\n",
    "    shutil.copy(path.join(get_fluxengine_root(), \"configs\", \"socatv4_sst_salinity_gradients-N00.conf\"), path.join(getcwd(), \"custom_config.conf\"));\n",
    "    print(\"Configuration file successfully copied to:\", path.join(getcwd(), \"custom_config.conf\"));\n",
    "except Exception as e:\n",
    "    print(\"There was a problem copying the configuration file. Check your working directory is correct.\");\n",
    "    print(e);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Anatomy of a configuration file\n",
    "Now that we have made a new configuration file based on the verification run, let's open it. If you kept the default directory, you can find it in the `FluxEngineTutorials/tutorial_01/custom_config.conf` in your home directory. Use a text editor such as Notepad++ to open this file and view its contents.\n",
    "\n",
    "<div class=\"alert alert-block alert-warning\">\n",
    "<b>Opening config files: </b> Configuration files are plain text formatted files and can be opened in software such as Notepad, Notepad++ or TextEdit. You should not use Microsoft Word or other word processing software to edit configuration files because they sometimes insert invisible formatting characters to files which can prevent the FluxEngine interpretting the file correctly. On some older Windows systems configuration files may display all on one line. The easiest way to avoid this is by [installing Notepad++](https://notepad-plus-plus.org/download/v7.7.html) (a free lightweight text editor), and using this to edit configuration files.\n",
    "</div>\n",
    "\n",
    "Configuration files contain a list of options/setting names assigned to values using the format `option = value`. The order that options are defined does not matter but it is often useful to group related options together. Any text preceeded by a `#` symbol is a comment. Comments provide helpful information about what a particular setting, or group of settings, does. It isn't necessary to understand all of the settings at this point, but see if you can identify where the following settings are defined by reading the comments:\n",
    " - Where is input data specified? What types of input data are supplied (e.g. salinity data)?\n",
    " - Where is the flux calculation selected? Which flux equation (e.g. the 'rapid' or 'bulk') does the verification run use?\n",
    " - Where is gas transfer velocity parameterisation specified?\n",
    " - Where is the output directory set? Optional: Locate the output directory and open one of the files (see the information box below if you need for help opening netCDF files).\n",
    "\n",
    "If you want to find out more information about a particular setting, you can look at the description in [the instructions](https://github.com/oceanflux-ghg/FluxEngine/blob/master/FluxEngineV3_instructions.pdf), (see section 7.2).\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Opening netCDF files:</b> FluxEngine uses netCDF formatted files for input and creates new netCDF formatted files to store output. The easiest way to open netCDF files is to use Panoply (or similar software) which provides a graphical user interface to open, view and plot data in netCDF format. You can download Panoply for Mac, Windows or Linux [here](https://www.giss.nasa.gov/tools/panoply/). Alternatively, you can use a programming language such as Python, R or Matlab to read netCDF files and plot them. This is beyond the scope of these tutorials, and it is recommended that you use Panoply unless you already have experience reading and plotting netCDF files.\n",
    "</div> "
   ]
  },
  {
   "attachments": {
    "netcdf_anatomy.png": {
     "image/png": "[encoded data, 151916 characters]"
    },
    "netcdf_anatomy_pt2.png": {
     "image/png": "[encoded data, 113468 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modifying the config file\n",
    "\n",
    "### Specifying input data\n",
    "FluxEngine requires a minimum of six types of input data to perform the flux calculation. These are:\n",
    "- Sea surface temperature (skin, sub-skin or both)\n",
    "- Salinity\n",
    "- Air pressure\n",
    "- Wind speed\n",
    "- Aqueous gas (partial pressure, fugacity or concentration)\n",
    "- Atmospheric gas (partial pressure, fugacity or concentration)\n",
    "\n",
    "Each input dataset is described as a 'data layer', within FluxEngine. These are supplied as netCDF (.nc) files. NetCDF files can store several geospatial variables together in one file along with metadata (description, units, expected range, etc.) and information about the dimensions (spatial and temporal coordinates). The figure below shows a netCDF file opened in Panoply with the list of variables circled on the left and the dimensions that the data uses on circles on the right.\n",
    "\n",
    "![netcdf_anatomy.png](attachment:netcdf_anatomy.png)\n",
    "\n",
    "If you click on a variable you can see meta data about that variable, such as the units and dimensions. In the below picture we've selected the `sst_skin_mean` variable and we can read the metadata about the variable on the left hand side (units and dimensions are highlighted).\n",
    "\n",
    "![netcdf_anatomy_pt2.png](attachment:netcdf_anatomy_pt2.png)\n",
    "\n",
    "\n",
    "To define an input data layer in the FluxEngine configuration file we need to specify a file path and a 'product' name. The product name is just the name of the variable within the netCDF file (e.g. `sst_skin_mean` in the above example). An example from the current configuration file that specifies the wind speed input data is\n",
    "```\n",
    "windu10_path = <FEROOT>/data/verification_data/globwave/<YYYY>/<YYYY><MM>_OCF-WSP-GLO-1M-100-MGD-GW-v2.nc\n",
    "windu10_prod = wind_speed_cor_mean\n",
    "```\n",
    "\n",
    "We're defining the file path to our wind speed netCDF file using `windu10_path` and the product name using `windu10_prod`. Notice that tokens are used to represent the year and month (`<YYYY>` and `<MM>`). These are substituted for the numerical representation of the current year and month when FluxEngine runs, allowing different input files to be selected for different time steps. The `<FEROOT>` token is substituted for the installation directory of the FluxEngine package, and is used by internal configuration files to access data that comes with FluxEngine (in this case the data used to run the verification).\n",
    "\n",
    "For our custom flux calculation we will use a salinity dataset from the [World Ocean Atlas (WOA)](https://www.nodc.noaa.gov/SatelliteData/sss/). To do this we need to specify the location of the files containing the new salinity data, and the product name of the salinity variable within those files. Find the section of the configuration file which defines the salinity input and modify them to look like this:\n",
    "\n",
    "```\n",
    "salinity_path = copied_data/WOA_salinity/surface_woa18_A5B7_s<MM>_01.nc\n",
    "salinity_prod = salinity_mean\n",
    "```\n",
    "\n",
    "Now FluxEngine will look for salinity data in the relative path (i.e. from your current working directory). It will expect the data to be in a folder called `copied_data`. We need to create this folder and put the salinity data in there. The salinity data is included with FluxEngine, so we'll just use a Python script to copy it into our working directory (alternatively you could do this using your file browser).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fluxengine.core.fe_setup_tools import get_fluxengine_root; #Gets filepath to fluxengine root directory\n",
    "from os import mkdir, path, getcwd; #cross-platform filepath manipulation and access to current working directly\n",
    "import shutil;\n",
    "\n",
    "try:\n",
    "    shutil.copytree(path.join(get_fluxengine_root(), \"tutorials\", \"01_introduction\", \"data\"), path.join(getcwd(), \"copied_data\"));\n",
    "    print(\"WOA salinity data successfully copied to:\", path.join(getcwd(), \"copied_data\"));\n",
    "except Exception as e:\n",
    "    print(\"There was a problem copying the the WOA salinity data. The folder 'copied_data' may already exist. If so, try deleting it before rerunning this cell.\");\n",
    "    print(e);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now there will be a new directory named `copied_data` in the tutorial's working directory. Inside will be a copy of the WOA salinity data. You'll also notice that there is a separate file for each month. So that FluxEngine will use the correct month we used the `<MM>` token to specify the numerical representation of the month when defining the path to the data.\n",
    "\n",
    "If you open one of the WOA salinity data netCDF files using Panoply, there are three variables: `lon`, `lat` and `salinity_mean`. The first two define the lon-lat grid and the last contains the mean monthly sea surface salinity. We want to use the sea surface salinity, which is why we set `salinity_prod = salinity_mean`.\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Extra info:</b> There is a general pattern to specifying data layers, which is `datalayername_suffix`. You've already seen the `path` and `prod` suffixes used to define file paths and product names. Others can be used to define other properties of a data layer. For example, minimum and maximum allowed ranges for the input data, preprocessing such as unit conversions, or whether the input file has a temporal dimension. [The manual](https://github.com/oceanflux-ghg/FluxEngine/blob/master/documentation_FluxEngineV4.pdf) (section 7.6.1) provides more information on using suffixes to change the way specific input data layers are handled.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Selecting a gas transfer velocity parameterisation\n",
    "We'll use a more recent wind-based gas transfer velocity (k) parameterisation too. Find the line that starts `k_parameterisation`. It's currently set to use the parameterisation described by [Nightingale et al., 2000](https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/1999GB900091). There are lots of different gas transfer velocity parameterisations built into FluxEngine and you can use Python to create new ones. The most common parameterisations are listed in the comments of the configuration file and we'll stick to using one of these for now. [Wanninkhof (2014)](t) is a more recent parameterisation which is applicable to a wider range of ocean conditions. To configure FluxEngine to use this parameterisation set `k_parameterisation = k_Wanninkhof2014`.\n",
    "\n",
    "We'll also update the Schmidt number parameterisation to use the relationship described in the [same paper](https://aslopubs.onlinelibrary.wiley.com/doi/pdf/10.4319/lom.2014.12.351). Currently the configuration file doesn't define a Schmidt number parameterisation and so it will use the default version described by [Wanninkhof (1992)] which is a little outdated compared to the 2014 version. Add a new line that reads `schmidt_parameterisation = schmidt_Wanninkhof2014`. Now our gas transfer parameterisation and Schmidt number calculation will be consistent. Note that it doesn't matter where you add this line, but it might help you find it again if you add it below the k parameterisation line.\n",
    "\n",
    "### Setting the output directory\n",
    "Finally we need to change the output directory, otherwise we will overwrite the output produced by the verification run. Set the `output_dir` setting in the configuration file to `tutorial_output`. When FluxEngine runs our custom configuration file it will create a new subdirectory in the working directory called `tutorial_output` and store all of the output files here. By default, FluxEngine will sort these output files into further subdirectories based on year and month. We're only going to run our new configuration file for a single month so we don't really want all these extra directories. We can overwrite the output directory structure by adding the following line:\n",
    "\n",
    "`output_structure =`\n",
    "\n",
    "We purposefully leave the value empty to tell FluxEngine that we don't need any additional directories to be created. Now any output files will be put in the root output directory (i.e. the one defined in `output_dir`).\n",
    "\n",
    "Make sure you save the configuration file. For more information about each configuration file option you can look in section 7.2 of [the manual](https://github.com/oceanflux-ghg/FluxEngine/blob/master/FluxEngineV4_instructions.pdf)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Running FluxEngine using our custom configuration file\n",
    "All that's left to do now is to run FluxEngine with the new configuration file. There are two ways to run FluxEngine. The first is using the command line tool and specifying a configuration file, a start date and a stop date as options. This is the approach we will use because it doesn't assume knowledge of Python and because is is flexible enough to be used for most analyses. \n",
    "\n",
    "The second method is to import FluxEngine as a Python module and write a custom script to drive it, i.e. by using the `run_fluxengine` function from the `fluxengine.core.fe_setup_tools` module. The main advantage of using the Python module is that it allows FluxEngine to be embedded in other software, e.g. as part of a larger model or analysis pipeline. It is also possible access lower-level functions and gain a lot of flexibility over how FluxEngine runs. For example, you can hook into the way FluxEngine reads configuration files and make small runtime modifications to the parameters so that a single configuration file can be used as a template to perform a series of similar runs (e.g. as part of a sensitivity analysis).\n",
    "\n",
    "The command line tool can be run by running `python fe_run.py path/to/configuration/file.config -s startdate -e enddate` in your Command Prompt (Windows) or Terminal (MacOS/Linux) window. The first part of the command `python fe_run.py` tells the computer to use Python to run the command line tool which will in turn set up and run FluxEngine. The next part of the command specifies the path of the configuration file to use. Next the `-s` and `-e` specify the start and end dates, which must be provided in the one of the following formats: `YYYY`, `YYYY-MM-DD` or `YYYY-MM-DD hh:mm`. For more information on using the `fe_run.py` tool you can run it with the `-h` comment ('h' for help) or see [the instructions](https://github.com/oceanflux-ghg/FluxEngine/blob/master/FluxEngineV4_instructions.pdf) (section 5.1).\n",
    "\n",
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>The `!` prefix:</b> Remember that code which starts with `!` is interpretted as a command line instruction, and is equivalent to entering the command (without the `!`) into your command prompt (Windows) or terminal (Mac/Linux) window.\n",
    "</div>\n",
    "\n",
    "Run the command below to perform our new flux calculation. Notice, to reduce execution time we have set the end date to the end of January so that we only calculate the fluxes for a single month."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!fe_run.py \"custom_config.conf\" -s \"2010-01-01\" -e \"2010-01-31\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Command line help (part 2)</b> - In the above command <tt>!fe_run.py \"custom_config.conf\" -s \"2010-01-01\" -e \"2010-01-31\"</tt> we're running a Python script (<tt>fe_run.py</tt>) that runs FluxEngine. The script takes a number of options to tell it what and how to run. These are specifies as a list of parameters, the first (and only mandatory) of these is the path to the FluxEngine configuration file (in this case it is in the root of the tutorial's working directory, so we just specify the file name: <tt>custom_config.conf</tt>). Next are two optional parameters which specify the start and end dates. Optional parameters start with a <tt>-</tt> or <tt>--</tt> and often appear as name value pairs (e.g. the name <tt>-s</tt>, indicating start date, appears before the value <tt>2010-01-01</tt>). They're optional because they have default values and so if you don't explicitly specify them FluxEngine will use the default values. The default values can be found in the help (by running <tt>python fe_run.py -h</tt>).\n",
    "</div> "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualising output\n",
    "If all has gone well you'll see a log of the FluxEngine run and a message at the end saying 'completed successfully'. If you look at the output directory `tutorial_output` you should see a file called `OceanFluxGHG-month01-jan-2010-v0.nc`. If you have Panoply installed, you can open this file directly and look at its contents."
   ]
  },
  {
   "attachments": {
    "Panoply.png": {
     "image/png": "[encoded data, 182496 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Panoply.png](attachment:Panoply.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you double click a variable name (e.g. 'OF' - the air-sea gas flux) Panoply will give you the option to plot it. Spend some time exploring the FluxEngine outputs. Outputs that may be of interest are the air-sea gas flux (OF), the gas transfer velocity (OK3) and the interface and aqueous gas concentrations (OIC1 and OSFC, respectively). You'll also notice that a copy of the input datasets are provided in the output file. This makes it convenient to see what values were used to make the calculation.\n",
    "\n",
    "Plotting in Panoply is fine, but it is not very flexible. For example, if we want to compare the calculated CO<sub>2</sub> fluxes between the verification run and our custom run, we wouldn't be able to do this directly in Panoply. You can export data to a .csv file by right-clicking on a variable and choosing one of the export options. Then you could import the data into your preferred data visualisation or analysis software and perform any operations you like. However, since we're already running a Python interpretter in this Jupyter notebook, we might as well do this directly with Python! Let's compare the air-sea CO<sub>2</sub> fluxes between the verification and custom runs now, and don't worry if you don't follow the Python code - R and Matlab both provide their own libraries for importing data stored in netCDF files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from netCDF4 import Dataset #allows reading of netCDF files\n",
    "import matplotlib.pyplot as plt #for plotting data\n",
    "import numpy as np #matrix manipulation\n",
    "\n",
    "#read in the FluxEngine output files\n",
    "verificationNetCDF = Dataset(\"verification_output/verification_socatv4_sst_salinity_N00/2010/01/OceanFluxGHG-month01-jan-2010-v0.nc\", 'r') #Read the January data from the verification run\n",
    "verificationFlux = verificationNetCDF.variables[\"OF\"][:] #Extract the flux data from the 'OF' variable in the netCDF file\n",
    "customNetCDF = Dataset(\"tutorial_output/OceanFluxGHG-month01-jan-2010-v0.nc\", 'r') #Read the January data from the custom run\n",
    "customFlux = customNetCDF.variables[\"OF\"][:] #Extract the flux data from the 'OF' variable in the netCDF file\n",
    "\n",
    "#calculate difference between the new run and the verification\n",
    "fluxDifference = customFlux - verificationFlux #Calculate the change in calculated flux\n",
    "fluxDifference = np.squeeze(fluxDifference) #remove any dimensions of size 1 (in this case, the time dimension)\n",
    "\n",
    "#plot the differences in flux\n",
    "plt.figure(figsize=(8, 8))\n",
    "plt.imshow(fluxDifference)\n",
    "cbar = plt.colorbar(orientation=\"horizontal\")\n",
    "cbar.set_label(r\"Difference in flux (gC m$^{-2}$ s$^{-1}$)\", fontsize=16)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not the nicest looking figure, but good enough for an exploratory look at the results! You can see that the difference between the calculated fluxes is largest around regions where there are typically high wind speeds (open the `WS1_mean` variable in the FluxEngine output using Panoply if you want to visualise the wind speed data). The change in salinity data is unlikely to have a big effect, so it is likely that these differences are due to the updated gas transfer velocity parameterisation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Next tutorial\n",
    "Next, we'll look at how to use FluxEngine to calculate gas fluxes using in situ data. Return to the Jupyter dashboard and open the notebook stored in [Tutorials/02_using_insitu_data/02_using_insitu_data.ipynb](../02_using_insitu_data/02_using_insitu_data.ipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
