{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FVGP Single Task Notebook\n",
    "In this notebook we will go through a few features of fvGP. We will be primarily concerned with regression over a single dimension output and single task. See the multiple_task_test_notebook.ipynb for single dimension and multiple task example. The extension to multiple dimensions is straight forward."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import fvgp and relevant libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fvgp\n",
    "from fvgp import gp\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining some input data and testing points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def test_data_function(x):\n",
    "    return np.sin(0.1 * x)+np.cos(0.05 * x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_input = np.linspace(-2*np.pi, 10*np.pi,20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_output = test_data_function(x_input)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_input_test = np.linspace(-2*np.pi, 10*np.pi, 100)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting up the fvgp single task object\n",
    "NOTE: The input data need to be given in the form (N x input_space_dim). The output can either be a N array or N x 1 array where N is the number of data points. See help(gp.GP) for more information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj = gp.GP(input_space_dim = 1, points = x_input.reshape(-1,1),\n",
    "                            values = y_output.reshape(-1,1),\n",
    "                           init_hyperparameters = np.array([10,10]),use_inv = False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training our gaussian process regression on given data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "hyper_param_bounds = np.array([[0.0001, 1000000],[ 0.000001, 100]])\n",
    "##this will block the main thread, even if you use \"hgdl\", another option is \"global\" or \"local\"\n",
    "print(\"Blocking main thread...\")\n",
    "obj.train(hyper_param_bounds, method = \"hgdl\")\n",
    "print(\"free again\")\n",
    "print(\"this also killed the client\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking the posterior mean at the test points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "post_mean= obj.posterior_mean(x_input_test.reshape(-1,1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "plt.plot(x_input_test, post_mean['f(x)'], label='gp interpolation')\n",
    "plt.scatter(x_input, y_output, label='data')\n",
    "plt.plot(x_input_test,test_data_function(x_input_test), label = '')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training Asynchronously "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj = gp.GP(input_space_dim = 1, points = x_input.reshape(-1,1),\n",
    "                            values = y_output.reshape(-1,1),\n",
    "                           init_hyperparameters = np.array([10,10]),\n",
    "                            variances = np.zeros(y_output.reshape(-1,1).shape))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "hyper_param_bounds = np.array([[0.0001, 100], [ 0.0001, 100]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "async_obj = obj.train_async(hyper_param_bounds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Updating asynchronously\n",
    "Updates hyperparameters to current optimization values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "obj.update_hyperparameters(async_obj)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Killing training \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "obj.kill_training(async_obj)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking the posterior mean at the test points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "post_mean= obj.posterior_mean(x_input_test.reshape(-1,1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "plt.plot(x_input_test, post_mean['f(x)'], label='interpolation')\n",
    "plt.plot(x_input_test, test_data_function(x_input_test), label='ground truth')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Custom Kernels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def kernel_l1(x1,x2, hp, obj):\n",
    "    ################################################################\n",
    "    ###standard anisotropic kernel in an input space with l1########\n",
    "    ################################################################\n",
    "    d1 = abs(np.subtract.outer(x1[:,0],x2[:,0])) \n",
    "    return hp[0] * np.exp(-d1/hp[1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj = gp.GP(input_space_dim = 1, points = x_input.reshape(-1,1),\n",
    "                            values = y_output.reshape(-1,1),\n",
    "                           init_hyperparameters = np.array([10,10]),\n",
    "                            variances = np.zeros(y_output.reshape(-1,1).shape),\n",
    "                            gp_kernel_function = kernel_l1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training our gaussian process regression on given data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "hyper_param_bounds = np.array([[0.0001, 1000],[ 0.0001, 1000]])\n",
    "obj.train(hyper_param_bounds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking the posterior mean at the test points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "post_mean= obj.posterior_mean(x_input_test.reshape(-1,1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "plt.plot(x_input_test, post_mean['f(x)'], label='interpolation')\n",
    "plt.plot(x_input_test, test_data_function(x_input_test), label='ground truth')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prior Mean Functions\n",
    "### NOTE: The prior mean function must return a 1d vector, e.g., (100,)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def example_mean(gp_obj,x,hyperparameters):\n",
    "    return np.ones(len(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obj = gp.GP(input_space_dim = 1, points = x_input.reshape(-1,1),\n",
    "                            values = y_output.reshape(-1,1),\n",
    "                           init_hyperparameters = np.array([10,10]),\n",
    "                            variances = np.zeros(y_output.reshape(-1,1).shape),\n",
    "                            gp_mean_function = example_mean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training our gaussian process regression on given data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "hyper_param_bounds = np.array([[0.0001, 1000],[ 0.0001, 1000]])\n",
    "obj.train(hyper_param_bounds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Looking the posterior mean at the test points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "post_mean= obj.posterior_mean(x_input_test.reshape(-1,1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "plt.plot(x_input_test, post_mean['f(x)'], label='interpolation')\n",
    "plt.plot(x_input_test, test_data_function(x_input_test), label='ground truth')\n",
    "plt.scatter(x_input_test,post_mean['f(x)'])\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "fvgp",
   "language": "python",
   "name": "fvgp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
