{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Renewable Energy: Forecasting hourly solar irradiance\n",
    "The training will occur on 90% of the data, reserving the last 10% for evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ywzhdfn2uqLf",
    "outputId": "95decf15-d410-45c9-b703-91fd68891e7f"
   },
   "outputs": [],
   "source": [
    "if 'google.colab' in str(get_ipython()):\n",
    "    !pip install git+https://github.com/ourownstory/neural_prophet.git # may take a while\n",
    "    #!pip install neuralprophet # much faster, but may not have the latest upgrades/bugfixes\n",
    "    \n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from neuralprophet import NeuralProphet, set_log_level\n",
    "set_log_level(\"ERROR\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# data_location = \"https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/\"\n",
    "data_location = '../../../neuralprophet-data/datasets/'\n",
    "\n",
    "sf_pv_df = pd.read_csv(data_location +  'energy/SF_PV.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0Krto6fIvHit"
   },
   "source": [
    "### Generic forecast: Time-based features only\n",
    "In this first section, we will train a model with time-features only like we would do with Facebook Prophet.\n",
    "\n",
    "From the start, we know that the sun intensity depends on the time of day and season in the year. \n",
    "Further, we know that the daily peak intensity is proportional to the season. As of now, neither Prophet nor NeuralProphet can multiply two seasonalities, thus, the peak will bee to low in summer and may turn negative in winter. Not ideal, but ok for a first attempt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 148,
     "referenced_widgets": [
      "4ac0917121f8498698e087259b787dcf",
      "94108fe9090f47c7ba2216479e0d3fac",
      "2d8235496ec642af8192f52d9f2692b1",
      "c94a8ae41b994c55a96ad44806b0f1c7",
      "24bf564f55644476911a6cf004a395e7",
      "87c170d1e00742a29e7f797e98c49cc2",
      "8a192ccc35e94e9f8be85898ed583e2c",
      "9467345334da47a8beadc770feef952a",
      "dc468cd35d2b4f0e8eb287689ac15412",
      "f35fc9cbd82c4187a4cdc08c3ac26998",
      "aab682cd3df24821a80331720f7c24e5"
     ]
    },
    "id": "s7faUgnrvGFN",
    "outputId": "50da2450-767f-4e3b-f03d-d03226d24ff8"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 45068 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                        \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    yearly_seasonality=3,\n",
    "    weekly_seasonality=False,\n",
    "    daily_seasonality=8,\n",
    "    growth=\"off\",\n",
    "    learning_rate=0.1,\n",
    ")\n",
    "df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)\n",
    "\n",
    "metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 111
    },
    "id": "OnEPYrkscVtf",
    "outputId": "fe0218f4-0fbe-4d74-86f3-0e74195681a7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SmoothL1Loss</th>\n",
       "      <th>MAE</th>\n",
       "      <th>MSE</th>\n",
       "      <th>RegLoss</th>\n",
       "      <th>SmoothL1Loss_val</th>\n",
       "      <th>MAE_val</th>\n",
       "      <th>MSE_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>0.009197</td>\n",
       "      <td>92.454981</td>\n",
       "      <td>14308.832125</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.013244</td>\n",
       "      <td>127.825119</td>\n",
       "      <td>20605.609375</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SmoothL1Loss        MAE           MSE  RegLoss  SmoothL1Loss_val  \\\n",
       "107      0.009197  92.454981  14308.832125      0.0          0.013244   \n",
       "\n",
       "        MAE_val       MSE_val  \n",
       "107  127.825119  20605.609375  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.tail(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 113864 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "forecast = m.predict(sf_pv_df)\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "G0E4dLGxcbsO",
    "outputId": "9590ed3b-6e76-4fdd-d4bf-c310c04ac7ec",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 60300 characters]\n",
      "text/plain": [
       "<Figure size 720x648 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_param = m.plot_parameters()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 440
    },
    "id": "5v-4bpNUvELW",
    "outputId": "089e3649-a238-4ec1-9593-32c2844d0ee4"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 131912 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "forecast = m.predict(df_test)\n",
    "m = m.highlight_nth_step_ahead_of_each_forecast(1)\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 440
    },
    "id": "pmx8KTT0cgyR",
    "outputId": "16062765-4e93-43c6-c434-857ad4b19008"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 38248 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_prediction = m.plot(forecast[-48:])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1-step ahead forecast with Auto-Regresseion\n",
    "In this second section, we will train a 1-step ahead forecaster on solar irradiance data (that can be a proxy for solar PV production). We can train this forecaster considering the privious 24 steps and disabling trend and seasonality.\n",
    "\n",
    "The paramters that we can modify are the number of lags and the AR sparsity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 148,
     "referenced_widgets": [
      "4ac0917121f8498698e087259b787dcf",
      "94108fe9090f47c7ba2216479e0d3fac",
      "2d8235496ec642af8192f52d9f2692b1",
      "c94a8ae41b994c55a96ad44806b0f1c7",
      "24bf564f55644476911a6cf004a395e7",
      "87c170d1e00742a29e7f797e98c49cc2",
      "8a192ccc35e94e9f8be85898ed583e2c",
      "9467345334da47a8beadc770feef952a",
      "dc468cd35d2b4f0e8eb287689ac15412",
      "f35fc9cbd82c4187a4cdc08c3ac26998",
      "aab682cd3df24821a80331720f7c24e5"
     ]
    },
    "id": "s7faUgnrvGFN",
    "outputId": "50da2450-767f-4e3b-f03d-d03226d24ff8"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 32720 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                        \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    growth='off',\n",
    "    yearly_seasonality=False,\n",
    "    weekly_seasonality=False,\n",
    "    daily_seasonality=False,\n",
    "    n_lags=3*24,\n",
    "    learning_rate=0.01,\n",
    ")\n",
    "\n",
    "df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)\n",
    "\n",
    "metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 111
    },
    "id": "OnEPYrkscVtf",
    "outputId": "fe0218f4-0fbe-4d74-86f3-0e74195681a7"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SmoothL1Loss</th>\n",
       "      <th>MAE</th>\n",
       "      <th>MSE</th>\n",
       "      <th>RegLoss</th>\n",
       "      <th>SmoothL1Loss_val</th>\n",
       "      <th>MAE_val</th>\n",
       "      <th>MSE_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>0.001957</td>\n",
       "      <td>30.255167</td>\n",
       "      <td>3045.023082</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000602</td>\n",
       "      <td>18.303713</td>\n",
       "      <td>935.846252</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SmoothL1Loss        MAE          MSE  RegLoss  SmoothL1Loss_val  \\\n",
       "107      0.001957  30.255167  3045.023082      0.0          0.000602   \n",
       "\n",
       "       MAE_val     MSE_val  \n",
       "107  18.303713  935.846252  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.tail(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 139800 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "forecast = m.predict(sf_pv_df)\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 440
    },
    "id": "5v-4bpNUvELW",
    "outputId": "089e3649-a238-4ec1-9593-32c2844d0ee4"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 125888 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "forecast = m.predict(df_test)\n",
    "m = m.highlight_nth_step_ahead_of_each_forecast(1)\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 871
    },
    "id": "0RrB1J5QcZ1u",
    "outputId": "acc2e734-f506-4ef7-adfb-ab18076b66e1",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 75792 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_comp = m.plot_components(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "G0E4dLGxcbsO",
    "outputId": "9590ed3b-6e76-4fdd-d4bf-c310c04ac7ec",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21356 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_param = m.plot_parameters()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 440
    },
    "id": "pmx8KTT0cgyR",
    "outputId": "16062765-4e93-43c6-c434-857ad4b19008"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 36336 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_prediction = m.plot(forecast[-48:])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Sparsifying the AR coefficients\n",
    "By setting an `ar_sparsity < 1` we can reduce the number of non-zero AR coefficients. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 148,
     "referenced_widgets": [
      "4ac0917121f8498698e087259b787dcf",
      "94108fe9090f47c7ba2216479e0d3fac",
      "2d8235496ec642af8192f52d9f2692b1",
      "c94a8ae41b994c55a96ad44806b0f1c7",
      "24bf564f55644476911a6cf004a395e7",
      "87c170d1e00742a29e7f797e98c49cc2",
      "8a192ccc35e94e9f8be85898ed583e2c",
      "9467345334da47a8beadc770feef952a",
      "dc468cd35d2b4f0e8eb287689ac15412",
      "f35fc9cbd82c4187a4cdc08c3ac26998",
      "aab682cd3df24821a80331720f7c24e5"
     ]
    },
    "id": "s7faUgnrvGFN",
    "outputId": "50da2450-767f-4e3b-f03d-d03226d24ff8"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 32072 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                        \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    growth='off',\n",
    "    yearly_seasonality=False,\n",
    "    weekly_seasonality=False,\n",
    "    daily_seasonality=False,\n",
    "    n_lags=3*24,\n",
    "    ar_sparsity=0.8,\n",
    "    learning_rate=0.01,\n",
    ")\n",
    "\n",
    "df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)\n",
    "\n",
    "metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SmoothL1Loss</th>\n",
       "      <th>MAE</th>\n",
       "      <th>MSE</th>\n",
       "      <th>RegLoss</th>\n",
       "      <th>SmoothL1Loss_val</th>\n",
       "      <th>MAE_val</th>\n",
       "      <th>MSE_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>0.002148</td>\n",
       "      <td>32.992762</td>\n",
       "      <td>3342.54368</td>\n",
       "      <td>0.001164</td>\n",
       "      <td>0.00068</td>\n",
       "      <td>19.354525</td>\n",
       "      <td>1058.18103</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SmoothL1Loss        MAE         MSE   RegLoss  SmoothL1Loss_val  \\\n",
       "107      0.002148  32.992762  3342.54368  0.001164           0.00068   \n",
       "\n",
       "       MAE_val     MSE_val  \n",
       "107  19.354525  1058.18103  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.tail(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "G0E4dLGxcbsO",
    "outputId": "9590ed3b-6e76-4fdd-d4bf-c310c04ac7ec",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 22004 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "m = m.highlight_nth_step_ahead_of_each_forecast(1)\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1-step ahead forecast with Auto-Regression including Integration\n",
    "Next, we will add the differences of the series as a lagged covariate.\n",
    "This basically extends the model from AR to ARI, where the I stands for 'integrated' time series."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>I</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>8757</th>\n",
       "      <td>2015-12-31 22:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8758</th>\n",
       "      <td>2015-12-31 23:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8759</th>\n",
       "      <td>2016-01-01 00:00:00</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       ds  y  I\n",
       "8757  2015-12-31 22:00:00  0  0\n",
       "8758  2015-12-31 23:00:00  0  0\n",
       "8759  2016-01-01 00:00:00  0  0"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = sf_pv_df.copy(deep=True)\n",
    "df[\"I\"] = np.append(0, sf_pv_df[\"y\"].values[1:] - sf_pv_df[\"y\"].values[:-1])\n",
    "df.tail(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 41224 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                        \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    growth='off',\n",
    "    yearly_seasonality=False,\n",
    "    weekly_seasonality=False,\n",
    "    daily_seasonality=False,\n",
    "    n_lags=3*24,\n",
    "    learning_rate=0.01,\n",
    ")\n",
    "m = m.add_lagged_regressor(\"I\", normalize=\"standardize\")\n",
    "df_train, df_test = m.split_df(df, freq='H', valid_p = 0.10)\n",
    "\n",
    "metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SmoothL1Loss</th>\n",
       "      <th>MAE</th>\n",
       "      <th>MSE</th>\n",
       "      <th>RegLoss</th>\n",
       "      <th>SmoothL1Loss_val</th>\n",
       "      <th>MAE_val</th>\n",
       "      <th>MSE_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>0.001941</td>\n",
       "      <td>29.899686</td>\n",
       "      <td>3019.55232</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000599</td>\n",
       "      <td>18.041283</td>\n",
       "      <td>931.748108</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SmoothL1Loss        MAE         MSE  RegLoss  SmoothL1Loss_val  \\\n",
       "107      0.001941  29.899686  3019.55232      0.0          0.000599   \n",
       "\n",
       "       MAE_val     MSE_val  \n",
       "107  18.041283  931.748108  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.tail(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "G0E4dLGxcbsO",
    "outputId": "9590ed3b-6e76-4fdd-d4bf-c310c04ac7ec",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 43176 characters]\n",
      "text/plain": [
       "<Figure size 720x648 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "m = m.highlight_nth_step_ahead_of_each_forecast(1)\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eG80MrTSeM0r"
   },
   "source": [
    "## 1 step ahead forecast with AR-Net: Using a Neural Network\n",
    "There is something to consider here, if we consider a neural network with at least one hidden layer. For a high enough learning rate (probably > 0.1), the gradient seems to vanish and forces the AR net output to 0.  \n",
    "\n",
    "From below, we can see that the forecast output has a strange behavior. it looks like the output is exactly the sum of the weekly, daily seasonlality with trend. With no noise and changes at all."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5KIVKB_teK-n",
    "outputId": "c8dc6c51-2257-4f1f-f394-b5b941fb34c0"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46416 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                        \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    growth='off',\n",
    "    yearly_seasonality=False,\n",
    "    weekly_seasonality=False,\n",
    "    daily_seasonality=False,\n",
    "    n_lags=3*24,\n",
    "    num_hidden_layers=4,\n",
    "    d_hidden=16,\n",
    "    learning_rate=1.0,\n",
    ")\n",
    "\n",
    "df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)\n",
    "\n",
    "metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 20500 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_comp = m.plot_components(forecast)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ukfe9jUlfk-_"
   },
   "source": [
    "### Learning Rate matters when training a Neural Network.\n",
    "An easy fix is to set the learning rate at a low enough value, likely around 0.01 to 0.001."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-aqM32pCfjEG",
    "outputId": "86bc014e-1e90-48e0-ac25-c9597a0fbcfe"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 30992 characters]\n",
      "text/plain": [
       "<Figure size 864x576 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                        \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    growth='off',\n",
    "    yearly_seasonality=False,\n",
    "    weekly_seasonality=False,\n",
    "    daily_seasonality=False,\n",
    "    n_lags=3*24,\n",
    "    num_hidden_layers=4,\n",
    "    d_hidden=16,\n",
    "    learning_rate=0.003,\n",
    ")\n",
    "\n",
    "df_train, df_test = m.split_df(sf_pv_df, freq='H', valid_p = 0.10)\n",
    "\n",
    "metrics = m.fit(df_train, freq='H', validation_df=df_test, plot_live_loss=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SmoothL1Loss</th>\n",
       "      <th>MAE</th>\n",
       "      <th>MSE</th>\n",
       "      <th>RegLoss</th>\n",
       "      <th>SmoothL1Loss_val</th>\n",
       "      <th>MAE_val</th>\n",
       "      <th>MSE_val</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>107</th>\n",
       "      <td>0.001331</td>\n",
       "      <td>20.654574</td>\n",
       "      <td>2070.965949</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.00055</td>\n",
       "      <td>13.492601</td>\n",
       "      <td>855.404907</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     SmoothL1Loss        MAE          MSE  RegLoss  SmoothL1Loss_val  \\\n",
       "107      0.001331  20.654574  2070.965949      0.0           0.00055   \n",
       "\n",
       "       MAE_val     MSE_val  \n",
       "107  13.492601  855.404907  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.tail(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21340 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 440
    },
    "id": "vajganJ4f54_",
    "outputId": "e240a1bc-b2e6-41af-a89a-0048d16245a0"
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 117116 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "m = m.highlight_nth_step_ahead_of_each_forecast(1)\n",
    "forecast = m.predict(df_test)\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 69348 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_comp = m.plot_components(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 33068 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig_prediction = m.plot(forecast[-48:])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xp1y3gaBhCdo"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "energy_data_example.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "np-dev",
   "language": "python",
   "name": "np-dev"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "24bf564f55644476911a6cf004a395e7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_aab682cd3df24821a80331720f7c24e5",
      "placeholder": "​",
      "style": "IPY_MODEL_f35fc9cbd82c4187a4cdc08c3ac26998",
      "value": " 264/297 [00:04&lt;00:00, 59.93it/s]"
     }
    },
    "2d8235496ec642af8192f52d9f2692b1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8a192ccc35e94e9f8be85898ed583e2c",
      "placeholder": "​",
      "style": "IPY_MODEL_87c170d1e00742a29e7f797e98c49cc2",
      "value": " 89%"
     }
    },
    "4ac0917121f8498698e087259b787dcf": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2d8235496ec642af8192f52d9f2692b1",
       "IPY_MODEL_c94a8ae41b994c55a96ad44806b0f1c7",
       "IPY_MODEL_24bf564f55644476911a6cf004a395e7"
      ],
      "layout": "IPY_MODEL_94108fe9090f47c7ba2216479e0d3fac"
     }
    },
    "87c170d1e00742a29e7f797e98c49cc2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8a192ccc35e94e9f8be85898ed583e2c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "94108fe9090f47c7ba2216479e0d3fac": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9467345334da47a8beadc770feef952a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "aab682cd3df24821a80331720f7c24e5": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c94a8ae41b994c55a96ad44806b0f1c7": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_dc468cd35d2b4f0e8eb287689ac15412",
      "max": 297,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9467345334da47a8beadc770feef952a",
      "value": 264
     }
    },
    "dc468cd35d2b4f0e8eb287689ac15412": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f35fc9cbd82c4187a4cdc08c3ac26998": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
