{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ourownstory/neural_prophet/blob/master/tutorials/feature-use/autoregression_yosemite_temps.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Autoregression\n",
    "Here we fit NeuralProphet to data with 5-minute resolution (daily temperatures at Yosemite). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "if 'google.colab' in str(get_ipython()):\n",
    "    !pip install git+https://github.com/ourownstory/neural_prophet.git # may take a while\n",
    "    #!pip install neuralprophet # much faster, but may not have the latest upgrades/bugfixes\n",
    "    \n",
    "import pandas as pd\n",
    "from neuralprophet import NeuralProphet, set_log_level\n",
    "set_log_level(\"ERROR\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-05-01 00:00:00</td>\n",
       "      <td>27.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-05-01 00:05:00</td>\n",
       "      <td>27.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-05-01 00:10:00</td>\n",
       "      <td>26.8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    ds     y\n",
       "0  2017-05-01 00:00:00  27.8\n",
       "1  2017-05-01 00:05:00  27.0\n",
       "2  2017-05-01 00:10:00  26.8"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_location = \"https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/\"\n",
    "df = pd.read_csv(data_location + \"yosemite_temps.csv\")\n",
    "df.head(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next-step forecast\n",
    "Based on our first contact with the data, we set:\n",
    "* First, we disable `weekly_seasonality`, as nature does not follow the human week's calendar.\n",
    "* Second, we increase `n_changepoints`, and increase `changepoints_range`, as we are doing short-term predictions.\n",
    "\n",
    "Further, we can make use of the fact that tomorrow's weather is most likely similar to yesterdays weather.\n",
    "This means that we will regress our time series on it's most recent past values, also known as autoregression.\n",
    "\n",
    "We can achieve this by setting `n_lags` to the desired number of past observations to regress over. This value is also known as the 'AR order'. \n",
    "\n",
    "Here, we predict the temperature in the next 5 minutes based on the last hour:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - (NP.forecaster._handle_missing_data) - 12 NaN values in column y were auto-imputed.\n",
      "INFO - (NP.utils.set_auto_seasonalities) - Disabling yearly seasonality. Run NeuralProphet with yearly_seasonality=True to override this.\n",
      "INFO - (NP.config.set_auto_batch_epoch) - Auto-set batch_size to 64\n",
      "INFO - (NP.config.set_auto_batch_epoch) - Auto-set epochs to 87\n",
      "Epoch[87/87]: 100%|████████████████████████████████████████| 87/87 [00:30<00:00,  2.86it/s, SmoothL1Loss=6.92e-5, MAE=0.304, MSE=0.413, RegLoss=0]\n"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    n_lags=12,\n",
    "    changepoints_range=0.95,\n",
    "    n_changepoints=30,\n",
    "    weekly_seasonality=False,\n",
    "    learning_rate=0.1,\n",
    ")\n",
    "metrics = m.fit(df, freq='5min', plot_live_loss=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please note that a model with an autoregressive component can be harder to fit. The automatic selection of hyperparameters may not lead to ideal results. For best results, consider changing these manually (in order of importance):\n",
    "* `learning_rate`\n",
    "* `epochs`\n",
    "* `batch_size`\n",
    "\n",
    "The automatically set hyperparameters (printed out as 'INFO' level logs) can serve as a good starting point."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - (NP.forecaster._handle_missing_data) - 12 NaN values in column y were auto-imputed.\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 198196 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "forecast = m.predict(df)\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The predictions are now very precise, but this is not a big surprise, as we are just forecasting the very next 5 minutes.\n",
    "\n",
    "When plotting the model parameters, the panel 'AR weight' displays the weight given to the 12 last observed values, these can be interpreted as our 'AR coefficients':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 81500 characters]\n",
      "text/plain": [
       "<Figure size 720x864 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# fig_comp = m.plot_components(forecast)\n",
    "m = m.highlight_nth_step_ahead_of_each_forecast(1) # temporary workaround to plot actual AR weights\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The 'AR weight' plot shows that the most recent observations are given significantly more weight compared to more distant observations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-step forecast\n",
    "To predict multiple steps into the future, we could 'unroll' our single-step model, by predicting a step ahead, adding the forecasted value to our data, and then forecasting the next step until we reach the horizon we are interested in. \n",
    "However, there is a better way to do this: We can directly forecast multiple steps ahead with NeuralProphet.\n",
    "\n",
    "We can set `n_forecasts` to the desired number of steps we would like to forecast (also known as 'forecast horizon').\n",
    "NeuralProphet will forecast `n_forecasts` steps into the future, at every single step. Thus, we have `n_forecasts` overlapping predictions of vaying age at every historic point. \n",
    "\n",
    "When increasing the forecast horizon `n_forecasts`, we should also increase the number of past observations `n_lags` to at least the same value.\n",
    "\n",
    "Here, we forecast the next 3 hours based on the last observed 6 hours, in 5-minute steps:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - (NP.forecaster._handle_missing_data) - 12 NaN values in column y were auto-imputed.\n",
      "INFO - (NP.utils.set_auto_seasonalities) - Disabling yearly seasonality. Run NeuralProphet with yearly_seasonality=True to override this.\n",
      "INFO - (NP.config.set_auto_batch_epoch) - Auto-set batch_size to 64\n",
      "INFO - (NP.config.set_auto_batch_epoch) - Auto-set epochs to 87\n",
      "Epoch[87/87]: 100%|█████████████████████████████████████████| 87/87 [00:40<00:00,  2.13it/s, SmoothL1Loss=0.000719, MAE=1.33, MSE=4.29, RegLoss=0]\n"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    n_lags=3*12,\n",
    "    n_forecasts=2*12,\n",
    "    changepoints_range=0.95,\n",
    "    n_changepoints=30,\n",
    "    weekly_seasonality=False,\n",
    "    learning_rate=0.1,\n",
    ")\n",
    "metrics = m.fit(df, freq='5min', plot_live_loss=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - (NP.forecaster._handle_missing_data) - 12 NaN values in column y were auto-imputed.\n",
      "WARNING - (NP.plotting.plot) - Legend is available only for the ten first handles\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 189876 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "forecast = m.predict(df)\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that our forecast sometimes miss-predicts for a day and then returns again to more accurate forecasts. Let's have a closer look at that over-prediction on day 6 of our data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - (NP.plotting.plot) - Legend is available only for the ten first handles\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 101728 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = m.plot(forecast[144:6*288])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On day 6, the redorded temperature drops at nighttime. Based on the observed lower nighttime temperatures, our model predicts a lower daytime peak. However, the actual daytime temperatures are anomalously low, barely higher than during the night. Thus, leading to an overprediction.\n",
    "\n",
    "We can again visualize the relative importance of the lags:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 76312 characters]\n",
      "text/plain": [
       "<Figure size 720x864 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# fig_comp = m.plot_components(forecast)\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the 'AR relevance' is the relative importance of a lag, averaged over all `n_forecasts`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reviewing a specific forecast step\n",
    "We can have a closer look at a particular forecast horizon by highlighting the n-th step ahead of each forecast. Here we focus on the temperature predicted 3 hours ahead (most distant prediction into the future). Let's have a closer look at the weights for the different 3 hour ahead forecast:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 77560 characters]\n",
      "text/plain": [
       "<Figure size 720x864 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "m = m.highlight_nth_step_ahead_of_each_forecast(2*12)\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we replot the first 6 days and compare it to the forecast one-step-ahead. We observe the single-step ahead forecast to be much more accurate compared to the 3 hour ahead forecast. However, neither is able to forecast the anomaly on day 6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 99712 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 95040 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig = m.plot(forecast[144:6*288])\n",
    "m = m.highlight_nth_step_ahead_of_each_forecast(1)\n",
    "fig = m.plot(forecast[144:6*288])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting the most recent forecast\n",
    "When we are more concerned with our actual forecast than the model fit, we can plot the last most recent forecast:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 22880 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "future = m.make_future_dataframe(df, n_historic_predictions=12)\n",
    "forecast = m.predict(future)\n",
    "m = m.highlight_nth_step_ahead_of_each_forecast(None) # reset highlight\n",
    "fig = m.plot_last_forecast(forecast)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last few historic forecasts can be included to assess how the forecast has changed over time.\n",
    "Here, we focus on the 3-hour ahead forecast, given over the last 2 hours."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 34940 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "m = m.highlight_nth_step_ahead_of_each_forecast(2*12)\n",
    "fig = m.plot_last_forecast(forecast, include_previous_forecasts=4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Larger forecast horizon\n",
    "For predictions further into the future, you could reduce the resulution of the data.\n",
    "Using a 5-minute resolution may be useful for a high-resolution short-term forecast, but counter-productive for a long-term forecast.\n",
    "As we only have a limited amount of data (approx 2 months), we want to avoid over-specifying the model.\n",
    "\n",
    "As an example: If we set the model to forecast 24 hours into the future (`nforecasts=24*12`) based on the last day's temperatures (`n_lags=24*12`), the number of parameters of our AR component grows to `24*12*24*12 = 82,944`. However, we only have about `2*30*24*12 = 17,280` samples in our dataset. The model would be overspecified.\n",
    "\n",
    "If we first downsample our data to hourly data, we reduce our dataset to `2*30*24=1440` and our model parameters to `24*24=576`. Thus, we are able to fit the model. However, it would be better to collect more data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1561"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.loc[:, \"ds\"] = pd.to_datetime(df.loc[:, \"ds\"])\n",
    "df_hourly = df.set_index('ds', drop=False).resample('H').mean().reset_index()\n",
    "len(df_hourly)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-05-01 00:00:00</td>\n",
       "      <td>24.891667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-05-01 01:00:00</td>\n",
       "      <td>16.741667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-05-01 02:00:00</td>\n",
       "      <td>11.733333</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   ds          y\n",
       "0 2017-05-01 00:00:00  24.891667\n",
       "1 2017-05-01 01:00:00  16.741667\n",
       "2 2017-05-01 02:00:00  11.733333"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_hourly.head(3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO - (NP.utils.set_auto_seasonalities) - Disabling yearly seasonality. Run NeuralProphet with yearly_seasonality=True to override this.\n",
      "INFO - (NP.config.set_auto_batch_epoch) - Auto-set batch_size to 32\n",
      "INFO - (NP.config.set_auto_batch_epoch) - Auto-set epochs to 162\n",
      "Epoch[162/162]: 100%|████████████████████████████████████████| 162/162 [00:10<00:00, 15.48it/s, SmoothL1Loss=0.002, MAE=2.15, MSE=11.6, RegLoss=0]\n"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(\n",
    "    n_lags=24,\n",
    "    n_forecasts=24,\n",
    "    changepoints_range=0.95,\n",
    "    n_changepoints=30,\n",
    "    weekly_seasonality=False,\n",
    "    learning_rate=0.3,\n",
    ")\n",
    "metrics = m.fit(df_hourly, freq='H', plot_live_loss=False)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - (NP.plotting.plot) - Legend is available only for the ten first handles\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 161504 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "forecast = m.predict(df_hourly)\n",
    "fig = m.plot(forecast)\n",
    "# fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we plot the most recent and last ten historic 24-hour forecasts, marking the 24-th hour ahead with an 'x'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING - (NP.plotting.plot) - Legend is available only for the ten first handles\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 36740 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "future = m.make_future_dataframe(df_hourly)\n",
    "forecast = m.predict(future)\n",
    "m = m.highlight_nth_step_ahead_of_each_forecast(24)\n",
    "fig = m.plot_last_forecast(forecast, include_previous_forecasts=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "np-dev",
   "language": "python",
   "name": "np-dev"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
