{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ourownstory/neural_prophet/blob/master/tutorials/feature-use/events_holidays_peyton_manning.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modeling Holidays and Special Events\n",
    "\n",
    "If you have holidays or other recurring events that you’d like to model, you must create a dataframe for them. It has two columns (`event` and `ds`) and a row for each occurrence of the holiday. It must include all occurrences of the holiday, both in the past (back as far as the historical data go) and in the future (out as far as the forecast is being made). If they won’t repeat in the future, Prophet will model them and then not include them in the forecast.\n",
    "\n",
    "Events can be added as either additive or multiplicative components.\n",
    "\n",
    "Windows can additionally be defined to include days before/after the event.\n",
    "\n",
    "We will use the time series of the log daily page views for the Wikipedia page for Peyton Manning as an example.\n",
    "First, we load the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "if 'google.colab' in str(get_ipython()):\n",
    "    !pip install git+https://github.com/ourownstory/neural_prophet.git # may take a while\n",
    "    #!pip install neuralprophet # much faster, but may not have the latest upgrades/bugfixes\n",
    "\n",
    "import pandas as pd\n",
    "from neuralprophet import NeuralProphet, set_log_level\n",
    "set_log_level(\"ERROR\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "data_location = \"https://raw.githubusercontent.com/ourownstory/neuralprophet-data/main/datasets/\"\n",
    "df = pd.read_csv(data_location + \"wp_log_peyton_manning.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a dataframe that includes the dates of all of Peyton Manning’s playoff appearances, including past events and future events:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "## user specified events\n",
    "# history events\n",
    "playoffs = pd.DataFrame({\n",
    "    'event': 'playoff',\n",
    "    'ds': pd.to_datetime([\n",
    "        '2008-01-13', '2009-01-03', '2010-01-16',\n",
    "        '2010-01-24', '2010-02-07', '2011-01-08',\n",
    "        '2013-01-12', '2014-01-12', '2014-01-19',\n",
    "        '2014-02-02', '2015-01-11', '2016-01-17',\n",
    "        '2016-01-24', '2016-02-07',\n",
    "    ]),\n",
    "})\n",
    "\n",
    "superbowls = pd.DataFrame({\n",
    "    'event': 'superbowl',\n",
    "    'ds': pd.to_datetime([\n",
    "        '2010-02-07', '2012-02-05', '2014-02-02', \n",
    "        '2016-02-07',\n",
    "    ]),\n",
    "})\n",
    "\n",
    "events_df = pd.concat((playoffs, superbowls))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  Additive Events\n",
    "\n",
    "Once the table is created, event effects are included in the model by adding them in with the `add_events` function.\n",
    "\n",
    "By default, events are modelled as additive events. Here, we model both playoffs as additive events.\n",
    "\n",
    "Additionally we change the `loss_func` to 'MSE', as the events we are fitting weights fore are a few rare outliers. \n",
    "Usually we would not want to skew our model with a few outliers, thus the default loss function is 'Huber'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# NeuralProphet Object\n",
    "m = NeuralProphet(loss_func=\"MSE\")\n",
    "\n",
    "# set the model to expect these events\n",
    "m = m.add_events([\"playoff\", \"superbowl\"])\n",
    "\n",
    "# create the data df with events\n",
    "history_df = m.create_df_with_events(df, events_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6c04f453b35b4c5d977f1c4fe9fa5606",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/273 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                  \r"
     ]
    }
   ],
   "source": [
    "# fit the model\n",
    "metrics = m.fit(history_df, freq=\"D\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "# forecast with events known ahead\n",
    "future = m.make_future_dataframe(df=history_df, events_df=events_df, periods=365, n_historic_predictions=len(df))\n",
    "forecast = m.predict(df=future)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The events effect can be seen in the forecast dataframe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ds</th>\n",
       "      <th>y</th>\n",
       "      <th>yhat1</th>\n",
       "      <th>residual1</th>\n",
       "      <th>trend</th>\n",
       "      <th>season_yearly</th>\n",
       "      <th>season_weekly</th>\n",
       "      <th>events_additive</th>\n",
       "      <th>event_playoff</th>\n",
       "      <th>event_superbowl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2190</th>\n",
       "      <td>2014-02-02</td>\n",
       "      <td>11.760520</td>\n",
       "      <td>11.739920</td>\n",
       "      <td>-0.020600</td>\n",
       "      <td>8.361824</td>\n",
       "      <td>0.776849</td>\n",
       "      <td>0.03111</td>\n",
       "      <td>2.570135</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>1.523094</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2532</th>\n",
       "      <td>2015-01-11</td>\n",
       "      <td>9.886596</td>\n",
       "      <td>9.889818</td>\n",
       "      <td>0.003223</td>\n",
       "      <td>7.923612</td>\n",
       "      <td>0.888056</td>\n",
       "      <td>0.03111</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2901</th>\n",
       "      <td>2016-01-17</td>\n",
       "      <td>9.273878</td>\n",
       "      <td>9.659006</td>\n",
       "      <td>0.385128</td>\n",
       "      <td>7.593356</td>\n",
       "      <td>0.987498</td>\n",
       "      <td>0.03111</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2908</th>\n",
       "      <td>2016-01-24</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9.660078</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.587125</td>\n",
       "      <td>0.994801</td>\n",
       "      <td>0.03111</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2922</th>\n",
       "      <td>2016-02-07</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.775986</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.574663</td>\n",
       "      <td>0.600078</td>\n",
       "      <td>0.03111</td>\n",
       "      <td>2.570135</td>\n",
       "      <td>1.047041</td>\n",
       "      <td>1.523094</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             ds          y      yhat1  residual1     trend  season_yearly  \\\n",
       "2190 2014-02-02  11.760520  11.739920  -0.020600  8.361824       0.776849   \n",
       "2532 2015-01-11   9.886596   9.889818   0.003223  7.923612       0.888056   \n",
       "2901 2016-01-17   9.273878   9.659006   0.385128  7.593356       0.987498   \n",
       "2908 2016-01-24        NaN   9.660078        NaN  7.587125       0.994801   \n",
       "2922 2016-02-07        NaN  10.775986        NaN  7.574663       0.600078   \n",
       "\n",
       "      season_weekly  events_additive  event_playoff  event_superbowl  \n",
       "2190        0.03111         2.570135       1.047041         1.523094  \n",
       "2532        0.03111         1.047041       1.047041         0.000000  \n",
       "2901        0.03111         1.047041       1.047041         0.000000  \n",
       "2908        0.03111         1.047041       1.047041         0.000000  \n",
       "2922        0.03111         2.570135       1.047041         1.523094  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "events = forecast[(forecast['event_playoff'].abs() + forecast['event_superbowl'].abs()) > 0]\n",
    "events.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The events effects will also show up in the plots, where we see that there is a spike on the days around playoff appearances, with an especially large spike for the superbowl:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 93260 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting forecasts\n",
    "fig = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 93300 characters]\n",
      "text/plain": [
       "<Figure size 720x864 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting components\n",
    "fig_comp = m.plot_components(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 104780 characters]\n",
      "text/plain": [
       "<Figure size 720x1080 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting parameters\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Event Windows\n",
    "You can also include the arguments `lower_window` and `upper_window` which extend the holiday out to `[lower_window, upper_window]` days around the date. \n",
    "\n",
    "For instance, if you wanted to include Christmas Eve in addition to Christmas you’d include `lower_window=-1, upper_window=0`. If you wanted to use Black Friday in addition to Thanksgiving, you’d include `lower_window=0, upper_window=1`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8512e1bb8c504870935d314d172d4ead",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/273 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                  \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(loss_func=\"MSE\")\n",
    "\n",
    "# set event configs to NeuralProphet object with windows\n",
    "m = m.add_events([\"playoff\"], upper_window=1)\n",
    "m = m.add_events([\"superbowl\"], lower_window=-1, upper_window=1)\n",
    "\n",
    "# create the data df with events\n",
    "history_df = m.create_df_with_events(df, events_df)\n",
    "\n",
    "# fit the model\n",
    "metrics = m.fit(history_df, freq=\"D\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "forecast = m.predict(history_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 96220 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 104760 characters]\n",
      "text/plain": [
       "<Figure size 720x1080 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting parameters\n",
    "fig = m.plot(forecast)\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multiplicative Events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "425c031f62ab4bbb84e142a99a34ad96",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/273 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                  \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet(loss_func=\"MSE\")\n",
    "\n",
    "# set event configs to NeuralProphet object with windows\n",
    "m = m.add_events([\"playoff\"], upper_window=1)\n",
    "m = m.add_events([\"superbowl\"], lower_window=-1, upper_window=1, mode=\"multiplicative\")\n",
    "\n",
    "# create the data df with events\n",
    "history_df = m.create_df_with_events(df, events_df)\n",
    "\n",
    "# fit the model\n",
    "metrics = m.fit(history_df, freq=\"D\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "forecast = m.predict(history_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 95768 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting components\n",
    "fig_comp = m.plot(forecast)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 121132 characters]\n",
      "text/plain": [
       "<Figure size 720x1296 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot parameters\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Country Specific Holidays"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "95ba15dca9034290a430f9b529cf5261",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/273 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "                                                                                                                                                  \r"
     ]
    }
   ],
   "source": [
    "m = NeuralProphet()\n",
    "\n",
    "# add the country specific holidays\n",
    "m = m.add_country_holidays(\"US\")\n",
    "\n",
    "# fit the model\n",
    "metrics = m.fit(df, freq=\"D\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "forecast = m.predict(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 94148 characters]\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 147588 characters]\n",
      "text/plain": [
       "<Figure size 720x1080 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting components\n",
    "fig = m.plot(forecast)\n",
    "fig_param = m.plot_parameters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "np-dev",
   "language": "python",
   "name": "np-dev"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
