{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using the Histogram Toolkit with DL-MONTE\n",
    "\n",
    "This note assumes you are familiar with the histogram toolkit, and DL-MONTE. See the accompanying notebooks:\n",
    "\n",
    "1. `intro-ising.ipynb`\n",
    "2. `dlmonte-util.ipynb`\n",
    "\n",
    "The histogram toolkit provides a implementation of the Observable\n",
    "data class for use with DL MONTE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Import standard os module for file manipulations\n",
    "import os\n",
    "import sys\n",
    "\n",
    "# You will need to set DL_MONTE_HOME appropriately for\n",
    "# the local system\n",
    "MONTEPYTHON_HOME = \"/home/jap93/DLMONTE3/dlmontepython_course/\"\n",
    "\n",
    "# Set PYTHONPATH to the 'htk' directory within DL_MONTE_HOME\n",
    "sys.path.append(MONTEPYTHON_HOME)\n",
    "\n",
    "%matplotlib inline\n",
    "import numpy\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import dlmontepython.htk.sources.dlmonte as dlmonte\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: muVT Ensemble\n",
    "\n",
    "Following Wilding (PRE 52 692, 1995), we consider simulations performed in the $\\mu VT$ (Grand Canonical) ensemble using DL-MONTE. The system under consideration consists of Lennard-Jones particles subject to insertion/deletion moves (only). The Hamiltonian may be written\n",
    "\n",
    "$$ {\\cal H} = -U + \\mu N $$\n",
    "\n",
    "where $U$ is the internal interaction energy, $\\mu$ is the chemical potential, and $N$ is the number of particles.\n",
    "\n",
    "The Lennard-Jones potential is standard\n",
    "\n",
    "$$ \\phi(r) = 4w[(\\sigma/r)^{12} - (\\sigma/r)^6 ] $$\n",
    "\n",
    "where $w$ is an energy (the well depth) and sigma is a length scale. In this work, $w$ is taken to be 1 eV, and $\\sigma = 1$ Angstrom. The potential is cut off at a radius $r_c$ (here 2.5$\\sigma$).\n",
    "\n",
    "A cubic system of side 4$r_c$ is used in this case ($m = 4$ for Wilding).\n",
    "\n",
    "The chemical potential is specified by means of the activity $z = \\exp(\\mu/k_bT)/\\Lambda^3$ where $\\Lambda$ is a thermal wavelength (see e.g., Allan and Tildesley). Here, the activity is 0.06177 A$^{-3}$.\n",
    "\n",
    "Mearurements of the total interaction energy\n",
    "\n",
    "$$ U = \\sum_{i,j} \\phi (r_{ij}) $$\n",
    "\n",
    "are collected along with the instantaneous particle number $N$. A non-dimensionalised density may be defined:\n",
    "\n",
    "$$\\rho^\\star = N\\sigma^3/V$$\n",
    "\n",
    "along with a non-dimensionalised energy density\n",
    "\n",
    "$$ u^\\star = (4w)^{-1} (\\sigma^3/V) U. $$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "<br>\n",
    "<br>\n",
    "## Running an example simulation\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading DL-MONTE data\n",
    "\n",
    "The DL-MONTE simulation is controlled by three separate input files which must be co-ordinated to produce the correct results.\n",
    "\n",
    "For a given DL-MONTE run, this is most easily done using the `DLMonteData` object supplied by the toolkit. This will read the relevant input (`CONTROL`, `CONFIG`, and `FIELD` files as outlined above) together with time series of instantaneous measurements from `PTFILE.000`.\n",
    "\n",
    "All these files are assumed to be available in a single directory or folder. We must also specify the ensemble appropropriate for the simulation (this is not determined automatically at the moment).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import dlmontepython.htk.sources.dlunits\n",
    "import dlmontepython.htk.histogram\n",
    "import dlmontepython.htk.util\n",
    "from dlmontepython.htk.util import Label\n",
    "\n",
    "data = dlmonte.DLMonteData(\"/home/jap93/util-dlmonte_workspace\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Source:      /home/jap93/util-dlmonte_workspace\n",
      "Type:        DL-MONTE YAMLDATA Format\n",
      "Ensemble:    muVT (Grand Canonical)\n",
      "\n",
      "Parameters: \n",
      "Label        Description          Units               Value\n",
      "volume       Volume               Angstrom^3         1000.0\n",
      "systemp      Temperature          K                13754.88\n",
      "zlj          Activity             Volume^-1         0.06177\n",
      "\n",
      "Observables:\n",
      "Label        Description          Units            No. Obs.\n",
      "t            DLM steps            time steps           1000\n",
      "nmollj       No. molecules lj     None                 1000\n",
      "energy       Total energy         unknown units         1000\n"
     ]
    }
   ],
   "source": [
    "table = data.to_table()\n",
    "print(table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The number of atoms (aka \"molecules\" here) varies as a function of time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.summary_time_series(plt, \"nmollj\", 271)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A histogram of the number of particles suggests we are getting some sampling of a low density phase and a high density phase, which is what we expect. (If we were exactly at coexistance, this histogram would be symmetric.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.summary_histogram(plt, \"nmollj\", 271)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One can check the correlations to make a judgement about how long the simulation needs to be run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.summary_autocorrelation(plt, \"energy\", 900000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data.summary_autocorrelation(plt, \"nmollj\", 900000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We  need to get hold of the parameters from the various input files. The volume of the cell is specified in `CONFIG` as a set of unit vectors. In this case the cell size is a 10 Angstrom cube.\n",
    "\n",
    "The temperature is specifed in the `CONTROL` file with the key `temperature`. The units are Kelvin. The value is 13754.88 K: the tempurature such that $k_BT = 1$eV is 11594.2 K, so the simulation temperature corresponds to about 1.1853 eV (cf epsilon in Lennard Jones of 1 eV). Wilding (1995) has a critical temperature corresponding to $kT \\approx 1.1876$ eV.\n",
    "\n",
    "Energy output (seen above to be in the range of -2000-0) is in units ?\n",
    "\n",
    "Chemical potential information appears in the `FIELD` file via the parameter \"lj\" (otherwise referred to as the activity). The value of the activity here is 0.06177 (per A$^3$). The activity is related to the chemical potential via\n",
    "$z = e^{ \\beta \\mu} / \\Lambda^3$. See Allen and Tildesley, where $\\Lambda$ is a thermal length scale.\n",
    "\n",
    "Here we equate $\\Lambda^3$ with the total volume of the system.\n",
    "Given the volume of the system is 10$^3$ A, $ln(zV) \\approx 4.12$ and $\\mu \\approx 4.8$ eV.\n",
    "\n",
    "The activity controls the ease with which molecules are created and destroyed in the system: a larger activity favours easier creation of particles, but not destruction of particles. I.e., larger activity favours the higher density phase. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "activity = data.parameter(\"zlj\")\n",
    "volume = data.parameter(\"volume\")\n",
    "temperature = data.parameter(\"systemp\")\n",
    "\n",
    "print (\"Activity    (A^-3)     \", activity)\n",
    "print (\"Volume      (A^3)      \", volume)\n",
    "print (\"Temperature (K)        \", temperature)\n",
    "\n",
    "k_b = dlmontepython.htk.sources.dlunits.k_boltzmann(\"ev\")\n",
    "mu = k_b*temperature*numpy.log(volume*activity)\n",
    "beta = 1.0/(k_b*temperature)\n",
    "\n",
    "print (\"beta            (eV^-1)\", beta)\n",
    "print (\"Chemical potential (eV)\", mu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Using the Histogram class\n",
    "\n",
    "The aim is to find the chemical potential for which the histrogram\n",
    "$x = a_M^{-1}(M - <M>)$ is symmetric (and is bimodal following the universal case).\n",
    "\n",
    "This will be done by reweighting the observations of the number of particles. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = data.observable(\"nmollj\").data\n",
    "print (\"Minimum and maximum number of particles: \", n.min(), n.max())\n",
    "np = n.copy()\n",
    "mbar = np.mean()\n",
    "print (\"Average number of particles, and variance: \", mbar, np.var())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get a histogram\n",
    "nbins = 161\n",
    "hn = dlmontepython.htk.histogram.Histogram(n, nbins)\n",
    "m0, m1, m2, m3, minf = hn.statistic()\n",
    "print (\"Moments: \", m0, m1, m2, m3, minf)\n",
    "hn.plot(plt)\n",
    "plt.xlabel(r\"Density $\\rho$\", fontsize = 16)\n",
    "plt.ylabel(r\"$P(\\rho)$\", fontsize = 16)\n",
    "plt.text(120, 0.0054, \"(a) Simulation not quite at coexistance\",\n",
    "         fontsize = 12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reweighting \n",
    "\n",
    "Can we reweight the Histogram? Specifically, we want to reweight the density wrt the chemical potential. So, to reweight obsevations of quantity $O$ wrt the number of particles to $\\mu'$ we would require\n",
    "\n",
    "$$ \\langle O \\rangle_{\\mu'} = \\frac{\\sum_i O_i exp[-\\beta(\\mu - \\mu')N_i]}{\\sum_i exp[-\\beta(\\mu -\\mu')N_i]}$$\n",
    "\n",
    "To reweight a histogram\n",
    "\n",
    "$$ h_{\\mu'}(\\rho^\\star_j) = (1/N_{obs}) \\sum_i \\delta(\\rho^\\star_i, \\rho^\\star_j) $$\n",
    "\n",
    "we would require\n",
    "\n",
    "$$ h_{\\mu'}(\\rho^\\star_j) = \\frac{ \\sum_i \\delta(\\rho^\\star_i, \\rho^\\star_j) exp[-\\beta(\\mu - \\mu')N_i]}{\\sum_i exp[-\\beta(\\mu - \\mu')N_i]} $$\n",
    "\n",
    "This can be done via the weights in the denominator.\n",
    "\n",
    "\n",
    "First, work out the expectation value of $\\rho^\\star$ at a new chemical potential:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dlmontepython.htk.parameter import Parameter\n",
    "\n",
    "# reduce the chemical potential (favours low density phase)\n",
    "mu_new = 4.886\n",
    "\n",
    "# Create a reweighting object for exp(-beta.mu.Ni)\n",
    "# This takes two Parameter objects, and the relevant Observable\n",
    "# as arguemnts:\n",
    "Mu = Parameter(mu, Label(\"mu\", \"Chemical potential\", \"eV\"))\n",
    "Beta = Parameter(-beta, Label(\"beta\", \"1/kT\", \"1/eV\"))\n",
    "\n",
    "r = dlmontepython.htk.histogram.Reweighter(\"mu\", Mu, Beta, data.observable(\"nmollj\"))\n",
    "\n",
    "print (\"Reweighted mean particle number: \", r.reweight_obs(n, mu_new))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Now try the histogram\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Register the reweighter with the histogtram only once...\n",
    "hn.add_reweighter(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot a reweighted histogram at mu_new\n",
    "mu_new = 4.884\n",
    "hn.weight_wrt(\"mu\", mu_new)\n",
    "hn.plot(plt)\n",
    "plt.ylim(ymax=0.01)\n",
    "plt.xlabel(r\"Density $\\rho$\", fontsize = 16)\n",
    "plt.ylabel(r\"$P(\\rho)$\", fontsize = 16)\n",
    "plt.text(120, 0.0054, \"(b) Reweight to new chemical potential\",\n",
    "         fontsize = 12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Optimisation problem"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def objective(munew, histogram):\n",
    "\n",
    "    \"\"\"This objective integrates up to the mean of the histogram\n",
    "    and measures how close to 1/2 is the result\"\"\"\n",
    "    \n",
    "    histogram.weight_wrt(\"mu\", munew)\n",
    "    m0, m1, m2, m3, minf = histogram.statistic()    \n",
    "    px, xbin = numpy.histogram(hn.obs, bins=hn.nbins, density=True,\n",
    "                               weights=hn.wgt)\n",
    "    dx = xbin[1] - xbin[0]\n",
    "    n = int((m1 - xbin[0])/dx)\n",
    "    fr = (m1 - xbin[0])/dx - n\n",
    "    obj = numpy.sum(dx*px[:n]) + fr*dx*px[n+1]\n",
    "    return numpy.abs(obj - 0.5)\n",
    "        \n",
    "# Note Don't be tempted to use the skewness of the distribution\n",
    "# as the basis for an objective here; single phase has low skewness\n",
    "# as well as balanced bimodal..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import scipy.optimize\n",
    "\n",
    "# brac = (4.88, 4.89) is used to constrain the range of acceptable\n",
    "# values of mu; it is easy to fall over into a single phase (a Gaussian)\n",
    "mu_opt = scipy.optimize.brent(objective, (hn,), brack = (4.88, 4.89), \\\n",
    "                              maxiter = 20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print (\"Best estimate mu_opt\", mu_opt)\n",
    "\n",
    "hn.weight_wrt(\"mu\", mu_opt)\n",
    "\n",
    "print (\"Reweighted moments \", hn.statistic())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "We can reweight the original histogram to the predicted co-existance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hn.weight_wrt(\"mu\", mu_opt)\n",
    "\n",
    "hn.plot(plt)\n",
    "plt.ylim(ymax=0.006)\n",
    "plt.xlabel(r\"Density $\\rho$\", fontsize = 16)\n",
    "plt.ylabel(r\"$P(\\rho)$\", fontsize = 16)\n",
    "plt.text(120, 0.0054, \"(c) Reweight to best estimate of coexistance\",\n",
    "         fontsize = 12)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3.7.4 64-bit",
   "language": "python",
   "name": "python37464bit5f73c22a59314460943a8a7e2d2be13a"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
