Metadata-Version: 2.1
Name: GPro
Version: 1.0.5
Summary: Preference Learning with Gaussian Processes.
Home-page: https://github.com/chariff/GPro
Author: Chariff Alkhassim
Author-email: chariff.alkhassim@gmail.com
License: MIT
Description: 
        
        # Preference learning with Gaussian processes.
        
        [![Build Status](https://travis-ci.org/chariff/GPro.svg?branch=master)](https://travis-ci.org/chariff/GPro)
        [![Codecov](https://codecov.io/github/chariff/GPro/badge.svg?branch=master&service=github)](https://codecov.io/github/chariff/GPro?branch=master)
        [![MIT license](http://img.shields.io/badge/license-MIT-brightgreen.svg)](http://opensource.org/licenses/MIT)
        
        
        Python implementation of a probabilistic kernel approach to preference 
        learning based on Gaussian processes. Preference relations are captured 
        in a Bayesian framework which allows in turn for global optimization of 
        the inferred functions (Gaussian processes) in as few iterations as possible.
        
        Installation.
        ============
        
        ### Installation
        * From PyPI:
        
              pip install GPro
        
        * From GitHub:
        
              pip install git+https://github.com/chariff/GPro.git
        
        ### Dependencies
        GPro requires:
        * Python (>= 3.5)
        * NumPy+mkl (>= 1.9.0)
        * SciPy (< 1.5.0)
        * Pandas (>= 0.24.0) 
        
        Brief guide to using GPro.
        =========================
        
        Checkout the package docstrings for more information.
        
        ## 1. Fitting and making Predictions.
        
        ```python
        from GPro.preference import ProbitPreferenceGP
        import numpy as np
        import matplotlib.pyplot as plt
        ```
        Training data consisting of numeric real positive values.
        A minimum of two values is required. The following example is in 1D.
        ```python
        X = np.array([2, 1]).reshape(-1, 1)
        ```
        M is an array containing preferences. A preference is an 
        array of positive integers of shape = (2,). The left integer of a 
        given preference is the index of a value in X which is preferred 
        over another value of X indexed by the right integer of the same
        preference array.
        In the following example, 2 is preferred over 1.
        ```python
        M = np.array([0, 1]).reshape(-1, 2)
        ```
        Instantiate a ProbitPreferenceGP object with default parameters.
        ```python
        gpr = ProbitPreferenceGP()
        ```
        Fit a Gaussian process. A flat prior with mean zero is applied by default.
        ```python
        gpr.fit(X, M, f_prior=None)
        ```
        Predict new values.
        ```python
        X_new = np.linspace(-6, 9, 100).reshape(-1, 1)
        predicted_values, predicted_deviations = gpr.predict(X_new, return_y_std=True)
        ```
        Plot.
        ```python
        plt.plot(X_new, np.zeros(100), 'k--', label='GP predictive prior')
        plt.plot(X_new, predicted_values, 'r-', label='GP predictive posterior')
        plt.plot(X.flat, gpr.predict(X).flat, 'bx', label='Preference')
        plt.ylabel('f(X)')
        plt.xlabel('X')
        # the predicted s.d. is divided for an aesthetic purpose.
        plt.gca().fill_between(X_new.flatten(),
                               (predicted_values - predicted_deviations / 50).flatten(),
                               (predicted_values + predicted_deviations / 50).flatten(),
                               color="#b0e0e6", label='GP predictive posterior s.d.')
        plt.legend()
        plt.show()
        ```
        The following plot shows how the posterior predictive Gaussian process is adjusted to 
        the data i.e. 2 is preferred to 1. One can also notice how the standard 
        deviation is small where there is data.  
        
        ![Gaussian process posterior](https://github.com/chariff/GPro/raw/master/examples/posterior_example.png)
        
        ## 2. Interactive bayesian optimization.
        
        Preference relations are captured in a Bayesian framework 
        which allows for global optimization of the latent function 
        (modelized by Gaussian processes) describing the preference relations.
        Interactive bayesian optimization with probit responses works by querying
        the user with a paired comparison and by subsequently updating the 
        Gaussian process model. The iterative procedure optimizes a utility function,
        seeking a balance between exploration and exploitation of the latent function, 
        to present the user with a new set of instances.
        ```python
        from GPro.kernels import Matern
        from GPro.posterior import Laplace
        from GPro.acquisitions import UCB
        from GPro.optimization import ProbitBayesianOptimization
        import numpy as np
        
        # 3D example. Initialization.
        X = np.random.sample(size=(2, 3)) * 10
        M = np.array([0, 1]).reshape(-1, 2)
        ```
        Custom parameters for the ProbitBayesianOptimization object. 
        Checkout the package docstrings for more informations on the parameters.
        ```python
        GP_params = {'kernel': Matern(length_scale=1, nu=2.5),
                     'post_approx': Laplace(s_eval=1e-5, max_iter=1000,
                                            eta=0.01, tol=1e-3),
                     'acquisition': UCB(kappa=2.576),
                     'alpha': 1e-5,
                     'random_state': None}
        ```
        Instantiate a ProbitBayesianOptimization object with custom parameters.
        ```python
        gpr_opt = ProbitBayesianOptimization(X, M, GP_params)
        ```
        Bounded region of optimization space.
        ```python
        bounds = {'x0': (0, 10), 'x1': (0, 10), 'x2': (0, 10)}
        ```
        Interactive optimization method.
        Checkout the package docstrings for more informations on the parameters.
        ```python
        console_opt = gpr_opt.interactive_optimization(bounds=bounds, n_init=100, n_solve=10)
        optimal_values, suggestion, X_post, M_post, f_post = console_opt
        print('optimal values: ', optimal_values)
        ```
            >>>                   x0        x1        x2
            >>> preference  0.306996  3.581879  4.844135
            >>> suggestion  0.000000  2.749200  3.287625
            >>> Iteration 0, preference (p) or suggestion (s)? (Q to quit): p
            >>>                   x0        x1        x2
            >>> preference  0.306996  3.581879  4.844135
            >>> suggestion  0.289541  4.118421  6.052125
            >>> Iteration 1, preference (p) or suggestion (s)? (Q to quit): s
            >>>                   x0        x1        x2
            >>> preference  0.289541  4.118421  6.052125
            >>> suggestion  1.601063  4.300604  5.208000
            >>> Iteration 2, preference (p) or suggestion (s)? (Q to quit): Q
            >>> optimal values:  [0.28954095 4.11842105 6.05212487]
          
        One can use informative prior. Let's use posterior as prior for the sake of
        example.
        ```python
        gpr_opt = ProbitBayesianOptimization(X_post, M_post, GP_params)
        console_opt = gpr_opt.interactive_optimization(bounds=bounds, n_init=100, 
                                                       n_solve=10, f_prior=f_post,
                                                       max_iter=1, print_suggestion=False)
        optimal_values, suggestion, X_post, M_post, f_post = console_opt
        print('optimal values: ', optimal_values)
        ```
            >>>                   x0        x1        x2
            >>> preference  0.289541  4.118421  6.052125
            >>> suggestion  1.601063  4.300604  5.208000
            >>> Iteration 2, preference (p) or suggestion (s)? (Q to quit): Q
            >>> optimal values:  [0.28954095 4.11842105 6.05212487]
        
        Download the algorithm with a GUI fully written in python on
        * https://sensguide.com
        ## 2. Bayesian optimization of a black-box function.
        
        **Disclaimer:** For testing purposes, we maximize a multivariate normal pdf.
        ```python
        from GPro.kernels import Matern
        from GPro.posterior import Laplace
        from GPro.acquisitions import UCB
        from GPro.optimization import ProbitBayesianOptimization
        from scipy.stats import multivariate_normal
        import numpy as np
        from sklearn import datasets
        import matplotlib.cm as cm
        import matplotlib.pyplot as plt
        ```
        Uniform sampling given bounds.
        ```python
        def random_sample(n, d, bounds, random_state=None):
            if random_state is None:
                random_state = np.random.randint(1e6)
            random_state = np.random.RandomState(random_state)
            sample = random_state.uniform(bounds[:, 0], bounds[:, 1],
                                          size=(n, d))
            return sample
        ```
        Sample parameters of a multivariate normal distribution.
        ```python
        def sample_normal_params(n, d, bounds, scale_sigma=1, random_state=None):
            # sample centroids.
            mu = random_sample(n=n, d=d, bounds=np.array(list(bounds.values())),
                               random_state=random_state)
            # sample covariance matrices.
            sigma = datasets.make_spd_matrix(d, random_state) * scale_sigma
            theta = {'mu': mu, 'sigma': sigma}
            return theta
        ```
        Example is in 2 dimensions.
        ```python
        d = 2
        # Bounded region of optimization space.
        bounds = {'x' + str(i): (0, 10) for i in range(0, d)}
        # Sample parameters of a d-multivariate normal distribution
        theta = sample_normal_params(n=1, d=d, bounds=bounds, scale_sigma=10, random_state=12)
        # function to be optimized.
        f = lambda x: multivariate_normal.pdf(x, mean=theta['mu'][0], cov=theta['sigma'])
        # X, M, init
        X = random_sample(n=2, d=d, bounds=np.array(list(bounds.values())))
        X = np.asarray(X, dtype='float64')
        # Target choices. A preference is an array of positive
        # integers of shape = (2,). preference[0], is an index
        # of X preferred over preference[1], which is an index of X.
        M = sorted(range(len(f(X))), key=lambda k: f(X)[k], reverse=True)
        M = np.asarray([M], dtype='int8')
        # Parameters for the ProbitBayesianOptimization object.
        GP_params = {'kernel': Matern(length_scale=1, nu=2.5),
                     'post_approx': Laplace(s_eval=1e-5, max_iter=1000,
                                            eta=0.01, tol=1e-3),
                     'acquisition': UCB(kappa=2.576),
                     'alpha': 1e-5,
                     'random_state': None}
        # instantiate a ProbitBayesianOptimization object with custom parameters
        gpr_opt = ProbitBayesianOptimization(X, M, GP_params)
        ```
        Function optimization method.
        ```python
        function_opt = gpr_opt.function_optimization(f=f, bounds=bounds, max_iter=50,
                                                     n_init=1000, n_solve=1)
        
        optimal_values, X_post, M_post, f_post = function_opt
        print('optimal values: ', optimal_values)
        ```
            >>> optimal values:  [1.45340052 7.22687626]
        ```python
        # rmse
        print('rmse: ', .5 * sum(np.sqrt((optimal_values - theta['mu'][0]) ** 2)))
        ```
            >>> rmse:  0.13092430596422377
        ```python
        # 2d plot
        if d == 2:
            resolution = 10
            x_min, x_max = bounds['x0'][0], bounds['x0'][1]
            y_min, y_max = bounds['x1'][0], bounds['x1'][1]
            x = np.linspace(x_min, x_max, resolution)
            y = np.linspace(y_min, y_max, resolution)
            X, Y = np.meshgrid(x, y)
            grid = np.empty((resolution ** 2, 2))
            grid[:, 0] = X.flat
            grid[:, 1] = Y.flat
            Z = f(grid)
            plt.imshow(Z.reshape(-1, resolution), interpolation="bicubic",
                       origin="lower", cmap=cm.rainbow, extent=[x_min, x_max, y_min, y_max])
            plt.scatter(optimal_values[0], optimal_values[1], color='black', s=10)
            plt.title('Target function')
            plt.colorbar()
            plt.show()
        ```
        
        ![](https://github.com/chariff/GPro/raw/master/examples/mvn_example.png)
        
        ### References:
        * http://mlg.eng.cam.ac.uk/zoubin/papers/icml05chuwei-pl.pdf
        * https://arxiv.org/pdf/1012.2599.pdf
        * https://www.cs.ox.ac.uk/people/nando.defreitas/publications/BayesOptLoop.pdf
        * http://www.gaussianprocess.org/gpml/
        
        
            -- Chariff Alkhassim
Platform: UNKNOWN
Description-Content-Type: text/markdown
