precisions_utils module — PredFlow documentation

# precisions_utils module

precisions_utils.precision_modulated_energy(model, r, theta=[], predictions_flow_upward=False, gamma=<tf.Tensor: shape=(), dtype=float32, numpy=0.5>)

  Energy (total squared L2 norm of precision-weighted errors + regularizer) computation and autodifferentiation with respect to representations and learnable parameters

  Parameters

    - model (list of `tf_utils.Dense` or `tf_utils.BiasedDense`) – description of a sequential network by a list of layers, can be generated e.g. using `tf_utils.mlp()`

    - r (list of 3d tf.Tensor of float32) – representations

    - theta (list of variable tf.Tensor of float32, optional) – learnable parameters, defaults to []

    - predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

    - gamma (float32, optional) – Regularization scalar, defaults to .5

  Returns

    energy, autodiff gradient tape

  Return type

    float32, tf.GradientTape

precisions_utils.precision_modulated_inference_step_backward_predictions(e, r, w, P, ir, f, df, update_last=True)

  Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

  Parameters

    - e (list of 3d tf.Tensor of float32) – prediction errors

    - r (list of 3d tf.Tensor of float32) – representations

    - w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using `tf_utils.mlp()`

    - P (list of 2d tf.Tensor of float32) – list of precision matrices

    - ir (float32) – inference rate

    - f (function) – activation function

    - df (function) – derivate of the activation function

    - update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

precisions_utils.precision_modulated_inference_step_forward_predictions(e, r, w, P, ir, f, df, update_last=True)

  Representations update using stochastic gradient descent with analytic expressions of the gradients of energy wrt representations, only applicable to an unbiased MLP (predictions come from lower layer)

  Parameters

    - e (list of 3d tf.Tensor of float32) – prediction errors

    - r (list of 3d tf.Tensor of float32) – representations

    - w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using `tf_utils.mlp()`

    - P (list of 2d tf.Tensor of float32) – list of precision matrices

    - ir (float32) – inference rate

    - f (float32) – activation function

    - df (function) – derivate of the activation function

    - update_last (bool, optional) – controls weither representations in the last layer are updated, defaults to True

precisions_utils.precision_modulated_weight_update_backward_predictions(w, e, r, P, lr, f)

  Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP with reversed flow (predictions come from higher layer)

  Parameters

    - w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using `tf_utils.mlp()`

    - e (list of 3d tf.Tensor of float32) – prediction errors

    - r (list of 3d tf.Tensor of float32) – representations

    - P (list of 2d tf.Tensor of float32) – list of precision matrices

    - lr (float32) – learning rate

    - f (function) – activation function

precisions_utils.precision_modulated_weight_update_forward_predictions(w, e, r, P, lr, f)

  Weight update using stochastic gradient descent with analytic expressions of the gradients of energy wrt weights, only applicable to an unbiased MLP (predictions come from lower layer)

  Parameters

    - w (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using `tf_utils.mlp()`

    - e (list of 3d tf.Tensor of float32) – prediction errors

    - r (list of 3d tf.Tensor of float32) – representations

    - P (list of 2d tf.Tensor of float32) – list of precision matrices

    - lr (float32) – learning rate

    - f (function) – activation function

precisions_utils.precisions_update_backward_predictions(e, P, lr)

  Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from higher layer)

  Parameters

    - e (list of 3d tf.Tensor of float32) – prediction errors

    - P (list of 2d tf.Tensor of float32) – list of precision matrices

    - lr (float32) – learning rate

precisions_utils.precisions_update_forward_predictions(e, P, lr, diagonal=False)

  Precision update using stochastic gradient descent with analytic expressions of the gradients of energy wrt precision, only applicable to an unbiased MLP (predictions come from lower layer)

  Parameters

    - e (list of 3d tf.Tensor of float32) – prediction errors

    - P (list of 2d tf.Tensor of float32) – list of precision matrices

    - lr (float32) – learning rate