Supervised, Fully-connected, strict-hierarchy, explicit Predictive Coding — PredFlow documentation

# Supervised, Fully-connected, strict-hierarchy, explicit Predictive Coding

Supervised predictive coding with analytic expressions of the gradients of the energy with respect to representations and learnable parameters.

supervised_fullyconnected_stricthierarchy_explicit_pc.infer(weights, data, ir=0.025, T=200, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False, target_shape=None)

  Implements the following logic:

      Initialize representations
      do T times
          e = r - W * r
          r += ir * (-e + tranpose(W) * e)
      return r

  Parameters

    - weights (list of 2d tf.Tensor of float32) – list of weight matrices, can be generated e.g. using `tf_utils.mlp()`

    - data (3d tf.Tensor of float32) – inuput data batch

    - ir (float, optional) – inference rate, defaults to 0.025

    - T (int, optional) – number of inference steps, defaults to 200

    - f (function, optional) – activation function, defaults to tf.nn.relu

    - df (function, optional) – derivate of the activation function, defaults to relu_derivate

    - predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False

    - target_shape (1d tf.Tensor of int32, optional) – shape of target minibatch, defaults to None

  Returns

    latent representations

  Return type

    list of 3d tf.Tensor of float32

supervised_fullyconnected_stricthierarchy_explicit_pc.learn(weights, data, target, ir=0.1, lr=0.001, T=20, f=<function relu>, df=<function relu_derivate>, predictions_flow_upward=False)

  Implements the following logic:

      Initialize representations
      do T times
          e = r - W * r
          r += ir * (-e + tranpose(W) * e)
      W += lr * (e * tranpose(r))

  Parameters

    - weights (list of 2d variable tf.Tensor of float32) – list of weight matrices, can be generated e.g. using `tf_utils.mlp()`

    - data (3d tf.Tensor of float32) – inuput data batch

    - target (3d tf.Tensor of float32) – output target batch

    - ir (float, optional) – inference rate, defaults to 0.1

    - lr (float, optional) – learning rate, defaults to 0.001

    - T (int, optional) – number of inference steps, defaults to 20

    - f (function, optional) – activation function, defaults to tf.nn.relu

    - df (function, optional) – derivate of the activation function, defaults to tf_utils.relu_derivate

    - predictions_flow_upward (bool, optional) – direction of prediction flow, defaults to False