nets — nets 0.1 documentation

Table of Contents
Shortcuts

# nets

NETS is divided in five main sub-packages:

- autograd refers to the autograd system used,

- nn refers to the core of the package, ie deep network models,

- optim refers to optimizers used during the backpropagation to update weights and biases,

- data refers to array container, like training and testing examples with their labels,

- datasets refers to popular machine learning libraries for image analysis and NLP.

# nets

## nets.tensor

Defines tensors for deep learning application. A tensor is multi-dimensional array, similar to `numpy` arrays.

class `nets.tensor.` `Tensor`(data, requires_grad=False, hooks=None)

  A Tensor is a multi dimensional array that track and record previous gradients, creating a dynamic computational graph.

  `append`(t, axis=0)

    Append a value(- or tensor) to a `Tensor`.

    Note

    The operation takes place in-place and does not support autograd.

    Parameters

      t (scale, Tensor) – object to add

    Returns

      None

  `astype`(new_type)

    Set a new type to the `Tensor`’s data.

    Parameters

      new_type (type) – new type to convert the data

    Returns

      None

  `backward`(grad=None)

    Compute a single backward pass on all `Tensor` linked to this one. The `Tensor` depending to this top-level `Tensor``are stored in the ``_hooks` attribute. The backward pass compute a gradient back-propagation on all `Tensor` registered in `_hooks`. The backward pass gradient in `grad` attribute (and add upstream gradient if the `Tensor` is used multiple times).

    Parameters

      grad (Tensor) – upstream gradient. Default is None, and will be set to `Tensor(1.0)`, a 0-dimensional `Tensor`.

    Returns

      None

    Note

    To be able to back-propagate, the top-level `Tensor` must have `requires_grad` set to `True` to propagate the gradient.

  `detach`()

    Unlink the `Tensor` to the computational graph. By calling this method, the attribute `_hooks` and `grad` are set to their default values, `None`.

    Returns

      None

  `flatten`()

    Flatten a `Tensor` with a new shape. The transformation is not made in-place.

    Returns

      Tensor

  `item`()

    Get the item (float, int etc.) of a 0-dimensional `Tensor`. It will detach the tensor from the computational graph by setting `_hooks = []` and `grad = None` to free memory and send this graph to the garbage collector.

    Returns

      Any

  `numpy`()

    Convert the `Tensor` data to a `numpy.ndarray` object.

    Returns

      numpy.ndarray

  `register_hook`(hook)

    Register a hook to a `Tensor`

    Parameters

      hook (Hook) – hook to register

    Returns

      None

  `reshape`(*shapes)

    Reshape a `Tensor` with a new shape. The transformation is not made in-place.

    Note

    The new shape must have the same size of the actual shape. If its not the case, the reshape method will raise an error.

    Parameters

      int (*shapes) – permutation

    Returns

      Tensor

  `sum`(axis=None)

    Sum the data along a given axis. If no axis are specified, all values within the `Tensor` will be summed.

    Parameters

      axis (int) – the index of the axis to sum on.

    Returns

      Tensor

  `tolist`()

    Convert the `Tensor` data to a list (of list eventually).

    Returns

      list

  `transpose`(*indices)

    Transpose the `Tensor`. The operation is not in-place.

    Parameters

      indices (tuple) – permutation

    Returns

      Tensor

  `zero_grad`()

    Set to a zero `Tensor` the gradient. This is call when initializing a `Tensor` that requires gradient tracking, or re-initialize parameters’s gradient after a training loop as they accumulate on each other.

    Returns

      None

`nets.tensor.` `to_array`(arrayable)

  Convert an object to a `numpy.ndarray` if possible.

  Parameters

    arrayable – object to convert

  Returns

    numpy.ndarray

`nets.tensor.` `to_tensor`(tensorable)

  Convert an object to a `Tensor` if possible.

  Parameters

    tensorable – object to convert

  Returns

    Tensor

## nets.numeric

The `functional` modules defines basic functions to generate tensors and transform data.

## nets.functional

# nets.autograd

## nets.autograd.functions

This modules defines more complex operations, and defines the most popular functions such as:

- exp,

- log,

- tanh,

- sigmoid;

- relu…

`nets.autograd.functions.` `exp`(t)

  Exponentiation f a tensor.

  \[T_{out} = \text{exp}(T)\]
  Parameters

    t (Tensor) – input tensor to transform

  Returns

    Tensor

`nets.autograd.functions.` `leaky_relu`(x: Array, alpha: Optional[float] = 0.01) → Array

  Variation from the original `relu` function, which can prevent ‘dying’ `relu` thanks to a slope `alpha`.

  \[\text{leaky_relu(x)} = \max{(\alpha \times x, x)}\]
  Parameters

    alpha (float, optional) – slope towards \(-\infty\).

  Shape:

    - input: x (numpy.array): input to compute the `leaky_relu` function on.

    - output: y (numpy.array): `leaky relu` output, with the same shape than `x`.

  Examples:

      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> alpha = 0.1
      >>> out_array = leaky_relu(in_array, alpha)

  See `LeakyReLU` for the activation implementation.

`nets.autograd.functions.` `leaky_relu_prime`(x: Array, alpha: Optional[float] = 0.01) → Array

  First order derivative of `leaky_relu` function, defined as:

  \[\begin{split}\text{leaky_relu'(x)} = \begin{cases} 1, &\quad x \ge 0 \\ \alpha, &\quad x < 0. \end{cases}\end{split}\]
  Parameters

    alpha (float, optional) – slope towards \(-\infty\).

  Shape:

    - input: x (numpy.array): input to compute the `leaky_relu_prime` function on.

    - output: y (numpy.array): derivative of the input, with the same shape than `x`.

  Examples:

      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> alpha = 0.1
      >>> out_array = leaky_relu_prime(in_array, alpha)

  See `LeakyReLU` for the activation implementation.

`nets.autograd.functions.` `log`(t)

  Compute the logarithm of a tensor object.

  \[T_{out} = \text{log}(T)\]
  Parameters

    t (Tensor like) – input tensor

  Returns

    Tensor

`nets.autograd.functions.` `maximum`(t1, t2)

  Get the maximum between two tensors.

  Parameters

    - t1 (Tensor) – input tensor to merge

    - t2 (Tensor) – input tensor to merge

  Returns

    Tensor

`nets.autograd.functions.` `minimum`(t1, t2)

  Get the minimum between two tensors.

  Parameters

    - t1 (Tensor) – input tensor to merge

    - t2 (Tensor) – input tensor to merge

  Returns

    Tensor

`nets.autograd.functions.` `pow`(t, power)

  Power a tensor-like object.

  \[T_{out} = T^2\]
  Parameters

    - t (Tensor like) – reference tensor

    - power (int) – power to elevate a tensor

  Returns

    Tensor

`nets.autograd.functions.` `relu`(x: Array) → Array

  `relu` is a standard activation function, defined as:

  \[\text{relu(x)} = \max{(0, x)}\]
  Shape:

    - input: x (numpy.array): input to compute the `relu` function on.

    - output: y (numpy.array): `relu` output, with the same shape than `x`.

  Examples:

      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> out_array = relu(in_array)

  See `ReLU` for the activation implementation.

`nets.autograd.functions.` `relu_prime`(x: Array) → Array

  First order derivative of the `relu` function.

  \[\begin{split}\text{relu'(x)} = \begin{cases} 1, &\quad x \ge 0 \\ 0, &\quad x < 0. \end{cases}\end{split}\]
  Shape:

    - input: x (numpy.array): input to compute the `leaky_relu` function on.

    - output: y (numpy.array): gradient of the input, with the same shape than `x`.

  Examples:

      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> out_array = relu_prime(in_array)

  See `ReLU` for the activation implementation.

`nets.autograd.functions.` `sigmoid`(x: Array) → Array

  Vanilla `sigmoid` function, defined as:

  \[\text{sigmoid}(x) = \frac{1}{1 + e^{-x}}\]
  Shape:

    - input: x (numpy.array): input to compute the `sigmoid` function on.

    - output: y (numpy.array): `sigmoid` output, with the same shape than `x`.

  Examples:

      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> out_array = sigmoid(in_array)

  See `Sigmoid` for the activation implementation.

`nets.autograd.functions.` `sigmoid_prime`(x: Array) → Array

  First order derivative of `sigmoid` function, defined as:

  \[\text{sigmoid'}(x) = (1 - \text{sigmoid}(x))\]
  Shape:

    - input: x (numpy.array): input to compute the `sigmoid derivative` function on.

    - output: y (numpy.array): gradient of the input, with the same shape than `x`.

  Examples:

      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> out_array = sigmoid_prime(in_array)

  See `Sigmoid` for the activation implementation.

`nets.autograd.functions.` `softmax`(x: Array, axis: Optional[int] = 0) → Tensor

  `softmax` function. The implementation chosen is not straight forward to prevent numerical instability.

  The \(i^{th}\) element of a softmax function evaluated on a vector \(x \in \mathbb{R}^n\) is given by:

  \[\text{softmax}(x_{i}) = \frac{e^{x_i}}{\sum_{j=1}^{n} e^{x_j}}\]
  Parameters

    axis (int) – axis to consider for the `softmax`. Default is `0`.

  Shape:

    - input: x (numpy.array): input to compute the `softmax` function on.

    - output: y (numpy.array): hyper-plane of the input.

  See `Softmax` for the activation implementation.

`nets.autograd.functions.` `sqrt`(t)

  Square root of a tensor-like object.

  \[T_{out} = T^2\]
  Parameters

    - t (Tensor like) – reference tensor

    - power (int) – power to elevate a tensor

  Returns

    Tensor

`nets.autograd.functions.` `tanh`(t: Tensor) → Tensor

  `tanh` standard function, definead as:

  \[\text{tanh}(T) = \frac{e^{T} - e^{-T}}{e^{T} + e^{-T}}\]
  Shape:

    - input: x (numpy.array): input to compute the `tanh` function on.

    - output: y (numpy.array): `tanh` output, with the same shape than \(T\).

  Examples:

      >>> import nets
      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> out_array = nets.tanh(in_array)

  See `Tanh` for the activation implementation.

`nets.autograd.functions.` `tanh_prime`(x: Array) → Array

  First order derivative of `tanh` function, defined as:

  \[\text{tanh'}(x) = 1 - \text{tanh}^2(x)\]
  Shape:

    - input: x (numpy.array): input to compute the `tanh derivative` function on.

    - output: y (numpy.array): gradient of the input, with the same shape than `x`.

  Examples:

      >>> in_array = np.array([-5, 2, 6, -2, 4])
      >>> out_array = tanh(in_array)

  See `Tanh` for the activation implementation.

`nets.autograd.functions.` `where`(cond, t1, t2)

  Transformation regarding a condition.

  \[\begin{split}T_{out} = \begin{cases} T_1, &\quad if \quad condition \\ T_2, &\quad else. \end{cases}\end{split}\]
  Parameters

    - cond (bool) – condition to merge two tensors

    - t1 (Tensor) – input tensor to merge

    - t2 (Tensor) – input tensor to merge

  Returns

    Tensor

## nets.autograd.hook

A `Hook` keeps track of gradients and operations.

class `nets.autograd.hook.` `Hook`(tensor, grad_fn)

  A hook is a collection of tensors and gradients.

## nets.autograd.numeric

This modules defines basic transformations on a `Tensor` like `transpose` or `reshape`.

`nets.autograd.numeric.` `append`(t, value)

  Append multiples `Tensor` from an iterable.

  Note

  The `Tensor` in `iterable` should and must have the same shape.

  Parameters

    t (Tensor) – list containing `Tensor` to concatenate.

  Returns

    the concatenation of all `Tensor`.

  Return type

    Tensor

`nets.autograd.numeric.` `argmax`(t, axis=None)

  Get the indices of maximum elements from a `Tensor`.

  Parameters

    - t (Tensor) – tensor get maximum indices from

    - axis (int, optional) – index of the axis. Default is `None`.

  Returns

    Tensor

`nets.autograd.numeric.` `concatenate`(iterable)

  Concatenate multiples `Tensor` from an iterable.

  Note

  The `Tensor` in `iterable` should and must have the same shape.

  Parameters

    iterable (tuple, list) – list containing `Tensor` to concatenate.

  Returns

    the concatenation of all `Tensor`.

  Return type

    Tensor

`nets.autograd.numeric.` `flatten`(t)

  Reshape in 1-dimensional a `Tensor`.

  Parameters

    t (Tensor) – tensor get reshape.

  Returns

    Tensor

`nets.autograd.numeric.` `max`(t, axis=None)

  Get the maximum from a `Tensor`.

  Parameters

    - t (Tensor) – tensor to transform

    - axis (int, optional) – index of the axis to search. Default is `None`.

  Returns

    Tensor

`nets.autograd.numeric.` `pad`(t, padding, constant_values=0)

  Reshape a `Tensor` to a bigger size and add a `padding` on the side, with a `0` constant value.

  Parameters

    - t (Tensor) – tensor to transform

    - padding (tuple) – padding dimensions

    - constant_values (scalar, optional) – scalar affected in the padding

  Returns

    Tensor

`nets.autograd.numeric.` `reshape`(t, shape)

  Reshape a `Tensor`.

  Parameters

    - t (Tensor) – tensor to transform

    - shape (tuple) – new shape of `t`

  Returns

    Tensor

`nets.autograd.numeric.` `transpose`(t, indices=None)

  Permutation a tensor object.

  Parameters

    - t (Tensor) –

    - indices (tuple, optional) – index to transpose.

  Returns

    Tensor

## nets.autograd.ops

Defines basic operations between two tensors, like addition, subtraction, dot product etc.

`nets.autograd.ops.` `add`(t1, t2)

  Add two tensor-like together.

  \[T_{out} = T_1 + T_2\]
  Parameters

    - t1 (Tensor like) – tensor to add

    - t2 (Tensor like) – second tensor to add with

  Returns

    the sum of two Tensor-like object

  Return type

    Tensor

`nets.autograd.ops.` `div`(t1, t2)

  Divide two tensor-like object.

  \[T_{out} = T_1 \times \frac{1}{T_2}\]
  Parameters

    - t1 (Tensor like) – tensor to multiply

    - t2 (Tensor like) – tensor to invert

  Returns

    Tensor

`nets.autograd.ops.` `dot`(t1, t2)

  Dot product of two matrices.

  \[T_{out} = (t_{i, j}^{[out]})_{i, j} \quad where \quad t_{i, j}^{[out]} = \sum_{k=1}^{n} t_{i, k}^{[1]} \times t_{k, j}^{[2]}\]
  Parameters

    - t1 (Tensor like) –

    - t2 (Tensor like) –

  Returns

    Tensor

`nets.autograd.ops.` `eq`(t, other)

  Return a boolean tensor for equal condition.

  \[condition = T == other\]
  Parameters

    - t (Tensor) – tensor to compare

    - other (Tensor like) – object to compare the tensor

  Returns

    Tensor

`nets.autograd.ops.` `ge`(t, other)

  Return a boolean tensor for greater or equal condition.

  \[condition = T \ge other\]
  Parameters

    - t (Tensor) – tensor to compare

    - other (Tensor like) – object to compare the tensor

  Returns

    Tensor

`nets.autograd.ops.` `gt`(t, other)

  Return a boolean tensor for greater than condition.

  \[condition = T > other\]
  Parameters

    - t (Tensor) – tensor to compare

    - other (Tensor like) – object to compare the tensor

  Returns

    Tensor

`nets.autograd.ops.` `inverse`(t)

  Inverse a tensor-like object.

  \[T_{out} = \frac{1}{T}\]
  Parameters

    t (Tensor like) – tensor to inverse.

  Returns

    Tensor

`nets.autograd.ops.` `le`(t, other)

  Return a boolean tensor for lower or equal condition.

  \[condition = T \le other\]
  Parameters

    - t (Tensor) – tensor to compare

    - other (Tensor like) – object to compare the tensor

  Returns

    Tensor

`nets.autograd.ops.` `lt`(t, other)

  Return a boolean tensor for lower than condition.

  \[condition = T < other\]
  Parameters

    - t (Tensor) – tensor to compare

    - other (Tensor like) – object to compare the tensor

  Returns

    Tensor

`nets.autograd.ops.` `multiply`(t1, t2)

  Elementwise multiplication of two tensors-like object.

  \[T_{out} = T_1 \times T_2\]
  Parameters

    - t1 (Tensor like) – tensor to multiply

    - t2 (Tensor like) – second tensor to multiply with

  Returns

    Tensor

`nets.autograd.ops.` `ne`(t, other)

  Return a boolean tensor for not equal condition.

  \[condition = T not other\]
  Parameters

    - t (Tensor) – tensor to compare

    - other (Tensor like) – object to compare the tensor

  Returns

    Tensor

`nets.autograd.ops.` `neg`(t)

  Oppose the values of a tensor.

  \[T_{out} = - T\]
  Parameters

    t (Tensor) – tensor to oppose.

  Returns

    Tensor

`nets.autograd.ops.` `slice`(t, indices)

  Slice a tensor from given indices.

  Parameters

    - t (Tensor) – tensor to slice

    - (tuple, int, (idxs) – ): indices to extract data

  Returns

    Tensor

`nets.autograd.ops.` `sub`(t1, t2)

  Subtract two tensor-like object

  \[T_{out} = T_1 - T_2\]
  Parameters

    - t1 (Tensor like) – tensor to subtract

    - t2 (Tensor like) – second tensor to subtract with

  Returns

    Tensor

`nets.autograd.ops.` `sum`(t, axis=None, keepdims=False)

  Compute the sum of all elements in a tensor, and update the gradients and hooks.

  \[\text{sum} = \sum_{idx} t_{idx}\]
  Parameters

    - t (Tensor) – tensor to get the sum from

    - axis (int) – axis to sum

    - keepdims (bool) – keep the same dimension in the resulting `Tensor` as the input if set to `True`. Default is `False`.

  Returns

    summed tensor

  Return type

    Tensor

## nets.autograd.parameter

A parameter is a trainable tensor.

class `nets.autograd.parameter.` `Parameter`(data=None, shape=None)

  Instantiate a parameter, made of trainable data. A trainable data is a value that will be updated during the back-propagation, usually it refers to `weights` and `biases` of a layer.

  classmethod `normal`(shape, mu=0, sigma=1)

    Generate a `Parameter` following a normal distribution center at `mu` with a standard deviation of `sigma`.

    Parameters

      - shape (tuple) – shape of the `Parameter`

      - mu (scalar) – mean of the normal distribution. Default is `0`.

      - sigma (scalar) – standard deviation of the normal distribution. Default is `1`.

    Returns

      Parameter

  classmethod `orthogonal`(shape)

    Initializes weight parameters orthogonally. From the [exercise 02456 from DTU course](https://github.com/DeepLearningDTU/02456-deep-learning-with-PyTorch).

    Note

    Refer to [this paper](https://arxiv.org/abs/1312.6120) for an explanation of this initialization.

    Parameters

      shape (tuple) – shape of dimensionality greater than 2 (weight matrix)

    Returns

      Parameter

  classmethod `scaled_weight`(input_dim, output_dim)

    Scaled initialization from \(He et al.\)

    Parameters

      - input_dim (int) – dimension of the input layer

      - output_dim (int) – dimension of the output layer

    Returns

      Parameter

  classmethod `uniform`(shape, low=-1, high=1)

    Generate a `Parameter` with data following a uniform distribution between `lower` and `upper`.

    Parameters

      - shape (tuple) – shape of the `Parameter`

      - low (scalar) – lower bound of the uniform distribution. Default is `-1`.

      - high (scalar) – upper bound of the uniform distribution. Default is `1`.

    Returns

      Parameter

  classmethod `zeros`(shape)

    Generate a zero-Parameter

    Parameters

      shape (tuple) – shape of the `Parameter`

    Returns

      Parameter

# nets.nn

The main core of NETS is located in nn package. The neural networks implemented are the most popular ones:

- Linear Neural Network,

- Dense Neural Network,

- Convolutional Neural Network,

- Recurent Neural Network.

## nets.nn.activation

This modules defines all activation function, used in neural networks to add non-linearity from one layer to another.

Usage:

    import numpy as np
    import nets.nn.activation as A

    # Define a ReLU activation function
    # This activation function is popular as an hidden activation function
    activation_function = A.ReLU()
    # Defines 3 1-D array of length 5, stacked together.
    batch_input = np.array([[1, -2, 5, 10, -7],
                            [2, 4, 3, -2, 5],
                            [-3, 2, 8, 4, 0]])

    # Check the result for one single pass
    batch_output = activation_function(batch_input)
    print(batch_output)

    # Check the backward pass
    backward = ReLU.backward(batch_ouput)
    print(backward)

> Out:

    array([[1, 0, 5, 10, 0],
           [2, 4, 3, 0, 5],
           [0, 2, 8, 4, 0]])
    array([[1, 0, 1, 1, 0],
           [1, 1, 1, 0, 1],
           [0, 1, 1, 1, 0]])

class `nets.nn.activation.` `Activation`(func, func_prime)

  An activation modules is a transformation that modify its inputs element wise, usually it uses non-linearity functions.

  `backward`(grad)

    One backward step.

  `forward`(inputs)

    One forward step. Gradients and outputs should be saved in the `_cache` when training, to be able to perform the backward pass.

class `nets.nn.activation.` `LeakyReLU`

  `LeakyReLU` activation function.

  \(\text{LeakyReLU.forward}(x) = \text{leaky_relu(x)} = \max{(\alpha \times x, x)}\)

  \(\text{LeakyReLU.backward}(x) = \text{leaky_relu'(x)} = \begin{cases} 1, &\quad x \ge 0 \\ \alpha, &\quad x < 0. \end{cases}\)

  Examples:

      >>> activation = LeakyReLU()
      >>> batch_input = np.array([[-5, 2, 6, -2, 4],
      ...                         [2, 5, -6, 7, -3]])
      >>> output = activation(batch_input)
      >>> gradient = activation.backward(output)

  See `leaky_relu()` and `leaky_relu_derivative()` for the functional implementation.

class `nets.nn.activation.` `ReLU`

  `ReLU` activation function.

  \(\text{ReLU.forward}(x) = \text{relu(x)} = \max{(0, x)}\)

  \(\text{ReLU.backward}(x) = \text{relu'(x)} = \begin{cases} 1, &\quad x \ge 0 \\ 0, &\quad x < 0. \end{cases}\)

  Examples:

      >>> activation = LeakyReLU()
      >>> batch_input = np.array([[-5, 2, 6, -2, 4],
      ...                         [2, 5, -6, 7, -3]])
      >>> output = activation(batch_input)
      >>> gradient = activation.backward(output)

  See `relu()` and `relu_derivative()` for the functional implementation.

class `nets.nn.activation.` `Sigmoid`

  `Sigmoid` activation function.

  \(\text{Sigmoid.forward}(x) = \text{sigmoid}(x) = \frac{1}{1 + e^{-x}}\)

  \(\text{Sigmoid.backward}(x) = \text{sigmoid'}(x) = (1 - \text{sigmoid}(x))\)

  Examples:

      >>> activation = Sigmoid()
      >>> batch_input = np.array([[-5, 2, 6, -2, 4],
      ...                         [2, 5, -6, 7, -3]])
      >>> output = activation(batch_input)
      >>> gradient = activation.backward(output)

  See `sigmoid()` and `sigmoid_derivative()` for the functional implementation.

class `nets.nn.activation.` `Softmax`(axis=0)

  Softmax activation function, which should be used only at the last layer to normalize outputs values between \([0, 1]\).

  `backward`(grad)

    One backward step.

  `forward`(inputs)

    One forward step. Gradients and outputs should be saved in the `_cache` when training, to be able to perform the backward pass.

class `nets.nn.activation.` `Tanh`

  `Tanh` activation function.

  \(\text{Tanh.forward}(x) = \text{tanh}(x) = \frac{e^{x} - e^{-x}}{e^{x} + e^{-x}}\)

  \(\text{Tanh.backward}(x) = \text{tanh'}(x) = 1 - \text{tanh}^2(x)\)

  Examples:

      >>> activation = Tanh()
      >>> batch_input = np.array([[-5, 2, 6, -2, 4],
      ...                         [2, 5, -6, 7, -3]])
      >>> output = activation(batch_input)
      >>> gradient = activation.backward(output)

  See `tanh()` and `tanh_derivative()` for the functional implementation.

## nets.nn.functional

Defines elementary functions used in Neural Network layers.

`nets.nn.functional.` `dropout`(t, prob=0.5)

  Zeros elements from a `Tensor` with a probability `prob`.

  \[\begin{split}\text{dropout}(T) = T \times Z \quad \text{where} Z = (z_{i})_{i} \quad and z_i = \begin{cases} 1, &\quad p \ge prob \\ 0, &\quad else. \end{cases}\end{split}\]
  Parameters

    - t (Tensor) – tensor to zeros

    - prob (float [0, 1]) – probability to zero an element

  Returns

    input tensor with some zeros

  Return type

    Tensor

## nets.nn.loss

Loss functions evaluate the precision and correctness of a model’s predictions. The most popular ones are `MSE`, `CrossEntropyLoss` and `Adam`. Each loss functions presents advantages, and results may vary when choosing one from another.

class `nets.nn.loss.` `CrossEntropyLoss`

  Cross Entropy Loss. First, a softmax transformation is used to map the predictions between \([0, 1]\), then the cost is computed:

  \[\text{CrossEntropyLoss} = - \frac{1}{N} \sum_{i=1}^{c}labels_{i}\log(pred_{i})\]
  `backward`(predictions, labels)

    One backward step.

  `forward`(predictions, labels)

    Compute the cross entropy cost function.

    Parameters

      - predictions (numpy.array) – tensor of un-normalized floats with shape \((N, c)\).

      - labels (numpy.array) – tensor of integer values with shape \((N)\).

    Returns

      the cost regarding the loss function.

    Return type

      cost (float)

class `nets.nn.loss.` `Loss`

  A loss function evaluate the correctness of a set of predictions regarding gold-labels. The predictions should be un-corrected, ie no transformations like `Softmax` should have been used before. The loss function will do the transformation if necessary. The attribute `history` keeps track of the cost when the loss function is called.

  `forward`(predictions, labels)

    Compute the cross entropy cost function.

    Parameters

      - predictions (numpy.array) – tensor of un-normalized floats with shape \((N, c)\).

      - labels (numpy.array) – tensor of integer values with shape \((N)\).

    Returns

      the cost regarding the loss function.

    Return type

      cost (float)

class `nets.nn.loss.` `MSELoss`

  Mean Square Error Loss, defined as:

  \[\text{MSE} = \frac{1}{N}\sum_{i=1}^{c}(predictions - labels)^2\]
  `backward`(predictions, labels)

    One backward step.

  `forward`(predictions, labels)

    Compute the cross entropy cost function.

    Parameters

      - predictions (numpy.array) – tensor of un-normalized floats with shape \((N, c)\).

      - labels (numpy.array) – tensor of integer values with shape \((N)\).

    Returns

      the cost regarding the loss function.

    Return type

      cost (float)

## nets.nn.utils

`nets.nn.utils.` `one_hot`(Y, num_classes)

  Perform one-hot encoding on input Y.

  \[\begin{split}\text{Y'}_{i, j} = \begin{cases} 1, &\quad if \quad Y_i = 0 \\ 0, &\quad else \end{cases}\end{split}\]
  Parameters

    - Y (Tensor) – 1D tensor of classes indices of length \(N\)

    - num_classes (int) – number of classes \(c\)

  Returns

    one hot encoded tensor of shape \((N, c)\)

  Return type

    Tensor

# nets.nn.modules

## nets.nn.conv

This modules defines a Convolution Neural Network (CNN) naively. This CNN is for test and comparisons purposes. If you wan to use a more appropriate CNN for your models, use the `CNN` instead.

class `nets.nn.modules.conv.` `Conv2d`(in_channels, out_channels, filter_size, stride=1, pad=0)

  Convolutional Neural Networks (CNN) are a class of Neural Networks that use convolution filters. Their particularity is their ability to synthesis information and learn spatial features. They are mainly used in Image Analysis, but are also known as sliding windows in Natural Language Processing.

  `Conv2d` networks applies a 2-d convolution on a 4-d tensor.

  `backward`(dout)

    One backward step.

  `forward`(inputs)

    One forward step. Gradients and outputs should be saved in the `_cache` when training, to be able to perform the backward pass.

  `inner_repr`()

    Display the inner parameter of a CNN

## nets.nn.dnn

class `nets.nn.modules.dnn.` `DNN`(layer_dimensions, activation_hidden=ReLU( (parameters): ))

  Dense Neural Network.

  `layer_dimensions`

    `list` of `int` containing all layer dimensions. The dimension at index `i` is the value of `layer_dimensions` at this index.

    Type

      list(int)

  `hidden_dimensions`

    `list` of `int` containing only hidden dimensions.

    Type

      list(int)

  `training`

    Boolean to indicate if we are training or not. This function can namely be used for inference only, in which case we do not need to store the features values.

    Type

      bool

  `activation_hidden`

    activation function used in hidden layers.

    Type

      Activation

  `activation_output`

    activation function used in the last layer.

    Type

      Activation

  `_parameters`

    `dict` containing values of weights and biases for each layers. - `weight_{i}`: contains the weight matrix of layer `i` - `bias_{i}`: contains the bias array of layer `i`

    Type

      dict

  `_cache`

    `dict` with - the linear combinations Z^[l] = W^[l]a^[l-1] + b^[l] for l in [1, L]. - the activations A^[l] = activation(Z^[l]) for l in [1, L]. We cache them in order to use them when computing gradients in the back propagation.

    Type

      dict

  `backward`(outputs, labels)

    Update parameters using backpropagation algorithm.

    Parameters

      - outputs (numpy.array) – matrix of floats with shape (num_classes, batch_size).

      - labels (numpy.array) – numpy array of integers with shape (num_classes, batch_size). Collection of one-hot encoded true input labels.

    Returns

      Dictionary with matrices that is to be used in the parameter update. Contains

        - the gradient of the weights, grad_W^[l] for l in [1, L].

        - the gradient of the biases grad_b^[l] for l in [1, L].

    Return type

      grad_params

  `forward`(inputs)

    One forward step.

    Parameters

      inputs (numpy.array) – float numpy array with shape (n^[0], batch_size). Input image batch.

    Returns

      float numpy array with shape (n^[L], batch_size). The output predictions of the

        network, where n^[L] is the number of prediction classes. For each input i in the batch, output[c, i] gives the probability that input `i` belongs to class `c`.

    Return type

      outputs (numpy.array)

  `init_parameters`()

    Initialize the parameters dictionary. For Dense Neural Network, the parameters are either weights or biases. They are saved in the dictionary with the following keys:

    > - w_{i}: weight matrix at layer i,

    > - b_{i}: bias vector at layer i.

## nets.nn.dropout

Define a `Dropout` layer.

class `nets.nn.modules.dropout.` `Dropout`(prob=0.5)

  Applies a `dropout` to an input `Tensor` with probability `prob`. The effect of this layer is to zeros elements of the incoming tensors, and cancel some neighbours effect / interactions from one layer to another.

  `backward`(outputs)

    One backward step.

  `forward`(inputs)

    One forward step. Gradients and outputs should be saved in the `_cache` when training, to be able to perform the backward pass.

## nets.nn.linear

Linear layers are widely used either for Dense Neural Module or Convolutional Neural Module, to cite the most popular. The architecture is made of two sets of `neurons` aka `perceptrons`, all connected with weights and biases.

class `nets.nn.modules.linear.` `Linear`(input_dim, output_dim)

  A linear layer is made of a weight matrix \(W\) of shape \((\text{input_dim}, \text{output_dim})\) and a bias vector \(b\) of shape \((\text{output_dim})\). The linear transformation for an incoming vector \(x\) of shape \((N, \text{input_dim})\) results in a vector \(y\) of shape \((N, \text{output_dim})\):

  \[y = x W + b\]

  Examples:

      >>> model = Linear(5, 10)
      >>> batch_input = np.array([[-5, 2, 6, -2, 4],
      ...                         [2, 5, -6, 7, -3]])
      >>> batch_pred = model(batch_input)

  `backward`(dout)

    Backward pass for a single Linear layer.

    Shape:

      - input (Tensor): upstream gradient.

      - output (Tensor): downstream gradient after a linear transformation.

  `forward`(x)

    Forward pass. Compute the linear transformation and save the results in the _cache, which will be used during the backward pass.

    Shape:

      - input (numpy.array): batch inputs of shape.

      - output (numpy.array): results of the linear transformation,

          of shape \((N, \text{input_dim})\), with \(N = \text{batch_size}\).

  `init_params`(mode=None)

    Initialize the parameters dictionary. For Dense Neural Module, the parameters are either weights or biases. They are saved in the dictionary _params with the following keys: weight matrix `w`, bias vector `b`. The initialization can be changed with `mode` parameter, between the default uniform \(\mathcal{U}(0, 1)\) initialization or use \(\text{He et al.} \quad \mathcal{N} (0, \frac{1}{input_dim})\).

  `inner_repr`()

    Display the inner parameter of a Linear layer

## nets.nn.module

Modules are the main architecture for all transformations from one tensor to another. In other words, a neural network is a succession of modules (layer, convolution, activation…). When building a custom neural network, your model must inherits from `Module` abstract class and override the `forward` method. Moreover, you can specify the back-propagation rule in `backward` method. Usually, the `backward` method computes the naive back-propagation using only local gradients saved in the modules’s `_cache`. If you don’t specify it, NETS will uses `autograd` functionality to compute all gradients.

class `nets.nn.modules.module.` `Module`

  Abstract Module architecture. All models used to transform tensors should extends from this class to benefits `forward` and `backward` propagation rules.

  `add`(*modules)

    Add modules to the current one.

    Parameters

      modules (Module) – modules to add

  `backward`(*outputs)

    One backward step.

  `cache`()

    Iterator through all cache dict

  `eval`()

    Set the `training` attribute to evaluation mode.

  abstract `forward`(*inputs)

    One forward step. Gradients and outputs should be saved in the `_cache` when training, to be able to perform the backward pass.

  `get_name`()

    Quick access to get the name of a modules.

    Returns

      modules’s name

    Return type

      string

  `gradients`()

    Iterator through all gradients

  `inner_repr`()

    Return the representation of a single modules. This method should be unique for each modules.

    Returns

      the representation of one modules.

    Return type

      string

  `load_state`(state_dict)

    Load parameters from a `state_dict` dictionary.

  `modules`()

    Iterator through all gradients

  `parameters`()

    Iterator through all parameters

  `save`(filename='model.pickle')

    Save a model as a PICKLE file.

  `save_dict`(filename='state_dict.json')

    Save the state as a JSON file.

  `state_dict`()

    Save all parameters in a dictionary.

  `train`()

    Set the `training` attribute to training mode.

  `zero_grad`()

    Zero grad all parameters within a modules

## nets.nn.pool

This modules defines a `Pooling` layer. Usually such layer is used after a convolutional layer.

class `nets.nn.modules.pool.` `MaxPool2d`(pool_size, stride=1, pad=0)

  A `Pooling` layer extract features from a multi dimensional `Tensor` and map them into another one. This extraction is used to decrease the dimension of the input, and often used after a convolutional layer.

  `backward`(dout)

    Manual backward pass for a MaxPool2d layer.

  `forward`(x)

    Forward pass.

  `inner_repr`()

    Display the inner parameter of a CNN

## nets.nn.rnn

Defines a basic Recurrent Neural Network.

class `nets.nn.modules.rnn.` `RNN`(input_dim, hidden_dim)

  Recurrent neural network (RNN) is a type of neural network that has been successful in modelling sequential data, e.g. language, speech, protein sequences, etc.

  A RNN performs its computations in a cyclic manner, where the same computation is applied to every sample of a given sequence. The idea is that the network should be able to use the previous computations as some form of memory and apply this to future computations. From the [exercise 02456 from DTU course](https://github.com/DeepLearningDTU/02456-deep-learning-with-PyTorch).

  `backward`(dout)

    Computes the backward pass of a vanilla RNN. Save gradients parameters in the `_grads` parameter.

    Parameters

      dout (Tensor) – upstream gradient.

    Returns

      downstream gradient

    Return type

      Tensor

  `forward`(inputs)

    Computes the forward pass of a vanilla RNN.

    \[ \begin{align}\begin{aligned}h_{0} = 0\\h_t = ext{tanh}(x W_{ih} + h_{t-1} W_{hh} + b_{h})\\y = h_t W_{ho} + b_{o}\end{aligned}\end{align} \]
    Parameters

      inputs – sequence of inputs to be processed

  `set_hidden_0`(hidden_cell)

    Set the first hidden cell.

## nets.nn.rnnbase

Defines the general structure of a Recurrent Network, which can takes the form of a:

- Recurrent Neural Network,

- Long Short Term Memory,

- Gated Recurrent Unit.

class `nets.nn.modules.rnnbase.` `RNNBase`(*args, **kwargs)

  Base architecture for Recurrent Networks. Every general recurrent layers should extends this class to inherits private attributes and helper methods.

## nets.nn.sequential

`Sequential` models are an ordered succession of modules.

class `nets.nn.modules.sequential.` `Sequential`(*modules)

  A `Sequential` model is build from a succession of `Modules`. All of them must have a `forward` method. In addition, a `backward` pass is created by default, running the back-propagation in all modules previously added. If for one of the modules the `backward` pass is not implemented, it will raise an error.

  `backward`(grad)

    Vanilla backward pass. This pass computes local gradients from `parameters` saved in its `_cache`.

    Shape:

      - inputs (Tensor): upstream gradient. The first downstream gradient is usually the `loss`.

      - outputs (Tensor): last downstream gradient.

  `forward`(inputs)

    Compute the forward pass for all modules within the sequential modules.

    Shape:

      - inputs (Tensor): incoming data.

      - outputs (Tensor): result of all forward pass.

# nets.optim

## nets.optim.optimizer

class `nets.optim.optimizer.` `Optimizer`(parameters)

  Optimizer. Modify a model’s parameters, and update its weights / biases.

  `backward`()

    Update rules without `autograd`

  abstract `step`()

    Update the parameters. Should be used only with `autograd` system

  `zero_grad`()

    Zero grad all parameters contained in `parameters` attribute.

    Returns

      None

## nets.optim.adagrad

class `nets.optim.adagrad.` `Adagrad`(parameters, lr=0.01, epsilon=1e-08)

  Adagrad optimizer. More sophisticated version of `SGD`, with learning rate decay.

  `step`()

    Performs Adagrad update rules.

## nets.optim.adam

class `nets.optim.adam.` `Adam`(parameters, lr=0.01, beta1=0.9, beta2=0.999, epsilon=1e-08)

  Uses the Adam update rule, which incorporates moving averages of both the gradient and its square and a bias correction term.

  `step`()

    Performs Adam update rules.

## nets.optim.rmsprop

Uses the RMSProp update rule, which uses a moving average of squared gradient values to set adaptive per-parameter learning rates.

class `nets.optim.rmsprop.` `RMSprop`(parameters, lr=0.01, decay=0.99, epsilon=1e-08)

  Uses the RMSProp update rule, which uses a moving average of squared gradient values to set adaptive per-parameter learning rates.

  `step`()

    Performs RMSProp update.

## nets.optim.sgd

Stochastic Gradient Descent is a popular optimizer for machine learning purposes.

class `nets.optim.sgd.` `SGD`(parameters, lr=0.01, momentum=0)

  Stochastic Gradient Descent optimizer follows the parameters optimization:

  Note

  A momentum can be added to this process.

  `step`()

    Performs stochastic gradient descent with momentum.

# nets.solver

## nets.solver.solver

class `nets.solver.solver.` `Solver`(model=None, criterion=None, optimizer=None)

  Train and evaluate models.

  Parameters

    - model (Module) – model to optimize or test.

    - criterion (Loss) – loss function.

    - optimizer (Optimizer) – optimizer for weights and biases.

  Attributes::

    model (Module): model to optimize or test. checkpoint (dict): checkpoint of the best model tested. criterion (Loss): loss function. optimizer (Optimizer): optimizer for weights and biases. performance (dict):

    > - train (dict):

    >     - loss (list(float))

    >     - accuracy (list(float))

    >     - precision (list(float))

    >     - recall (list(float))

    >     - macro_f1 (list(float))

    >     - confusion_matrix (list(list(list(int))))

    > - eval (dict):

    >     - loss (list(float))

    >     - accuracy (list(float))

    >     - precision (list(float))

    >     - recall (list(float))

    >     - macro_f1 (list(float))

    >     - confusion_matrix (list(list(list(int))))

  abstract `evaluate`(iterator, *args, **kwargs)

    Evaluate one time the model on iterator data.

    Parameters

      iterator (Iterator) – iterator containing batch samples of data.

    Returns

      the performance and metrics of the training session.

    Return type

      dict

  `get_accuracy`(y_tilde, y)

    Compute accuracy from predicted classes and gold labels.

    Parameters

      - y_tilde (Tensor) – 1D tensor containing the predicted classes for each predictions in the batch. This tensor should be computed through get_predicted_classes(y_hat) method.

      - y (Tensor) – gold labels. Note that y_tilde an y must have the same shape.

    Returns

      the mean of correct answers.

    Return type

      float

  `reset`()

    Reset the performance dictionary.

    Returns

      None

  `run`(epochs, train_iterator, eval_iterator, *args, **kwargs)

    Train and evaluate a model X times. During the training, both training and evaluation results are saved under the performance attribute.

    Parameters

      - epochs (int) – number of times the model will be trained.

      - train_iterator (Iterator) – iterator containing batch samples of data.

      - eval_iterator (Iterator) – iterator containing batch samples of data.

      - verbose (bool, optional) – if True display a progress bar and metrics at each epoch. The default is True.

    Returns

      None

    Examples:

        >>> performer = MySolver(model, criterion, optimizer)
        >>> # Train & eval EPOCHS times
        >>> EPOCHS = 10
        >>> performer.run(EPOCHS, train_iterator, eval_iterator, verbose=True)
            Epoch:        1/10
            Training:     100% | [==================================================] | Time: 2m 26s
            Validation:   100% | [==================================================] | Time: 0m 4s
            Stats Training:    | Loss: 0.349 | Acc: 84.33% | Prec.: 84.26% | Rec.: 84.33% | F1: 84.26%
            Stats Evaluation:  | Loss: 0.627 | Acc: 72.04% | Prec.: 72.22% | Rec.: 72.17% | F1: 72.22%
        >>> # ...

  `save`(filename=None, dirpath='.', checkpoint=True)

    Save the best torch model.

    Parameters

      - filename (str, optional) – name of the model. The default is “model.pt”.

      - dirpath (str, optional) – path to the desired foldre location. The default is “.”.

    Returns

      None.

  abstract `train`(iterator, *args, **kwargs)

    Train one time the model on iterator data.

    Parameters

      iterator (Iterator) – iterator containing batch samples of data.

    Returns

      the performance and metrics of the training session.

    Return type

      dict

`nets.solver.solver.` `get_metrics`(gold_labels, predictions, labels=None)

  Compute metrics for given predictions. This method returns basic metrics as a dictionary.

  Parameters

    - gold_labels (Tensor) – true labels.

    - predictions (Tensor) – predicted labels.

    - labels (list, optional) – list of classes indices.

  Returns

    dict

# nets.data

## nets.data.batch

A `Batch` is a set of examples, usually normalized and scaled for faster predictions.

class `nets.data.batch.` `Batch`(example, batch_size)

  A `Batch``depends on a ``Dataset` and is made of `batch_size` `Example`.

## nets.data.dataset

class `nets.data.dataset.` `Dataset`(examples, fields)

  Abstract Dataset class. All dataset for machine learning purposes can inherits from this architecture, for convenience.

  classmethod `download`(root)

    Download and unzip a web archive (.zip, .gz, or .tgz).

    Parameters

      root (str) – Folder to download data to.

    Returns

      Path to extracted dataset.

    Return type

      string

## nets.data.example

class `nets.data.example.` `Example`

  Store a single training / testing example, and store it as an attribute. Highly inspired from PyTorch [example](https://github.com/pytorch/text/blob/master/torchtext/data/example.py)

  classmethod `fromlist`(values, fields)

    Add an example from a list of data with respect to the fields.

    Parameters

      - values – raw data

      - fields (tuple(string, Field)) – fields to preprocess the data on

    Returns

      None

class `nets.data.example.` `Field`(transform=None, dtype=None)

  A `Field` defines the data to process from a raw dataset. It will convert the data into a tensor. The data can be a string, integers, float etc. The data is meant to be preprocessed and the attribute `transform` handles the way the user want to process the raw data.

  `process`(value)

    Applies the transformation and changes the data’s type if necessary.

    Parameters

      value (Tensor) –

    Returns:

## nets.data.iterator

This modules defines how the data should be called from a datasets. This takes into account `shuffle` mode.

class `nets.data.iterator.` `Iterator`(dataset, batch_size=32, shuffle=False)

  An `Iterator` call the data in batches. These batches can be shuffled and normalized. Usually, you want to feed a model with these batches.

# nets.datasets

## nets.datasets.cifar

Load and preprocess the CIFAR-10 dataset.

class `nets.datasets.cifar.` `CIFAR10`(filepath, transform=None)

  CIFAR-10 dataset, available at https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz

  The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images, with the following classes:

  Label | Description
  --- | ---
  0 | airplane
  1 | automobile
  2 | bird
  3 | cat
  4 | deer
  5 | dog
  6 | frog
  7 | horse
  8 | ship
  9 | truck

  Note

  The dataset is divided into five training batches and one test batch, each with 10000 images. The test batch contains exactly 1000 randomly-selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.

  classmethod `splits`(root='.data', train='data_batch_', test='test_batch', **kwargs)

    Loads training, validation, and test partitions of the cifar10 dataset (https://www.cs.toronto.edu/~kriz/cifar.html). If the data is not already contained in `root` folder, it will download it.

    Parameters

      - root (string) – relative or absolute path of the dataset.

      - train (string) – training data path

      - test (string) – testing data path

    Returns

      training and testing datasets

    Return type

      tuple(Dataset)

## nets.datasets.mnist

Defines and pre-process the MNIST dataset. The data will be converted into `Tensor` objects.

class `nets.datasets.mnist.` `MNIST`(path_data, path_label, transform=None)

  Loads training, validation, and test partitions of the mnist dataset (http://yann.lecun.com/exdb/mnist/). If the data is not already contained in data_dir, it will try to download it.

  This dataset contains 60000 training examples, and 10000 test examples of handwritten digits in {0, …, 9} and corresponding labels. Each handwritten image has an “original” dimension of 28x28x1, and is stored row-wise as a string of 784x1 bytes. Pixel values are in range 0 to 255 (inclusive).

  Parameters

    - data_dir – String. Relative or absolute path of the dataset.

    - devel_size – Integer. Size of the development (validation) dataset partition.

  Returns

    float64 numpy array with shape [784, 60000-devel_size] with values in [0, 1]. Y_train: uint8 numpy array with shape [60000-devel_size]. Labels. X_devel: float64 numpy array with shape [784, devel_size] with values in [0, 1]. Y_devel: uint8 numpy array with shape [devel_size]. Labels. X_test: float64 numpy array with shape [784, 10000] with values in [0, 1]. Y_test: uint8 numpy array with shape [10000]. Labels.

  Return type

    X_train

  classmethod `splits`(root='.data', train_data='train-images-idx3-ubyte.gz', train_label='train-labels-idx1-ubyte.gz', test_data='t10k-images-idx3-ubyte.gz', test_label='t10k-labels-idx1-ubyte.gz', **kwargs)

    Loads training and test partitions of the [mnist dataset](https://www.cs.toronto.edu/~kriz/cifar.html). If the data is not already contained in the `root` folder, it will download it.

    Parameters

      root (str) – relative or absolute path of the dataset.

    Returns

      training and testing datasets

    Return type

      tuple(Dataset)

## nets.datasets.svhn

# nets.utils

## nets.utils.display

This modules defines basic function to render a simulation, like progress bar and statistics table.

`nets.utils.display.` `describe_stats`(state_dict)

  Describe and render a dictionary. Usually, this function is called on a `Solver` state dictionary, and merged with a progress bar.

  Parameters

    state_dict (dict) – the dictionary to showcase

  Returns

    the dictionary to render.

  Return type

    string

`nets.utils.display.` `get_time`(start_time, end_time)

  Get ellapsed time in minutes and seconds.

  Parameters

    - start_time (float) – strarting time

    - end_time (float) – ending time

  Returns

    elapsed time in minutes elapsed_secs (float): elapsed time in seconds.

  Return type

    elapsed_mins (float)

`nets.utils.display.` `progress_bar`(current_index, max_index, prefix=None, suffix=None, start_time=None)

  Display a progress bar and duration.

  Parameters

    - current_index (int) – current state index (or epoch number).

    - max_index (int) – maximal numbers of state.

    - prefix (str, optional) – prefix of the progress bar. The default is None.

    - suffix (str, optional) – suffix of the progress bar. The default is None.

    - start_time (float, optional) – starting time of the progress bar. If not None, it will display the time spent from the beginning to the current state. The default is None.

  Returns

    None. Display the progress bar in the console.

## nets.utils.errors

Defines custom errors.

exception `nets.utils.errors.` `BackwardCallError`

  This error is used when the call to `backward` method is not legit.

## nets.utils.functions

Utility functions.

`nets.utils.functions.` `append2dict`(dict1, *dicts)

  Append key values to another dict with the same keys.

  Args: dict1 (dict): dictionary where values will be added. dict2 (dict) dictionaries to extract values and append to another one.

  > This dictionary should have the same keys as dict1.

  Returns

    None

  Examples:

      >>> dict1 = {"key1": [], "key2": []}
      >>> dict2 = {"key1": 0, "key1": 1}
      >>> append2dict(dict1, dict2)
      >>> dict1
          {"key1": [0], "key2": [1]}

      >>> dict3 = {"key1": 2, "key1": 3}
      >>> dict4 = {"key1": 4, "key1": 5}
      >>> append2dict(dict1, dict3, dict4)
      >>> dict1
          {"key1": [0, 2, 4], "key2": [1, 3, 5]}

## Docs

Access comprehensive developer documentation for Nets

View Docs

## Tutorials

Get beginners tutorials and create state-of-the-art models

View Tutorials

## Resources

Check the GitHub page and contribute to the project

View GitHub

To analyze traffic and optimize your experience, we serve cookies on this site. By clicking or navigating, you agree to allow our usage of cookies. As the current maintainers of this site, Facebook’s Cookies Policy applies. Learn more, including about available controls: Cookies Policy.