# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['deepstruct']

package_data = \
{'': ['*']}

install_requires = \
['importlib-metadata>=1.5.0,<2.0.0',
 'importlib-resources>=1.4.0,<2.0.0',
 'jsonschema>=3.2.0,<4.0.0',
 'networkx>=2.0',
 'semantic_version>=2.8.4,<3.0.0',
 'torch>=1.4']

setup_kwargs = {
    'name': 'deepstruct',
    'version': '0.5.0',
    'description': '',
    'long_description': '# deepstruct - tools for neural network graph topology analysis ![Tests](https://github.com/innvariant/pypaddle/workflows/Tests/badge.svg)  [![Maintenance](https://img.shields.io/badge/Maintained%3F-yes-green.svg)](https://GitHub.com/Naereen/StrapDown.js/graphs/commit-activity) [![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/) [![Python 3.7](https://img.shields.io/badge/python-3.7-blue.svg)](https://www.python.org/downloads/release/python-370/) [![Python 3.6](https://img.shields.io/badge/python-3.8-blue.svg)](https://www.python.org/downloads/release/python-380/)\nTools for fusing machine learning and graph theory.\nWe are fascinated with the interplay of end-to-end learnable, locally restricted models and their graph theoretical properties.\nSearching for evidence of the structural prior hypothesis.\nInterested in pruning, neural architecture search or learning theory in general?\n\n**Note** we are currently renaming *pypaddle* to *deepstruct* due to naming collisions and better semantical wording.\n\n## Installation\nVia *poetry* (**recommended** for projects) using PyPi:\n``poetry add deepstruct``\n\nDirectly with *pip* from PyPi:\n```bash\npip install deepstruct\n```\n\nVia *conda* in your *environment.yml* (recommended for reproducible experiments):\n```yaml\nname: exp01\nchannels:\n- defaults\ndependencies:\n- pip>=20\n- pip:\n    - deepstruct\n```\n\nFrom public GitHub:\n```bash\npip install --upgrade git+ssh://git@github.com:innvariant/pypaddle.git\n```\n\n## Models\n- *pypaddle.sparse.MaskableModule*: pytorch modules that contain explicit masks to enforce (mostly zero-ordered) structure\n- *pypaddle.sparse.MaskedLinearLayer*: pytorch module with a simple linear layer extended with masking capability.\nSuitable if you want to have linear-layers on which to enforce masks which could be obtained through pruning, regularization or other other search techniques.\n- *pypaddle.sparse.MaskedDeepFFN*: feed-forward neural network with any width and depth and easy-to-use masks.\nSuitable for simple and canonical pruning research on zero-ordered structure\n- *pypaddle.sparse.MaskedDeepDAN*: feed-forward neural network with skip-layer connections based on any directed acyclic network.\nSuitable for arbitrary structures on zero-order and on that level most flexible but also computationally expensive.\n- *pypaddle.sparse.DeepCellDAN*: complex module based on a directed acyclic network and custom cells on third-order structures.\nSuitable for large-scale neural architecture search\n\n## What is the orders of structure?\n- zero-th order: weight-level\n- first order: kernel-level (filter, channel, blocks, cells)\n- second order: layers\n\nThere is various evidence across empirical machine learning studies that the way artificial neural networks are structurally connected has a (minor?) influence on performance metrics such as the accuracy or probably even on more complex concepts such as adversarial robustness.\nWhat do we mean by "structure"?\nWe define structure over graph theoretic properties given a computational graph with very restricted non-linearities.\nThis includes all major neural network definitions and lets us study them from the perspective of their *representation* and their *structure*.\nIn a probabilistic sense, one can interprete structure as a prior to the model and despite single-layered wide networks are universal function approximators we follow the hypothesis that given certain structural priors we can find models with better properties.\n\n\n\n## Sparse Neural Network implementations\n![Sparse Network Connectivity on zeroth order with a masked deep feed-forward neural network](doc/masked-deep-ffn.png)\n![Sparse Network Connectivity on zeroth order with a masked deep neural network with skip-layer connections](doc/masked-deep-dan.png)\n![Sparse Network Connectivity on second order with a masked deep cell-based neural network](doc/masked-deep-cell-dan.png)\n\n**What\'s contained in pypaddle?**\n- ready-to-use models in pytorch for learning instances on common (supervised/unsupervised) datasets from which a structural analysis is possible\n- model-to-graph transformations for studying models from a graph-theoretic perspective\n\nBefore considering implementations, one should have a look on possible representations of Sparse Neural Networks.\nIn case of feed-forward neural networks (FFNs) the network can be represented as a list of weight matrices.\nEach weight matrix represents the connections from one layer to the next.\nHaving a network without some connections then simply means setting entries in those matrices to zero.\nRemoving a particular neuron means setting all entries representing its incoming connections to zero.\n\nHowever, sparsity can be employed on various levels of a general artificial neural network.\nZero order sparsity would remove single weights (representing connections) from the network.\nFirst order sparsity removes groups of weights within one dimension of a matrix from the network.\nSparsity can be employed on connection-, weight-, block-, channel-, cell-level and so on.\nImplementations respecting the areas for sparsification can have drastical differences.\nThus there are various ways for implementing Sparse Neural Networks.\n\n### Feed-forward Neural Network with sparsity\nThe simplest implementation is probably one which provides multiple layers with binary masks for each weight matrix.\nIt doesn\'t consider any skip-layer connections.\nEach layer is then connected to only the following one.\n```python\nimport pypaddle.sparse\n\nmnist_model = pypaddle.sparse.MaskedDeepFFN((1, 28, 28), 10, [100, 100])\n```\n\n\n```python\nimport pypaddle.sparse\n\nstructure  = pypaddle.sparse.CachedLayeredGraph()\n# .. add nodes & edges to the networkx graph structure\n\n# Build a neural network classifier with 784 input and 10 output neurons and the given structure\nmodel = pypaddle.sparse.MaskedDeepDAN(784, 10, structure)\nmodel.apply_mask()  # Apply the mask on the weights (hard, not undoable)\nmodel.recompute_mask()  # Use weight magnitude to recompute the mask from the network\npruned_structure = model.generate_structure()  # Get the structure -- a networkx graph -- based on the current mask\n\nnew_model = pypaddle.sparse.MaskedDeepDAN(784, 10, pruned_structure)\n```\n```python\nimport pypaddle.sparse\n\nmodel = pypaddle.sparse.MaskedDeepFFN(784, 10, [100, 100])\n# .. train model\nmodel.generate_structure()  # a networkx graph\n```\n',
    'author': 'Julian Stier',
    'author_email': 'julian.stier@uni-passau.de',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://github.com/innvariant/deepstruct',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'python_requires': '>=3.6.1',
}


setup(**setup_kwargs)
