Metadata-Version: 2.1
Name: swh.loader.core
Version: 5.11.0
Summary: Software Heritage Base Loader
Home-page: https://forge.softwareheritage.org/diffusion/DLDBASE
Author: Software Heritage developers
Author-email: swh-devel@inria.fr
Project-URL: Bug Reports, https://forge.softwareheritage.org/maniphest
Project-URL: Funding, https://www.softwareheritage.org/donate
Project-URL: Source, https://forge.softwareheritage.org/source/swh-loader-core
Project-URL: Documentation, https://docs.softwareheritage.org/devel/swh-loader-core/
Classifier: Programming Language :: Python :: 3
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: AUTHORS
Requires-Dist: psutil
Requires-Dist: requests
Requires-Dist: iso8601
Requires-Dist: pkginfo
Requires-Dist: python-debian
Requires-Dist: python-dateutil
Requires-Dist: typing-extensions
Requires-Dist: toml
Requires-Dist: packaging
Requires-Dist: swh.core>=2.23.0
Requires-Dist: swh.model>=6.5.1
Requires-Dist: swh.objstorage>=0.2.2
Requires-Dist: swh.scheduler>=0.4.0
Requires-Dist: swh.storage>=1.18.0
Provides-Extra: testing
Requires-Dist: pytest; extra == "testing"
Requires-Dist: pytest-mock; extra == "testing"
Requires-Dist: requests_mock; extra == "testing"
Requires-Dist: urllib3<2.0.0; extra == "testing"
Requires-Dist: swh-core[testing]; extra == "testing"
Requires-Dist: swh-scheduler[testing]>=0.5.0; extra == "testing"
Requires-Dist: swh-storage[testing]>=1.8.0; extra == "testing"
Requires-Dist: types-click; extra == "testing"
Requires-Dist: types-python-dateutil; extra == "testing"
Requires-Dist: types-pyyaml; extra == "testing"
Requires-Dist: types-requests; extra == "testing"

Software Heritage - Loader foundations
======================================

The Software Heritage Loader Core is a low-level loading utilities and
helpers used by :term:`loaders <loader>`.

The main entry points are classes:
- :class:`swh.loader.core.loader.BaseLoader` for VCS loaders (e.g. svn)
- :class:`swh.loader.core.loader.DVCSLoader` for DVCS loaders (e.g. git, ...)
- :class:`swh.loader.core.loader.ContentLoader` for Content loader
- :class:`swh.loader.core.loader.DirectoryLoader` for Directory loader
- :class:`swh.loader.package.loader.PackageLoader` for Package loaders (e.g. PyPI, Npm, ...)
- ...

Package loaders
---------------

This package also implements many package loaders directly, out of convenience,
as they usually are quite similar and each fits in a single file.

They all roughly follow these steps, explained in the
:py:meth:`swh.loader.package.loader.PackageLoader.load` documentation.
See the :ref:`package-loader-tutorial` for details.

VCS loaders
-----------

Unlike package loaders, VCS loaders remain in separate packages,
as they often need more advanced conversions and very VCS-specific operations.

This usually involves getting the branches of a repository and recursively loading
revisions in the history (and directory trees in these revisions),
until a known revision is found
