purl-next: Create decentralized, distributed PURL metadata collection #655

Create decentralized, distributed PURL metadata collection using scripts that can work as CI/CD jobs.

• Today data collection is mostly centralized and runs through the PurlDB API using either a batch importer (say, all of Maven Central at once), or an on-demand importer (get only metadata for this PURL), or a watch process (what for new PURLs for this PURL)

• The goal of this task is to evolve data collection so that these can run in a decentralized fashion and output FederatedCode git data and that PurlDB can consume these pre-collected data instead of doing the collection directly

• We support many ecosystems. Here the focus is to enable this support for about 12 ecosystems.

Here is the list of all candidate ecosystems:

• Collect PURLs from pypi - purl-next: Collect PURLs for PyPI ecosystem #662
• Collect PURLs from cargo - purl-next: Collect PURLs for Rust ecosystem #666
• Collect PURLs from debian - purl-next: Collect PURLs for Debian ecosystem #664
• Collect PURLs from maven - purl-next: Collect PURLs for Maven ecosystem #660
• Collect PURLs from nuget - purl-next: Collect PURLs for NuGet ecosystem #667
• Collect PURLs from npm - purl-next: Collect PURLs for npm ecosystem #661
• Collect PURLs from Alpine - purl-next: Collect PURLs for Alpine ecosystem #665
• Collect PURLs from Conan purl-next: Collect PURLs for conan C/C++ packages #680
• Collect PURLs from cran purl-next: Collect PURLs from cran for R packages #686
• Collect PURLs from packagist purl-next: Collect PURLs from packagist for Composer PHP packages #687
• Collect PURLs from swift - purl-next: Collect PURLs from swift #702
• Collect PURLs from cpan purl-next: Collect PURLs for CPAN Perl packages #685

And beyond these, see:

• More decentralized, distributed PURL metadata collection #727