[tdruez]

Issues

• Issue: code-genetics: Design and implement heuristics to detect ultimate origin #1930

Changes

• Add the shared copy, embedded archive, single tool and license header cases to the origin test data set
• Ignore the snippet origin matches covering fewer than 20 lines, such as a common license header
• Merge the MatchCode.io snippets of the same origin file, previously dropped by the unique constraint
• Fetch the path of the MatchCode.io matched files in their package from PurlDB, by batches of 1000 sha1
• Log a missing PurlDB configuration in the find_origins pipeline
• Add the containment rule: propose the candidate vendored by all the other candidates
• Add the highest-similarity rule: propose the candidate more similar than all the others by at least 20 points
• Add the earliest-release rule: propose the candidate released before all the others
• Add the preferred-version rule: propose the clearly most similar version, or else the only release
• Find the scan package of an origin despite PURL qualifiers
• Stop collecting the PurlDB results on a failed next page request
• Document the origin determination rules

Notes

Reputation is not covered: the match data has no reliable popularity signal, and the PURL type alone would favor a published copy over an original only hosted on GitHub, such as a Python binding embedding a C library.