This issue is two-pronged:
- File I/O for benchmarks is implemented on a per-benchmark-file basis, so it's difficult to know at a glance which files are currently being used for benchmarks. This also promotes copy-pasting of I/O code, so consolidating the I/O would be in line with the DRY principle.
- Partly as a consequence of (1), the fragmented I/O regime basically makes ASV retrieve the needed files at every single benchmark run. For benchmark suites on shared files, this can be enormously expensive.
Instead, I'm proposing that we have one benchmark helper file to support all needed file I/O, which will consolidate I/O code and enable single-retrieval for batches of benchmarks.
This issue is two-pronged:
Instead, I'm proposing that we have one benchmark helper file to support all needed file I/O, which will consolidate I/O code and enable single-retrieval for batches of benchmarks.