Thread limit introspection API, part 1: API scope

[itamarst]

Fixes #211.

Add info to determine the scope of the thread-limit-setting API (thread local, process-wide, unknown). Introspecting the semantics of the thread pool are going to be a separate PR, assuming that can be done reasonably.

[ogrisel]

I find it a bit disturbing that calling threadpoolctl.threadpool_info can temporarily mutate the libraries global state in a non-thread safe way.

The alternative would be to hardcode expected scopes in LibController for all known libraries (and return "unknown" otherwise) and only call _determine_api_scope in tests or by passing a non-default kwarg value: threadpool_ctl.threadpool_info(api_scopes="effective") (and api_scopes="expected" would be the default) for instance.

[ogrisel]

Another interesting finding by looking at the CI results: BLIS is current_thread scoped contrary to other BLAS implementations.

[itamarst]

I think it's useful to have it in python -m threadpoolctl for debugging and user bug reports, so maybe have the side-effect-y introspection gated behind a keyword argument to info()? Or it can just be some add-on processing that the command-line version does using lower-level functions.

[ogrisel]

I am fine with introducing an extra CLI flag to enable the extra inspection work.

[itamarst]

My feeling is:

• As you said it shouldn't be on by default in the Python API because of risk of side-effects.
• It should be on by default in the CLI because it won't actually run any user code, it's just for debugging, so may as well give all the available info.

[itamarst]

++ /home/runner/miniconda3/bin/conda shell.posix reactivate
Traceback (most recent call last):
  File "/home/runner/miniconda3/bin/conda", line 12, in <module>
    from conda.cli import main
ModuleNotFoundError: No module named 'conda'

That's a fun error.

Probably due to conda/conda#16327

[itamarst]

I addressed or replied to all review comments. CI is broken for now, will try again later.

[itamarst]

I wonder why a filter of {'internal_api': 'openblas', 'threading_layer': 'pthreads'} is returning a BLISController... will get to that next week, I guess.

[itamarst]

I wonder why a filter of {'internal_api': 'openblas', 'threading_layer': 'pthreads'} is returning a BLISController... will get to that next week, I guess.

Ah, I misunderstood the logic, it's OR not AND. Will fix.

[itamarst]

Ready for another round of review, I think.

[ogrisel]

Let's add a changelog entry for this. Otherwise, LGTM.

Any opinion @jeremiedbb?

[ogrisel]

The CI failures seem to be unrelated I think. If so we can fix them in a dedicated PR.

[itamarst]

The CI failures seem to be unrelated I think. If so we can fix them in a dedicated PR.

I believe there's an outstanding PR that fixes one of them, yes.

[itamarst]

Added changelog and explanatory comment.