Skip to content

Feat: credential and auth model (tokens, proxy, delagation) - #30

Open
Alexander-Auer wants to merge 70 commits into
DIRACGrid:mainfrom
Alexander-Auer:stack/credential-models
Open

Alexander-Auer wants to merge 70 commits into
DIRACGrid:mainfrom
Alexander-Auer:stack/credential-models

Conversation

@Alexander-Auer

@Alexander-Auer Alexander-Auer commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

feat: credentials (models, provider, freshness)

Part of #17. Follows #29 (merged).

Backends get their credentials from a provider that the consumer owns. Configuration never holds a secret.

How a consumer uses it

The consumer supplies the credentials: it creates a CredentialProvider, passes it to get_backend, and closes it when done. interCEde only decides when to ask, never where credentials come from, and never closes the provider.

A provider is any object with an async get method and async with support:

from intercede import registry
from intercede.auth import BearerToken, CredentialRequirements, CredentialSet

class MyTokenProvider:
    async def get(self, requirements: CredentialRequirements) -> CredentialSet:
        # requirements.resource is the endpoint asking; requirements.scopes the operation's scopes
        token = await my_token_service.mint(audience=requirements.resource)
        return CredentialSet([BearerToken(value=token.value, expires_at=token.expires_at)])

    async def __aenter__(self): return self
    async def __aexit__(self, *exc): ...

async with MyTokenProvider() as credentials:
    async with registry.get_backend(config, credentials) as ce:
        await ce.submit(spec, copies)

For fixed credentials, for example a key read from a file, the consumer can use StaticCredentials instead of writing a provider:

async with StaticCredentials([SSHPrivateKey(pem=key_bytes)]) as credentials:
    ...

One provider can be shared by every backend the consumer drives.

What's added

  • intercede.auth (public):
    • Credential types: BearerToken, X509Proxy, SSHPrivateKey, SSHPassword. Secrets never appear in repr, JSON or error messages.
    • CredentialSet, CredentialRequirements and the CredentialProvider protocol.
    • StaticCredentials.
  • Requirements (internal): each backend works out what to ask for from its config's auth field:
    • ARC and HTCondor-CE ask for a token, and optionally a proxy, for their endpoint.
    • SSH asks for a key or a password for user@host:port.
    • Local, cloud and SSH through an agent ask for nothing.
  • Freshness (internal): before each operation, a backend asks the provider again only when a credential is missing or expires within 5 minutes. Any provider failure becomes an AuthenticationError for the whole operation, never a per-job failure.
  • Registry: get_backend(config, credentials) passes the provider to backends and transports. Anything that isn't a provider raises a ConfigurationError. Building a backend never calls the provider.

Breaking change

registry.get_backend(config) now takes a second argument, credentials.

Open points

  • There is no credential type for cloud API keys yet.
  • Should a provider be able to change the 5-minute margin?

Tests

157 tests pass on Python 3.11 and 3.13, with 99% coverage. test_credentials.py imports intercede._backends.credentials directly, because no real backend uses it yet.

alexander and others added 30 commits October 2, 2026 16:54
* feat: Models the core contract needs

* Feat: Protocols and Exceptions added for the core contract

* Feat: Private protocols for the BatchBackend

* feat: a fake backend to run unit tests against

* fix: some code cleanup in the helper functions

* feat: setup for the unit test suite

* chore: style fix

* chore: styling, unit test fixture

* chore: single letter var `h` refering to hanlde changed to `handle`

* fix: unused parameter in `test_injected_failure_is_reported_per_job()` removed

* feat: Models the core contract needs

* Feat: Protocols and Exceptions added for the core contract

* Feat: Private protocols for the BatchBackend

* feat: a fake backend to run unit tests against

* fix: some code cleanup in the helper functions

* feat: setup for the unit test suite

* chore: style fix

* chore: styling, unit test fixture

* chore: single letter var `h` refering to hanlde changed to `handle`

* fix: unused parameter in `test_injected_failure_is_reported_per_job()` removed

* fix: made the OutputSpec optional as it is not suppoerted by all and a consumer should choose to use it or not

---------

Co-authored-by: alexander <alexander.auer@cern.ch>
alexander added 27 commits October 23, 2026 16:18
Keep secrets never showing, the set and requirement rules, and every way a provider can fail; drop tests of pydantic's own behaviour.
Move the API reference to docs/user/reference/api.md, link the user and developer documentation from the index, and stop referring to the ADRs there.
- Google-style docstrings for the public and batch protocols, with arguments, returns and raises.
- get_diagnostics takes a filename (a file in the job's working directory; None reads stdout) instead of a generic name; Diagnostics.filename matches.
- An input name may appear once in a SubmissionSpec: two sources for one file are refused when the spec is built.
- A copy with a secret named like an input is refused at submit with SpecificationRejectedError; SubmissionSpec.clashes_with() lets every backend apply the same rule.
- No references to the ADRs in code or docstrings: they are not part of the source and can be superseded.
Follows the review of the core contract: the ADRs are not part of the source and can be superseded. The registry's docstrings also say what they return.
# Conflicts:
#	tests/unit/test_fake_backend.py
#	tests/unit/test_models.py
# Conflicts:
#	src/intercede/_backends/config.py
Follows the review of the core contract: the ADRs are not part of the source and can be superseded. The provider protocol's docstring also gets the Google-style sections.
- Google-style docstrings with Args, Returns and Raises for resolve() and every registry and configuration helper, and docstrings for the settings classes; ruff now checks docstrings against the Google convention.
- Remove vm_max_lifetime_s: removing VMs after a lifetime is the consumer's policy, not interCEde's.
- Import Callable from collections.abc, like the rest of the code.
A verb says that the call builds something, and get_backend() no longer reads like the backend itself.
ARCConfig, HTCondorCEConfig, CloudConfig, BatchLocalConfig and BatchSSHConfig
replace the flat BackendConfig, which is now their union, chosen by kind (and
by transport for batch). A consumer can still pass a plain mapping, such as a
TOML table.

- Every field is declared once, on the model of its kind: the internal
  settings classes, the hand-kept field lists and resolve() are gone, so a
  configuration is validated once instead of twice.
- validate_config() checks a configuration and returns its model;
  registry.get_backend() uses it. A model is checked again, so one changed
  with model_copy() cannot carry fields of another transport to a backend.
- Errors never repeat a value they were given, since it may be a secret, and
  name the expected fields when a field does not belong to the kind.
- The configuration tests use the public API only.

The SSH auth mode is now one value (auth = "key") instead of a list.
griffe-pydantic renders every field of each configuration model, inherited
ones included, with its type, default and description, so the reference cannot
drift from the code like the hand-written table could.
The merge of stack/registry kept a helper and a section header from the old test_config.py; the helper referred to resolve() and BackendConfig.model_validate(), which no longer exist.
Brings in the per-kind configuration models of stack/registry, and moves the
credential code onto them:

- credential_requirements(config, scopes) in _backends/credentials.py replaces
  the methods on the removed internal settings classes, with the same rules:
  a CE asks for its endpoint with the configured token and/or proxy, SSH for
  user@host:port unless an SSH agent holds the key, and a local batch system
  for nothing.
- registry.get_backend(config, credentials) checks the provider and passes it
  to the parts that authenticate: single-class backends and transports.
- The credential requirement tests move from test_config.py to
  test_credentials.py and build their configurations through validate_config().
Two tests had moved without changing, which only made the diff harder to read.
# Conflicts:
#	src/intercede/_backends/fake.py
#	src/intercede/registry.py
#	tests/unit/conftest.py
#	tests/unit/test_registry.py
A value given by mistake, such as a misspelled field or plain data in a
CredentialSet, may be the secret itself. Validation errors now hide their
input, as the configuration models already do.
The message comes from the consumer's code and may hold a token or a
secret URL. The AuthenticationError names only its type; the exception
itself is still available as __cause__.
The fields were described in comments, which the generated reference does
not show. They now use attribute docstrings, as the configuration models do.
Name every argument and the return value, and drop the blank lines
before closing quotes.
@Alexander-Auer
Alexander-Auer requested a review from aldbr October 27, 2026 16:17
@Alexander-Auer
Alexander-Auer marked this pull request as ready for review October 27, 2026 16:18
Alexander-Auer pushed a commit to Alexander-Auer/intercede that referenced this pull request Oct 27, 2026
… into feat/HTCondorCEBackend

Replaces the earlier versions this branch was built on with what was merged
upstream (DIRACGrid#27, DIRACGrid#28, DIRACGrid#29) and the credentials under review (DIRACGrid#30). The HTCondor-CE
schedd setting moves onto HTCondorCEConfig.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant