Repository navigation
Conversation
added 30 commits
November 17, 2026 13:02
Brings up PanDAWMS/panda-compose (sibling checkout, compose-file merge, same staging order as BNLNPPS/swf-testbed), builds a snakemake worker image from this PR's plugin code, runs a one-job snakemake workflow through --executor panda, and asserts the task reaches 'done' and the output DID is registered in Rucio on MOCK-POSIX. Trigger: push/PR to master + workflow_dispatch (panda_compose_repo/ref and snakemake_version overridable).
…compose is a sibling (../panda-compose)
…ing-dir was ambiguous)
…and workflow run; drop working-directory
…ts fetched pull/N/merge, missing .env.example)
…mpty on pull_request events)
…heredoc); write patch script as separate file, COPY before RUN
…h_timeout.py/COPY plugin resolve
… plugin; use default context
…PACE} expression syntax
…WORKSPACE} (compose does not expand env syntax)
…mq, wait stable, then up -d (all services); drop nonexistent idds/ruciodb service names
…ssingClientParameter auth_host
…cols for debugging prefix KeyError
…e-rucio-timeout instead
…patch handles the cast; avoids the str-timeout crash entirely)
…rting the plugin loads RUCIO_CONFIG at import time)
…ible with the 41.x client (KeyError 'prefix' on upload)
… Docker Hub; use the exact 41.x tag)
added 23 commits
November 17, 2026 22:36
…aks against 41.x server)
…protocol selection, live UploadClient attempt
…ally) — latest may differ in upload path
…'prefix' KeyError source)
…p (rucio server bind-mounts it; posix.put falls into the OSError branch that reads self.rse['prefix'] -> KeyError when the dir is missing)
…missing grid proxy (server runs token_authType=None)
…er.veprbl..., rejected by the server as invalid outDS)
…> (Snakefile re-parses on the worker; uuid4 gave host/worker different names)
…archive on exit (rucio plugin raises NotImplementedError on delete, failing the step after the task completed)
…moves the workflow-sources archive on the rucio provider (delete raises NotImplementedError and failed the step)
…e via snakemake --deploy-sources, so the host must not upload/remove it on rucio (delete raises NotImplementedError and failed the run after 100% done)
…al_files=True, which is the actual gate for source-archive upload/remove)
… no-op the rucio plugin's remove() on host+worker instead — upstream raises NotImplementedError on delete, which failed the run after 100% done; worker-side job failure in 44386961232 was caused by the missing source archive
…ubmitter activity is invisible in the current tail
…ker-submitter lines from harvester container
…rns Python repr otherwise); dump harvester-worker container logs + /tmp/harvester_output
…es a type=collection search that never matches file DIDs)
…ate keys not values)
…io storage plugin patch into committed ci/ files - remove one-shot debug steps/dumps (workspace layout, RSE protocol dump, UploadClient probe that also polluted mock:snakemake-uploads with ci-probe-runner.txt, redundant always()-log commands, dead TIMEOUT env, no-op RUCIO_CLIENT_VERSION re-export) - ci/worker.Dockerfile replaces the runtime-heredoc Dockerfile+patch script; storage plugin version is now a build-arg pinned from workflow env - ci/patch_rucio_storage_plugin.py replaces the two drifting copies of the timeout int-cast + remove() no-op patches (host and worker now share it); idempotent, asserts its anchors, fails loudly on plugin version drift
veprbl
commented
Nov 20, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings up PanDAWMS/panda-compose (sibling checkout, compose-file merge, same staging order as BNLNPPS/swf-testbed), builds a snakemake worker image from this PR's plugin code, runs a one-job snakemake workflow through --executor panda, and asserts the task reaches 'done' and the output DID is registered in Rucio on MOCK-POSIX.
Trigger: push/PR to master + workflow_dispatch (panda_compose_repo/ref and snakemake_version overridable).