Skip to content

feat: add remote pipeline storage and recovery - #2748

Open
Mbeaulne wants to merge 4 commits into
mb/remote-pipelines-migrationfrom
mb/remote-pipelines-store
Open

Mbeaulne wants to merge 4 commits into
mb/remote-pipelines-migrationfrom
mb/remote-pipelines-store

Conversation

@Mbeaulne

@Mbeaulne Mbeaulne commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Description

Storage layer of the remote-pipeline stack. Base: #2747.

  • Add remote file/store implementations and storage-service integration, scoped to the current backend and account.
  • Preserve recovery drafts, concurrent saves, stable-identity retries, ownership checks, confirmed migration, and deletion.
  • Add cursor-page and complete-catalog summary loading without definition requests. Keep pending first uploads separate from saved remote entries, attach matching recovery by identity, and expose cached remote entries for outages without removing recovery from other pages.

The deployment flag remains hard-disabled here. Tests instantiate remote storage explicitly; environment-based activation is in #2739, after all callers are wired. Connected filesystem and Google Drive folders retain their existing drivers.

Review focus: preserving the newest draft, preventing duplicate uploads, and keeping account-scoped list results separate from pending or unfinished migrations.

Related pull requests

#2746#2747#2748#2749#2750#2751#2739#2780#2783

How to test

Passed at this layer: 68 API/store tests with the command below. Coverage includes cursor pages, summary-only requests, recovery identity and account isolation, pending uploads, cached entries, migration, ownership, and deletion.

pnpm exec vitest run src/services/cloudPipelineService.test.ts src/services/pipelineStorage/RemotePipelineStore.test.ts

For a manual check, the normal UI must remain on local storage even if VITE_REMOTE_PIPELINES_ENABLED=true is present at this layer. Integrated remote browser validation is tracked in #2739.

Change Size

Relative to mb/remote-pipelines-migration for this PR only. Changed lines = additions + deletions.

Category Files Added Removed Changed
Application code 6 +851 -9 860
Tests 2 +1186 -0 1186
Documentation / configuration 1 +1 -0 1
Total 9 +2038 -9 2047

Application code excludes test files. Tests include unit, component, and browser test source; test-runner configuration, CI, package scripts, compiler configuration, and documentation are counted separately.

@github-actions

github-actions Bot commented Sep 18, 2026

Copy link
Copy Markdown

🎩 Preview

A preview build has been created at: mb/remote-pipelines-store/fc51f9d

@Mbeaulne
Mbeaulne added this pull request to stack #2752 September 18, 2026 19:28
private manageFile(file: PipelineFile): PipelineFile {
if (this.canMigrate(file))
file.resolveRedirect = () =>
this.remote?.resolveLocal(file.id) ?? Promise.resolve(undefined);
@Mbeaulne
Mbeaulne force-pushed the mb/remote-pipelines-store branch from bad399b to fc51f9d Compare September 22, 2026 19:05

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant