Skip to content

feat(python-notebook-migration): add notebook migration orchestration service - #5262

Merged
mengw15 merged 34 commits into
apache:mainfrom
zyratlo:migration-tool-notebook-service
Jul 20, 2026
Merged

feat(python-notebook-migration): add notebook migration orchestration service#5262
mengw15 merged 34 commits into
apache:mainfrom
zyratlo:migration-tool-notebook-service

Conversation

@zyratlo

@zyratlo zyratlo commented May 28, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Introduces NotebookMigrationService, the frontend orchestration service that sits between the migration-tool UI and the lower layers: the LLM client (migration-tool-llm-client) and the backend notebook-migration microservice (migration-tool-backend-notebook-migration-service).

notebook-migration.service.ts

  • getAvailableModels()GET /api/models against the existing LiteLLM proxy, returns the model dropdown options.
  • sendToAIGenerateWorkflow(notebook, modelType) — drives the full NotebookMigrationLLM lifecycle (initialize → verify connection → convert → close in finally) and returns { workflowContent, mappingContent }.
  • sendNotebookToJupyter(notebookData)POST /api/notebook-migration/set-notebook; surfaces a NotificationService toast on success and failure; returns 1 / 0.
  • getJupyterURL(), getJupyterIframeURL() — calls the matching microservice endpoints to retrieve URLs to embed.
  • storeNotebookAndMapping(wid, vid, mappingContent, notebookContent)POST /api/notebook-migration/store-notebook-and-mapping; returns the HttpClient observable directly so callers can compose with switchMap.
  • Mapping cache — small in-memory dictionary { [key: string]: MappingContent } keyed by mapping_wid_<workflowId>, with hasMapping, getMapping, setMapping, deleteMapping.

notebook-migration.service.spec.ts

  • getAvailableModels: maps the LiteLLM data[].id array correctly; falls back to an empty array on HTTP error.
  • sendNotebookToJupyter: success → returns 1; error → returns 0 and toasts.
  • getJupyterURL / getJupyterIframeURL: success → returns the URL; non-OK response or thrown error → returns null.
  • Mapping cache: setMapping then getMapping round-trips; deleteMapping removes the entry.
  • storeNotebookAndMapping: makes the expected POST to the persistence endpoint.

Any related issues, documentation, discussions?

Closes #5261
Parent issue #4301

How was this PR tested?

The new notebook-migration.service.spec.ts adds HttpClientTestingModule-driven test cases

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 4.7)

@github-actions github-actions Bot added the frontend Changes related to the frontend GUI label May 28, 2026
@Yicong-Huang Yicong-Huang changed the title feat(python-notebook-migration, frontend): add notebook migration orchestration service feat(python-notebook-migration): add notebook migration orchestration service May 28, 2026
@codecov-commenter

codecov-commenter commented Jun 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.20253% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.03%. Comparing base (c543d77) to head (e6e65be).

Files with missing lines Patch % Lines
...kspace/service/notebook-migration/migration-llm.ts 33.33% 2 Missing ⚠️
...e/notebook-migration/notebook-migration.service.ts 98.68% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #5262      +/-   ##
============================================
+ Coverage     73.99%   74.03%   +0.03%     
- Complexity     3435     3439       +4     
============================================
  Files          1157     1158       +1     
  Lines         45621    45694      +73     
  Branches       5031     5043      +12     
============================================
+ Hits          33757    33828      +71     
- Misses        10184    10187       +3     
+ Partials       1680     1679       -1     
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø) Carriedforward from 7bd8dc1
agent-service 76.76% <ø> (ø) Carriedforward from 7bd8dc1
amber 66.82% <ø> (+0.02%) ⬆️ Carriedforward from 7bd8dc1
computing-unit-managing-service 17.72% <ø> (ø) Carriedforward from 7bd8dc1
config-service 66.66% <ø> (ø) Carriedforward from 7bd8dc1
file-service 66.80% <ø> (ø) Carriedforward from 7bd8dc1
frontend 77.18% <96.20%> (+0.05%) ⬆️
notebook-migration-service 78.94% <ø> (ø) Carriedforward from 7bd8dc1
pyamber 91.79% <ø> (ø) Carriedforward from 7bd8dc1
workflow-compiling-service 55.14% <ø> (ø) Carriedforward from 7bd8dc1

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@zyratlo
zyratlo marked this pull request as draft June 5, 2026 21:28
@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@zyratlo
zyratlo marked this pull request as ready for review June 25, 2026 23:18
@zyratlo

zyratlo commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

/request-review @mengw15

@github-actions
github-actions Bot requested a review from mengw15 June 25, 2026 23:18
@mengw15
mengw15 requested a review from Copilot June 26, 2026 08:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

@zyratlo

zyratlo commented Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Test-isolation fix touching migration-llm.* (from the already-merged #5260), added in commit 5797a27.

migration-llm.spec.ts stubs the LLM transport by mocking the ai module (vi.mock("ai", …)). That mock is unreliable under the Angular unit-test (Vitest) builder once another spec loads the real ai module, and notebook-migration.service.spec.ts (added in this PR) does, transitively: it imports NotebookMigrationServiceNotebookMigrationLLMai. When the real module wins (scheduling-dependent, green on ubuntu, hung on macOS), generateText is unmocked and convertNotebookToWorkflow times out on a real network call. It was also emitting the "vi.mock not at top level" warning.

Fix: route the SDK call through a callModel seam on NotebookMigrationLLM and stub that in the spec, dropping the ai/@ai-sdk/openai module mocks entirely. Stubbing at the class boundary is per-instance and immune to module-load ordering, so it's deterministic across OSes.

Since #5260 is already merged, this fix necessarily modifies that landed code, so it's part of this PR's diff. I applied the fix here on #5262 to get CI green. Flagging it because it's outside this PR's nominal scope (the notebook-migration orchestration service). @mengw15 do you want it kept here, or pulled into a dedicated follow-up PR against migration-llm.*? I lean toward keeping it here — this PR is what surfaces the failure, and CI needs it green — but happy to split it if you'd rather isolate the change.

@mengw15 mengw15 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments

@mengw15

mengw15 commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Test-isolation fix touching migration-llm.* (from the already-merged #5260), added in commit 5797a27.

migration-llm.spec.ts stubs the LLM transport by mocking the ai module (vi.mock("ai", …)). That mock is unreliable under the Angular unit-test (Vitest) builder once another spec loads the real ai module, and notebook-migration.service.spec.ts (added in this PR) does, transitively: it imports NotebookMigrationServiceNotebookMigrationLLMai. When the real module wins (scheduling-dependent, green on ubuntu, hung on macOS), generateText is unmocked and convertNotebookToWorkflow times out on a real network call. It was also emitting the "vi.mock not at top level" warning.

Fix: route the SDK call through a callModel seam on NotebookMigrationLLM and stub that in the spec, dropping the ai/@ai-sdk/openai module mocks entirely. Stubbing at the class boundary is per-instance and immune to module-load ordering, so it's deterministic across OSes.

Since #5260 is already merged, this fix necessarily modifies that landed code, so it's part of this PR's diff. I applied the fix here on #5262 to get CI green. Flagging it because it's outside this PR's nominal scope (the notebook-migration orchestration service). @mengw15 do you want it kept here, or pulled into a dedicated follow-up PR against migration-llm.*? I lean toward keeping it here — this PR is what surfaces the failure, and CI needs it green — but happy to split it if you'd rather isolate the change.

we can keep the change in this pr

@zyratlo
zyratlo requested a review from mengw15 July 20, 2026 16:54

@mengw15 mengw15 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the detailed replies. Both make sense — keeping getJupyterURL() here alongside getJupyterIframeURL() for #5265 keeps the service cohesive, and the per-user-pod filename rationale (plus the new code comment) matches the architectural direction we agreed on in #5258. LGTM.

@mengw15
mengw15 added this pull request to the merge queue Jul 20, 2026
Merged via the queue into apache:main with commit 0467c76 Jul 20, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

frontend Changes related to the frontend GUI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Notebook Migration] Add frontend notebook migration orchestration service

4 participants