feat(python-notebook-migration): add LLM client for notebook-to-workflow conversion

[zyratlo]

What changes were proposed in this PR?

Introduces the frontend LLM session class that converts a Jupyter notebook into a Texera workflow JSON plus a bidirectional cell to operator mapping, along with the prompt library it uses. Two files under frontend/src/app/workspace/service/notebook-migration/, totalling ~700 lines (~410 of which is prompt text).

migration-llm.ts — defines NotebookMigrationLLM, an @Injectable class wrapping a Vercel AI SDK chat session against the LiteLLM proxy already exposed on main at /api/chat/completion.

• initialize(modelType, apiKey) — builds an OpenAI-compatible chat client via createOpenAI({ baseURL: AppSettings.getApiEndpoint() }), seeds the message history with Texera documentation as system messages.
• verifyConnection() — does a 10-token ping call to validate that the API key works against the configured model.
• convertNotebookToWorkflow(notebook) — extracts code cells (each tagged with a UUID in metadata.uuid), sends WORKFLOW_PROMPT + the notebook to get a JSON of UDF operators / edges, then sends MAPPING_PROMPT to get the cell↔operator mapping. Assembles a complete Texera workflow JSON (PythonUDFV2 operators with stub input/output ports, links derived from the LLM's edge list, default settings) plus a bidirectional operator_to_cell / cell_to_operator mapping. Returns both as a JSON string.
• close() — clears the message history and the model reference.

migration-prompts.ts — string constants used by migration-llm.ts: TEXERA_OVERVIEW, TUPLE_DOCUMENTATION, TABLE_DOCUMENTATION, OPERATOR_DOCUMENTATION, UDF_INPUT_PORT_DOCUMENTATION, EXAMPLE_OF_GOOD_CONVERSION, VISUALIZER_DOCUMENTATION, EXAMPLE_OF_MULTIPLE_UDF_CONVERSION, WORKFLOW_PROMPT, MAPPING_PROMPT.

Any related issues, documentation, discussions?

Closes #5259
Parent issue #4301

How was this PR tested?

No unit tests were included for these reasons:

• A large portion of the changes are prompt text, which are not testable, only readable. However the prompt text can be changed to improve the performance of the LLM.
• Testing would require mocking a significant amount of logic that will be introduced in later PRs, since the logic in migration-llm.ts is parsing a response.

However I am open to writing tests based on review feedback.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 4.7)

[codecov-commenter]

Codecov Report

❌ Patch coverage is 81.57895% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 54.64%. Comparing base (0c179ce) to head (a1a81aa).

Files with missing lines	Patch %	Lines
...kspace/service/notebook-migration/migration-llm.ts	79.61%	14 Missing and 7 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #5260      +/-   ##
============================================
- Coverage     54.81%   54.64%   -0.18%     
+ Complexity     2958     2918      -40     
============================================
  Files          1115     1111       -4     
  Lines         43019    42919     -100     
  Branches       4633     4623      -10     
============================================
- Hits          23580    23452     -128     
- Misses        18053    18097      +44     
+ Partials       1386     1370      -16     

Flag	Coverage Δ		*Carryforward flag
access-control-service	69.69% <ø> (-0.31%)	⬇️	Carriedforward from cc94414
agent-service	34.36% <ø> (ø)		Carriedforward from cc94414
amber	56.56% <ø> (-0.46%)	⬇️	Carriedforward from cc94414
computing-unit-managing-service	0.28% <ø> (+0.28%)	⬆️	Carriedforward from cc94414
config-service	54.83% <ø> (+3.27%)	⬆️	Carriedforward from cc94414
file-service	59.31% <ø> (+0.28%)	⬆️	Carriedforward from cc94414
frontend	48.64% <81.57%> (+0.24%)	⬆️
notebook-migration-service	?
pyamber	90.20% <ø> (ø)		Carriedforward from cc94414
python	90.76% <ø> (ø)		Carriedforward from cc94414
workflow-compiling-service	57.57% <ø> (+2.42%)	⬆️	Carriedforward from cc94414

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[zyratlo]

/request-review @mengw15

[mengw15]

Left some comments

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @PG1204
  You can notify them by mentioning @PG1204 in a comment.

[mengw15]

A couple of cleanups on migration-prompts.ts

[mengw15]

Two more comments on migration-llm.ts

[Copilot]

Pull request overview

Adds a frontend notebook-migration LLM client that calls Texera’s LiteLLM proxy to convert Jupyter notebooks into Texera workflow JSON, alongside a prompt library and unit tests.

Changes:

• Introduces NotebookMigrationLLM Angular service to run a 2-step LLM flow (workflow JSON + cell↔operator mapping) and assemble a Texera workflow payload.
• Adds a prompt-constant library used to seed the LLM session with Texera/UDF documentation and conversion instructions.
• Adds Vitest unit tests for JSON parsing and deterministic workflow/mapping construction, and wires in @ai-sdk/openai.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 13 comments.

Show a summary per file

File	Description
frontend/package.json	Adds @ai-sdk/openai dependency used by the new migration client.
frontend/yarn.lock	Locks the added @ai-sdk/openai dependency.
frontend/src/app/workspace/service/notebook-migration/migration-prompts.ts	Adds the prompt/documentation constants for notebook-to-workflow conversion.
frontend/src/app/workspace/service/notebook-migration/migration-llm.ts	Implements the notebook migration LLM session, prompting, parsing, and workflow/mapping assembly.
frontend/src/app/workspace/service/notebook-migration/migration-llm.spec.ts	Adds unit tests for parsing and conversion behavior with mocked LLM transport.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mengw15]

Left a follow-up note on Copilot's apiKey/JWT comment.

[mengw15]

LGTM. Every substantive point from prior rounds (both mine and Copilot's) is addressed in this iteration — a few highlights:

• JWT auth wired through AuthService.getAccessToken() with a clear comment explaining the AI-SDK-Authorization-header / backend-master-key-substitution split.
• seedDocumentation() extracted and called at the start of each convertNotebookToWorkflow, so the same instance can convert multiple notebooks without cross-conversion context leakage.
• The binary-attributeType follow-up resolved cleverly: intermediate UDFs (those that are an edge source) keep binary so rich objects round-trip via pickle, terminal UDFs default to string so the result panel renders typed values. Better than just flipping the default.

[Yicong-Huang]

@zyratlo how come adding this new dependency does not need change in the LICENSE-binary file?

cc @bobbai00

[zyratlo]

Thanks for pointing this out, we missed it during the development and review phase. We should fix it and change the LICENSE file if it is necessary.

I am currently traveling and do not have reliable Internet connection, if anything requires an urgent fix please work with @mengw15 , otherwise I will fix it when I am back.

[zyratlo]

I looked into it, the reason we didn't need to change LICENSE-binary even though we added a new dependency is because there's no consumer for this code in main right now (we're merging the whole tool incrementally across multiple PRs). The license checker diffs against what the production build actually bundles, and with no consumer the migration client is tree-shaken out, so @ai-sdk/openai never reaches the bundle. When the consumer is merged in #5273, it'll be bundled and we'll add it to LICENSE-binary then.