Skip to content

Export compiled design tree as a JSON#482

Merged
ducky64 merged 25 commits intomasterfrom
design-tree-dump
Apr 26, 2026
Merged

Export compiled design tree as a JSON#482
ducky64 merged 25 commits intomasterfrom
design-tree-dump

Conversation

@ducky64
Copy link
Copy Markdown
Collaborator

@ducky64 ducky64 commented Apr 26, 2026

Exports the compiled design tree as a JSON, including connectivity and parameter value data. Useful for getting compiled design information when not using the IDE, and perhaps in the future as a easier to use feature than the proto definitions. Meant to be human-usable.

Defines the JSON structure using Pydantic. Does some regex-based postprocessing to condense range values and params.

Adds a functional TransformUtil that makes this easier. Eventually, the other TransformUtil users may be refactored to use this.

Refactors the compiler to track the link-side port and include the block-side port <-> link side port data in the compiled result. This is done by storing the port suffix with the link path.

Also fixes range equality for empty ranges.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “compiled design export” pathway to emit the fully-compiled design tree as human-oriented JSON (including connectivity and solved parameter values), supported by compiler/proto updates to expose block-port ↔ link-port connectivity and a small fix to Range equality semantics for empty ranges.

Changes:

  • Extend CompilerResult protobuf (and Scala compiler server) to include explicit port connectivity mappings (connections).
  • Introduce a functional design-tree transform base (FnTransformBase) and a CompiledDesignExportTransform that builds a Pydantic JSON-ready structure with some post-serialization compaction.
  • Emit a *.compiled.json artifact from compile_board, and add unit tests covering basic hierarchy connectivity + solved range values.

Reviewed changes

Copilot reviewed 14 out of 16 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
proto/edgrpc/compiler.proto Adds CompilerResult.connections message/field for connectivity export.
edg/hdl_server/main.py Bumps expected proto version to 10.
edg/edgrpc/compiler_pb2.pyi Updates generated typings for new connections field.
edg/edgrpc/compiler_pb2.py Updates generated protobuf module for new connections field.
edg/core/test_compiled_design_export.py Adds tests for hierarchy connectivity + solved parameter/range export.
edg/core/init.py Exports CompiledDesignExportTransform from edg.core.
edg/core/ScalaCompilerInterface.py Plumbs connections into CompiledDesign and adds connectivity lookup helpers.
edg/core/Range.py Fixes equality behavior for empty ranges encoded as NaN/NaN.
edg/core/FnTransformUtil.py Adds a functional traversal/transform base class for design-tree walking.
edg/core/CompiledDesignExport.py Adds Pydantic models + transform to export compiled design as JSON + regex postprocessing.
edg/BoardCompiler.py Writes *.compiled.json alongside existing compilation outputs.
compiler/src/main/scala/edg/compiler/ConstProp.scala Tracks connected link port suffixes and exposes all port connections.
compiler/src/main/scala/edg/compiler/CompilerServerMain.scala Serializes connections into CompilerResult.
compiler/src/main/scala/edg/compiler/Compiler.scala Bumps expected proto version and plumbs suffix-based connectivity tracking.
.gitignore Ignores examples/*/*.compiled.json outputs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +9 to +11
TransformedPort = TypeVar("TransformedPort", default=None)
TransformedBlock = TypeVar("TransformedBlock", default=None)
TransformedLink = TypeVar("TransformedLink", default=None)
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TypeVar(..., default=None) requires newer Python (PEP 696) and will raise at runtime on this repo’s supported Python versions (pyproject.toml sets requires-python >=3.9). Please remove the default= arguments and, if you want the type params to be optional, instead make the return types explicitly Optional[...] / use overloads / provide concrete Generic[...] parameters at subclasses.

Suggested change
TransformedPort = TypeVar("TransformedPort", default=None)
TransformedBlock = TypeVar("TransformedBlock", default=None)
TransformedLink = TypeVar("TransformedLink", default=None)
TransformedPort = TypeVar("TransformedPort")
TransformedBlock = TypeVar("TransformedBlock")
TransformedLink = TypeVar("TransformedLink")

Copilot uses AI. Check for mistakes.
@@ -0,0 +1,226 @@
from typing import Optional, Dict, List, Any, Union, Mapping
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from typing import ... override is not available on Python 3.9–3.11. Elsewhere in this repo override is consistently imported from typing_extensions (and typing_extensions is already a dependency). Please switch this import to from typing_extensions import override to avoid runtime ImportError.

Copilot uses AI. Check for mistakes.
Comment thread edg/core/CompiledDesignExport.py Outdated
Comment on lines +217 to +223
r""""type":\s*"range",(\s*)"value":\s*\[\s*([\S]+),\s*([\S]+)\s*\]""",
lambda m: f""""type": "range",{m.group(1)}"value": [{m.group(2)}, {m.group(3)}]""",
json_str,
)
json_str = re.sub(
r"""\{\s*"type":\s*"(\S+)",\s*"value":\s*(.+)\s*\}""",
lambda m: f"""{{ "type": "{m.group(1)}", "value": {m.group(2)} }}""",
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regex pattern string here starts with r""""type"... (4 quotes), which Python parses as two adjacent string literals (an empty raw string + a normal string). That’s very easy to misread and can trigger invalid-escape warnings for sequences like \s as Python tightens handling. Please rewrite these patterns/replacements as single raw strings (eg starting with r'"type":\s*...') to avoid accidental literal concatenation and future incompatibilities.

Suggested change
r""""type":\s*"range",(\s*)"value":\s*\[\s*([\S]+),\s*([\S]+)\s*\]""",
lambda m: f""""type": "range",{m.group(1)}"value": [{m.group(2)}, {m.group(3)}]""",
json_str,
)
json_str = re.sub(
r"""\{\s*"type":\s*"(\S+)",\s*"value":\s*(.+)\s*\}""",
lambda m: f"""{{ "type": "{m.group(1)}", "value": {m.group(2)} }}""",
r'"type":\s*"range",(\s*)"value":\s*\[\s*([\S]+),\s*([\S]+)\s*\]',
lambda m: rf'"type": "range",{m.group(1)}"value": [{m.group(2)}, {m.group(3)}]',
json_str,
)
json_str = re.sub(
r'\{\s*"type":\s*"(\S+)",\s*"value":\s*(.+)\s*\}',
lambda m: rf'{{ "type": "{m.group(1)}", "value": {m.group(2)} }}',

Copilot uses AI. Check for mistakes.
@ducky64 ducky64 merged commit 05995ba into master Apr 26, 2026
12 checks passed
@ducky64 ducky64 deleted the design-tree-dump branch April 26, 2026 02:56
@ducky64 ducky64 linked an issue Apr 26, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dump compiled design tree into a json or some other structured data

2 participants