Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new “compiled design export” pathway to emit the fully-compiled design tree as human-oriented JSON (including connectivity and solved parameter values), supported by compiler/proto updates to expose block-port ↔ link-port connectivity and a small fix to Range equality semantics for empty ranges.
Changes:
- Extend
CompilerResultprotobuf (and Scala compiler server) to include explicit port connectivity mappings (connections). - Introduce a functional design-tree transform base (
FnTransformBase) and aCompiledDesignExportTransformthat builds a Pydantic JSON-ready structure with some post-serialization compaction. - Emit a
*.compiled.jsonartifact fromcompile_board, and add unit tests covering basic hierarchy connectivity + solved range values.
Reviewed changes
Copilot reviewed 14 out of 16 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| proto/edgrpc/compiler.proto | Adds CompilerResult.connections message/field for connectivity export. |
| edg/hdl_server/main.py | Bumps expected proto version to 10. |
| edg/edgrpc/compiler_pb2.pyi | Updates generated typings for new connections field. |
| edg/edgrpc/compiler_pb2.py | Updates generated protobuf module for new connections field. |
| edg/core/test_compiled_design_export.py | Adds tests for hierarchy connectivity + solved parameter/range export. |
| edg/core/init.py | Exports CompiledDesignExportTransform from edg.core. |
| edg/core/ScalaCompilerInterface.py | Plumbs connections into CompiledDesign and adds connectivity lookup helpers. |
| edg/core/Range.py | Fixes equality behavior for empty ranges encoded as NaN/NaN. |
| edg/core/FnTransformUtil.py | Adds a functional traversal/transform base class for design-tree walking. |
| edg/core/CompiledDesignExport.py | Adds Pydantic models + transform to export compiled design as JSON + regex postprocessing. |
| edg/BoardCompiler.py | Writes *.compiled.json alongside existing compilation outputs. |
| compiler/src/main/scala/edg/compiler/ConstProp.scala | Tracks connected link port suffixes and exposes all port connections. |
| compiler/src/main/scala/edg/compiler/CompilerServerMain.scala | Serializes connections into CompilerResult. |
| compiler/src/main/scala/edg/compiler/Compiler.scala | Bumps expected proto version and plumbs suffix-based connectivity tracking. |
| .gitignore | Ignores examples/*/*.compiled.json outputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| TransformedPort = TypeVar("TransformedPort", default=None) | ||
| TransformedBlock = TypeVar("TransformedBlock", default=None) | ||
| TransformedLink = TypeVar("TransformedLink", default=None) |
There was a problem hiding this comment.
TypeVar(..., default=None) requires newer Python (PEP 696) and will raise at runtime on this repo’s supported Python versions (pyproject.toml sets requires-python >=3.9). Please remove the default= arguments and, if you want the type params to be optional, instead make the return types explicitly Optional[...] / use overloads / provide concrete Generic[...] parameters at subclasses.
| TransformedPort = TypeVar("TransformedPort", default=None) | |
| TransformedBlock = TypeVar("TransformedBlock", default=None) | |
| TransformedLink = TypeVar("TransformedLink", default=None) | |
| TransformedPort = TypeVar("TransformedPort") | |
| TransformedBlock = TypeVar("TransformedBlock") | |
| TransformedLink = TypeVar("TransformedLink") |
| @@ -0,0 +1,226 @@ | |||
| from typing import Optional, Dict, List, Any, Union, Mapping | |||
There was a problem hiding this comment.
from typing import ... override is not available on Python 3.9–3.11. Elsewhere in this repo override is consistently imported from typing_extensions (and typing_extensions is already a dependency). Please switch this import to from typing_extensions import override to avoid runtime ImportError.
| r""""type":\s*"range",(\s*)"value":\s*\[\s*([\S]+),\s*([\S]+)\s*\]""", | ||
| lambda m: f""""type": "range",{m.group(1)}"value": [{m.group(2)}, {m.group(3)}]""", | ||
| json_str, | ||
| ) | ||
| json_str = re.sub( | ||
| r"""\{\s*"type":\s*"(\S+)",\s*"value":\s*(.+)\s*\}""", | ||
| lambda m: f"""{{ "type": "{m.group(1)}", "value": {m.group(2)} }}""", |
There was a problem hiding this comment.
The regex pattern string here starts with r""""type"... (4 quotes), which Python parses as two adjacent string literals (an empty raw string + a normal string). That’s very easy to misread and can trigger invalid-escape warnings for sequences like \s as Python tightens handling. Please rewrite these patterns/replacements as single raw strings (eg starting with r'"type":\s*...') to avoid accidental literal concatenation and future incompatibilities.
| r""""type":\s*"range",(\s*)"value":\s*\[\s*([\S]+),\s*([\S]+)\s*\]""", | |
| lambda m: f""""type": "range",{m.group(1)}"value": [{m.group(2)}, {m.group(3)}]""", | |
| json_str, | |
| ) | |
| json_str = re.sub( | |
| r"""\{\s*"type":\s*"(\S+)",\s*"value":\s*(.+)\s*\}""", | |
| lambda m: f"""{{ "type": "{m.group(1)}", "value": {m.group(2)} }}""", | |
| r'"type":\s*"range",(\s*)"value":\s*\[\s*([\S]+),\s*([\S]+)\s*\]', | |
| lambda m: rf'"type": "range",{m.group(1)}"value": [{m.group(2)}, {m.group(3)}]', | |
| json_str, | |
| ) | |
| json_str = re.sub( | |
| r'\{\s*"type":\s*"(\S+)",\s*"value":\s*(.+)\s*\}', | |
| lambda m: rf'{{ "type": "{m.group(1)}", "value": {m.group(2)} }}', |
Exports the compiled design tree as a JSON, including connectivity and parameter value data. Useful for getting compiled design information when not using the IDE, and perhaps in the future as a easier to use feature than the proto definitions. Meant to be human-usable.
Defines the JSON structure using Pydantic. Does some regex-based postprocessing to condense range values and params.
Adds a functional TransformUtil that makes this easier. Eventually, the other TransformUtil users may be refactored to use this.
Refactors the compiler to track the link-side port and include the block-side port <-> link side port data in the compiled result. This is done by storing the port suffix with the link path.
Also fixes range equality for empty ranges.