Repository navigation
Round-trip every remaining e2e JSON response through its schema - #1332
Conversation
Signed-off-by: Juan Cruz Viotti <jv@jviotti.com>
There was a problem hiding this comment.
1 issue found across 7 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="enterprise/e2e/html/hurl/schemas-fragment.all.hurl">
<violation number="1" location="enterprise/e2e/html/hurl/schemas-fragment.all.hurl:10">
P2: The fragment-bearing `POST /schemas/evaluate` response is still not validated against its advertised schema, so this test does not cover every JSON response as intended. Add a capture block after that response and evaluate it through the captured error schema before continuing.</violation>
</file>
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
| Access-Control-Allow-Origin: * | ||
| Access-Control-Expose-Headers: Link, ETag | ||
| Link: </self/v1/schemas/api/error>; rel="describedby" | ||
| [Captures] |
There was a problem hiding this comment.
P2: The fragment-bearing POST /schemas/evaluate response is still not validated against its advertised schema, so this test does not cover every JSON response as intended. Add a capture block after that response and evaluate it through the captured error schema before continuing.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At enterprise/e2e/html/hurl/schemas-fragment.all.hurl, line 10:
<comment>The fragment-bearing `POST /schemas/evaluate` response is still not validated against its advertised schema, so this test does not cover every JSON response as intended. Add a capture block after that response and evaluate it through the captured error schema before continuing.</comment>
<file context>
@@ -7,6 +7,9 @@ Content-Type: application/problem+json
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Link, ETag
Link: </self/v1/schemas/api/error>; rel="describedby"
+[Captures]
+last_response: body
+schema_path: header "Link" regex "<([^>]+)>"
</file context>
🤖 Augment PR SummarySummary: Extends schema round-trip coverage in enterprise end-to-end tests.
🤖 Was this summary useful? React with 👍 or 👎 |
There was a problem hiding this comment.
Benchmark (community)
Details
| Benchmark suite | Current: 3c85722 | Previous: f5b6e93 | Ratio |
|---|---|---|---|
Add one schema (0 existing) |
264 ms |
284 ms |
0.93 |
Add one schema (100 existing) |
43 ms |
54 ms |
0.80 |
Add one schema (1000 existing) |
88 ms |
98 ms |
0.90 |
Add one schema (10000 existing) |
1139 ms |
755 ms |
1.51 |
Update one schema (1 existing) |
35 ms |
39 ms |
0.90 |
Update one schema (101 existing) |
43 ms |
51 ms |
0.84 |
Update one schema (1001 existing) |
89 ms |
102 ms |
0.87 |
Update one schema (10001 existing) |
864 ms |
899 ms |
0.96 |
Cached rebuild (1 existing) |
9 ms |
18 ms |
0.50 |
Cached rebuild (101 existing) |
15 ms |
18 ms |
0.83 |
Cached rebuild (1001 existing) |
29 ms |
51 ms |
0.57 |
Cached rebuild (10001 existing) |
243 ms |
359 ms |
0.68 |
Index 100 schemas |
506 ms |
570 ms |
0.89 |
Index 1000 schemas |
1079 ms |
1489 ms |
0.72 |
Index 10000 schemas |
11543 ms |
13359 ms |
0.86 |
Index 10000 schemas (custom meta-schema) |
15096 ms |
16653 ms |
0.91 |
Index 10000 schemas ($ref fan-out) |
13822 ms |
15709 ms |
0.88 |
test/e2e/html: Schema Fetch (p50) |
304 us |
379 us |
0.80 |
test/e2e/html: Schema Fetch (p99) |
391 us |
538 us |
0.73 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Benchmark (enterprise)
Details
| Benchmark suite | Current: 3c85722 | Previous: f5b6e93 | Ratio |
|---|---|---|---|
Add one schema (0 existing) |
574 ms |
378 ms |
1.52 |
Add one schema (100 existing) |
121 ms |
92 ms |
1.32 |
Add one schema (1000 existing) |
170 ms |
145 ms |
1.17 |
Add one schema (10000 existing) |
772 ms |
1121 ms |
0.69 |
Update one schema (1 existing) |
111 ms |
86 ms |
1.29 |
Update one schema (101 existing) |
127 ms |
90 ms |
1.41 |
Update one schema (1001 existing) |
177 ms |
157 ms |
1.13 |
Update one schema (10001 existing) |
803 ms |
1006 ms |
0.80 |
Cached rebuild (1 existing) |
15 ms |
12 ms |
1.25 |
Cached rebuild (101 existing) |
20 ms |
16 ms |
1.25 |
Cached rebuild (1001 existing) |
45 ms |
39 ms |
1.15 |
Cached rebuild (10001 existing) |
304 ms |
316 ms |
0.96 |
Index 100 schemas |
802 ms |
623 ms |
1.29 |
Index 1000 schemas |
2016 ms |
1371 ms |
1.47 |
Index 10000 schemas |
16706 ms |
12169 ms |
1.37 |
Index 10000 schemas (custom meta-schema) |
16733 ms |
12490 ms |
1.34 |
Index 10000 schemas ($ref fan-out) |
22195 ms |
15729 ms |
1.41 |
enterprise/e2e/auth: Schema Anonymous (p50) |
385 us |
212 us |
1.82 |
enterprise/e2e/auth: Schema Anonymous (p99) |
506 us |
334 us |
1.51 |
enterprise/e2e/auth: Schema API Key Identity (p50) |
390 us |
216 us |
1.81 |
enterprise/e2e/auth: Schema API Key Identity (p99) |
497 us |
316 us |
1.57 |
enterprise/e2e/auth: Schema API Key SHA256 (p50) |
398 us |
215 us |
1.85 |
enterprise/e2e/auth: Schema API Key SHA256 (p99) |
517 us |
295 us |
1.75 |
enterprise/e2e/auth: Schema JWT (p50) |
518 us |
313 us |
1.65 |
enterprise/e2e/auth: Schema JWT (p99) |
666 us |
460 us |
1.45 |
test/e2e/html: Schema Fetch (p50) |
391 us |
219 us |
1.79 |
test/e2e/html: Schema Fetch (p99) |
478 us |
389 us |
1.23 |
This comment was automatically generated by workflow using github-action-benchmark.
Signed-off-by: Juan Cruz Viotti jv@jviotti.com