Speed up test parsing by lazily compiling one exhaustive for errors

[jviotti]

Signed-off-by: Juan Cruz Viotti jv@jviotti.com

[augmentcode]

🤖 Augment PR Summary

Summary: This PR makes exhaustive test-schema compilation lazy.

Changes:

• Moves compilation dependencies and compilation options into TestSuite.
• Keeps fast-validation templates eagerly compiled during parsing.
• Replaces the public exhaustive-template vector with accessor methods.
• Adds a per-target optional cache for exhaustive templates.
• Compiles and caches an exhaustive template on its first accessor call.
• Centralizes target compilation and schema-resolution error translation.
• Updates the runner to obtain fast templates through the new accessor.
• Adjusts parser tests for the encapsulated template storage.
• Adds coverage for fast validation, exhaustive annotations, caching, and deferred compilation.

Technical note: Deferred exhaustive compilation retains the resolver, walker, compiler, defaults, and tweaks in the suite for use after parsing.

_{🤖 Was this summary useful? React with 👍 or 👎}

[augmentcode]

Review completed. 1 suggestion posted.

Comment augment review to trigger a new review at any time.

[cubic-dev-ai]

1 issue found across 4 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/test/test_parser.cc">

<violation number="1" location="src/test/test_parser.cc:270">
P2: The new `exhaustive()` accessor performs an unsynchronized, stateful lazy compile: the first call mutates `this->schemas_exhaustive[target_index]` (compiling the schema and caching it) and every subsequent call serves the cached `Template`. Because this cache is mutated on demand, `exhaustive()` had to be made non-const (so a `const TestSuite&` can no longer request the exhaustive template, whereas the old public vector member was readable from any instance), and sharing a parsed `TestSuite` across threads would now be a data race on `schemas_exhaustive` — previously all schemas were fully compiled during `parse()` and only read afterward. Consider making the cache `mutable` and `exhaustive()` `const` for const-correctness, and either guarding the lazy compile with a mutex or documenting the single-threaded requirement explicitly so concurrent/const reuse doesn't silently miscompile or race.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[github-actions]

Benchmark (linux/llvm)

Details

Benchmark suite	Current: a32ae02	Previous: 33c3ad0	Ratio
Schema_Iterator_ISO_Language	3057759.666666647 ns/iter	2926367.30472127 ns/iter	1.04
Schema_Tracker_ISO_Language	4892357.088434424 ns/iter	4566563.039215788 ns/iter	1.07
Schema_Tracker_ISO_Language_To_JSON	20193840.23529202 ns/iter	20891775.272730228 ns/iter	0.97
Schema_Frame_WoT_References	5019139.204225059 ns/iter	4504939.1225802535 ns/iter	1.11
Schema_Frame_OMC_References	22976539.965517577 ns/iter	19270836.19444324 ns/iter	1.19
Schema_Frame_OMC_Locations	20730448.32353416 ns/iter	17864890.225001775 ns/iter	1.16
Schema_Frame_ISO_Language_Locations	102730106.49999984 ns/iter	96880521.71428093 ns/iter	1.06
Schema_Frame_KrakenD_References	45041792.18750437 ns/iter	36370407.47369275 ns/iter	1.24
Schema_Frame_KrakenD_Reachable	577814257 ns/iter	548631103.9999237 ns/iter	1.05
Schema_Frame_ISO_Language_Locations_To_JSON	127772218.00001825 ns/iter	115390725.79998901 ns/iter	1.11
Schema_Frame_Many_Resources_References	380807162.9999859 ns/iter	347981299.5000202 ns/iter	1.09
Schema_Bundle_Meta_2020_12	1475318.523100649 ns/iter	1458489.337505379 ns/iter	1.01
Schema_Format_ISO_Language_To_JSON	109283676.66671573 ns/iter	103378367.57142733 ns/iter	1.06
E2E_Compiler_adaptivecard	58143829.66665714 ns/iter	57138504.000004806 ns/iter	1.02
E2E_Compiler_ansible_meta	25468923.321430534 ns/iter	25039020.857142143 ns/iter	1.02
E2E_Compiler_aws_cdk	313841.18970917445 ns/iter	323474.2095062284 ns/iter	0.97
E2E_Compiler_babelrc	2338149.8466665107 ns/iter	2300782.2499996684 ns/iter	1.02
E2E_Compiler_clang_format	16551071.857141625 ns/iter	16274580.813953519 ns/iter	1.02
E2E_Compiler_cmake_presets	25545784.11111295 ns/iter	24962306.428571016 ns/iter	1.02
E2E_Compiler_code_climate	2342703.7391301016 ns/iter	2303073.8782897457 ns/iter	1.02
E2E_Compiler_cql2	12915500.240738867 ns/iter	12767077.781820098 ns/iter	1.01
E2E_Compiler_cspell	22860241.870969933 ns/iter	22340480.258061424 ns/iter	1.02
E2E_Compiler_cypress	3920286.620110834 ns/iter	3905782.833333483 ns/iter	1.00
E2E_Compiler_deno	7080538.353535541 ns/iter	7001639.4100002805 ns/iter	1.01
E2E_Compiler_dependabot	3579523.7384613403 ns/iter	3505887.244999713 ns/iter	1.02
E2E_Compiler_draft_04	2452895.571929746 ns/iter	2422122.70242241 ns/iter	1.01
E2E_Compiler_fabric_mod	4329328.351852865 ns/iter	4235117.242423615 ns/iter	1.02
E2E_Compiler_geojson	20710844.61765087 ns/iter	19974468.285714205 ns/iter	1.04
E2E_Compiler_gitpod_configuration	4529380.279220052 ns/iter	4485931.403847551 ns/iter	1.01
E2E_Compiler_helm_chart_lock	602463.2590361901 ns/iter	623291.729443036 ns/iter	0.97
E2E_Compiler_importmap	236805.3540960833 ns/iter	244393.51066058376 ns/iter	0.97
E2E_Compiler_jasmine	1218728.9337976007 ns/iter	1221932.3350786297 ns/iter	1.00
E2E_Compiler_jshintrc	3888498.4833329227 ns/iter	3854102.6153842434 ns/iter	1.01
E2E_Compiler_jsconfig	20409209.794111654 ns/iter	20042647.17143087 ns/iter	1.02
E2E_Compiler_krakend	137615482.19999894 ns/iter	133916769.79999828 ns/iter	1.03
E2E_Compiler_lazygit	31222105.954549823 ns/iter	30038239.434786547 ns/iter	1.04
E2E_Compiler_lerna	1431356.195918425 ns/iter	1430901.0224951261 ns/iter	1.00
E2E_Compiler_nest_cli	6827529.203883829 ns/iter	6656641.009434902 ns/iter	1.03
E2E_Compiler_omc	152209711.8000147 ns/iter	155460231.79999793 ns/iter	0.98
E2E_Compiler_omnisharp	6077433.090908099 ns/iter	5974304.405171928 ns/iter	1.02
E2E_Compiler_openapi	23075722.766664814 ns/iter	22841778.548391014 ns/iter	1.01
E2E_Compiler_pre_commit_hooks	4693266.503355767 ns/iter	4656465.880001027 ns/iter	1.01
E2E_Compiler_pulumi	3665648.0785343284 ns/iter	3600298.4536082163 ns/iter	1.02
E2E_Compiler_semantic_release	1368085.5000002268 ns/iter	1374090.3602359956 ns/iter	1.00
E2E_Compiler_stale	1401161.430583586 ns/iter	1402186.3757455791 ns/iter	1.00
E2E_Compiler_stylecop	5210863.207407278 ns/iter	5234354.559702093 ns/iter	1.00
E2E_Compiler_tmuxinator	1693656.8024097558 ns/iter	1690893.3623187034 ns/iter	1.00
E2E_Compiler_ui5	48119646.266665466 ns/iter	45732598.60000386 ns/iter	1.05
E2E_Compiler_ui5_manifest	318424862.50002366 ns/iter	308301360.9999625 ns/iter	1.03
E2E_Compiler_unreal_engine_uproject	4201919.401197584 ns/iter	4093513.928571231 ns/iter	1.03
E2E_Compiler_users_array	759135.8782608792 ns/iter	779939.1486031959 ns/iter	0.97
E2E_Compiler_vercel	12263797.49122667 ns/iter	11964937.474577723 ns/iter	1.02
E2E_Compiler_yamllint	288702.8580857986 ns/iter	303940.3418543444 ns/iter	0.95
E2E_Evaluator_adaptivecard	25623.012972655153 ns/iter	25721.395118349334 ns/iter	1.00
E2E_Evaluator_ansible_meta	222954.95981012797 ns/iter	246475.14054815943 ns/iter	0.90
E2E_Evaluator_aws_cdk	45589.48308799868 ns/iter	53490.476971416436 ns/iter	0.85
E2E_Evaluator_babelrc	97992.74173582686 ns/iter	116615.63772453615 ns/iter	0.84
E2E_Evaluator_cerebrum_criminal_case	1100019.7012780148 ns/iter	1034129.3185185215 ns/iter	1.06
E2E_Evaluator_clang_format	115042.69087920456 ns/iter	120153.13559319814 ns/iter	0.96
E2E_Evaluator_cmake_presets	4506296.343284268 ns/iter	4712302.54666807 ns/iter	0.96
E2E_Evaluator_code_climate	161207.00764955615 ns/iter	184036.73636605724 ns/iter	0.88
E2E_Evaluator_cql2	195497.08531743314 ns/iter	217451.07977739576 ns/iter	0.90
E2E_Evaluator_cspell	617247.1224488919 ns/iter	649850.9527856299 ns/iter	0.95
E2E_Evaluator_cypress	177622.76782493622 ns/iter	199566.54339082513 ns/iter	0.89
E2E_Evaluator_deno	267775.9724559492 ns/iter	338469.0878962849 ns/iter	0.79
E2E_Evaluator_dependabot	329902.1623160097 ns/iter	337527.2395181358 ns/iter	0.98
E2E_Evaluator_draft_04	11462858.285714868 ns/iter	9079936.041669296 ns/iter	1.26
E2E_Evaluator_fabric_mod	705366.3748620528 ns/iter	718906.4391752123 ns/iter	0.98
E2E_Evaluator_geojson	18804110.945948932 ns/iter	16211214.534884442 ns/iter	1.16
E2E_Evaluator_gitpod_configuration	219894.43657071702 ns/iter	254206.9569185692 ns/iter	0.87
E2E_Evaluator_helm_chart_lock	232849.38323352303 ns/iter	324787.4337016139 ns/iter	0.72
E2E_Evaluator_importmap	50830.42020572017 ns/iter	50265.98953929099 ns/iter	1.01
E2E_Evaluator_jasmine	100074.3835814338 ns/iter	110273.89893117185 ns/iter	0.91
E2E_Evaluator_jshintrc	1192068.0560274029 ns/iter	1218348.360139857 ns/iter	0.98
E2E_Evaluator_jsconfig	414714.12373593455 ns/iter	440240.85651349486 ns/iter	0.94
E2E_Evaluator_krakend	189092.0413736683 ns/iter	215624.2125195651 ns/iter	0.88
E2E_Evaluator_lazygit	129030.51978648962 ns/iter	153083.225841941 ns/iter	0.84
E2E_Evaluator_lerna	114634.37574354291 ns/iter	148699.7343618186 ns/iter	0.77
E2E_Evaluator_nest_cli	184416.77848942304 ns/iter	192680.17106002299 ns/iter	0.96
E2E_Evaluator_omc	22192.514142523025 ns/iter	22278.514718466537 ns/iter	1.00
E2E_Evaluator_omnisharp	570058.6888517481 ns/iter	621461.0044169718 ns/iter	0.92
E2E_Evaluator_openapi	18266999.236845404 ns/iter	15188074.044443864 ns/iter	1.20
E2E_Evaluator_pre_commit_hooks	514480.1870967944 ns/iter	521249.5846267393 ns/iter	0.99
E2E_Evaluator_pulumi	679313.1352140544 ns/iter	689453.5113077423 ns/iter	0.99
E2E_Evaluator_semantic_release	87411.78739862311 ns/iter	103298.71096151276 ns/iter	0.85
E2E_Evaluator_stale	153565.4080649053 ns/iter	159118.37445935904 ns/iter	0.97
E2E_Evaluator_stylecop	273926.3634204275 ns/iter	302153.3496257749 ns/iter	0.91
E2E_Evaluator_tmuxinator	85469.05609876434 ns/iter	100941.46451331317 ns/iter	0.85
E2E_Evaluator_ui5	510177.8731989565 ns/iter	529525.0272162477 ns/iter	0.96
E2E_Evaluator_ui5_manifest	2485895.3697186196 ns/iter	2626544.4160306305 ns/iter	0.95
E2E_Evaluator_unreal_engine_uproject	441251.0088663713 ns/iter	454728.04193547834 ns/iter	0.97
E2E_Evaluator_users_array	668387.5969581369 ns/iter	590538.1786015941 ns/iter	1.13
E2E_Evaluator_vercel	264607.35521814093 ns/iter	323025.9436228563 ns/iter	0.82
E2E_Evaluator_yamllint	8583.524589355373 ns/iter	9437.792220037598 ns/iter	0.91
JSONLD_Catalog_Simple	25870130.535711918 ns/iter	25803520.82142261 ns/iter	1.00
JSONLD_Catalog_Override_Agreeing	27687788.79999445 ns/iter	26108869.32142518 ns/iter	1.06
JSONLD_Catalog_Override_Shadowing	38520832.05554714 ns/iter	37687328.89474641 ns/iter	1.02
Micro_Draft4_Meta_1_No_Callback	202.10619509006526 ns/iter	205.5183649094742 ns/iter	0.98
Micro_Draft4_Required_Properties	920.8065961639189 ns/iter	893.9817821065997 ns/iter	1.03
Micro_Draft4_Many_Optional_Properties_Minimal_Match	26.50501398949624 ns/iter	23.16701985196419 ns/iter	1.14
Micro_Draft4_Few_Optional_Properties_Minimal_Match	11.800305490585963 ns/iter	12.152312645880674 ns/iter	0.97
Micro_Draft4_Items_Schema	280.6682468768037 ns/iter	280.68129995343685 ns/iter	1.00
Micro_Draft4_Nested_Object	1.436235677177285 ns/iter	1.8383762463856588 ns/iter	0.78
Micro_Draft4_Properties_Triad_Optional	329.2745783493841 ns/iter	323.28764524156225 ns/iter	1.02
Micro_Draft4_Properties_Triad_Closed	274.1732646835019 ns/iter	267.6057673533586 ns/iter	1.02
Micro_Draft4_Properties_Triad_Required	347.46667749640994 ns/iter	347.5955197891747 ns/iter	1.00
Micro_Draft4_Properties_Closed	64.96492985813622 ns/iter	59.7353457720372 ns/iter	1.09
Micro_Draft4_Non_Recursive_Ref	16.16742665771026 ns/iter	14.502366435759292 ns/iter	1.11
Micro_Draft4_Pattern_Properties_True	148.91463976769535 ns/iter	152.06779445394037 ns/iter	0.98
Micro_Draft4_Ref_To_Single_Property	11.997728956508656 ns/iter	12.170556030200741 ns/iter	0.99
Micro_Draft4_Additional_Properties_Type	37.9793080128016 ns/iter	33.30166169848023 ns/iter	1.14
Micro_Draft4_Nested_Oneof	57.520879080717705 ns/iter	57.58376115313549 ns/iter	1.00
Micro_Draft4_Short_Enum	10.926456912061768 ns/iter	9.973378839640061 ns/iter	1.10
Micro_Draft4_Long_Enum	14.077381660275538 ns/iter	14.35146396905173 ns/iter	0.98
Micro_Draft4_Long_Enum_Short_Strings	10.930015201459264 ns/iter	9.975565939032407 ns/iter	1.10
Micro_Draft4_Type_Object	3.5267553243210394 ns/iter	3.4275497359711236 ns/iter	1.03
Micro_Draft4_Ref_Single_100	2306525.8098361734 ns/iter	2329721.690000497 ns/iter	0.99
Micro_Draft4_Compile_Ref_Many_Nested	1703382.8619855796 ns/iter	1688227.830917956 ns/iter	1.01
Micro_Draft4_Compile_Wrap	2375445.4214045885 ns/iter	2314272.5460527763 ns/iter	1.03
Micro_Draft6_Property_Names	123.93980152419238 ns/iter	119.89511892393348 ns/iter	1.03
Micro_Draft6_Compile_FHIR	14884477353.999954 ns/iter	13291250221.999916 ns/iter	1.12
Micro_Draft7_If_Then_Else	23.609940941876275 ns/iter	22.443123055605156 ns/iter	1.05
Micro_2019_09_Unevaluated_Properties	27.338191823011055 ns/iter	26.806986637607224 ns/iter	1.02
Micro_2019_09_Compile_Wrap	311977843.0000224 ns/iter	278607574.333364 ns/iter	1.12
Micro_2020_12_Dynamic_Ref	716.2802330240314 ns/iter	704.7945331266162 ns/iter	1.02
Micro_2020_12_Dynamic_Ref_Single	3.5278973110313223 ns/iter	3.4305890289834133 ns/iter	1.03
Micro_2020_12_Simple_Output_Mask	1051.363395368891 ns/iter	1069.2925871028708 ns/iter	0.98
Micro_2020_12_Simple_Output_Annotations	64146.711519499906 ns/iter	69111.30193714649 ns/iter	0.93
Micro_2020_12_Simple_Output_Annotation_Dropping	1581820.082004518 ns/iter	1651190.117924353 ns/iter	0.96
Micro_2020_12_Compile_NonCircular_Shared_Refs	961824.8280606291 ns/iter	976740.9971951427 ns/iter	0.98
Micro_2020_12_Exhaustive_Deep_Numeric	8520.404402983404 ns/iter	8747.415971097687 ns/iter	0.97
Micro_2020_12_Exhaustive_Deep_Numeric_SimpleOutput	45220.55596258615 ns/iter	44387.586877961556 ns/iter	1.02
Micro_2020_12_Exhaustive_Deep_Numeric_TraceOutput	23681.292528142356 ns/iter	22900.287456276306 ns/iter	1.03
Micro_2020_12_Exhaustive_Deep_Numeric_Fail	162.9968498070849 ns/iter	151.55428691722452 ns/iter	1.08
Micro_2020_12_Exhaustive_Deep_Numeric_Fail_SimpleOutput	2636.6239709056545 ns/iter	2776.0176152724584 ns/iter	0.95
TestSuite_Parse_WoT	23016150.866669707 ns/iter	44379603.999999516 ns/iter	0.52
Alterschema_Check_Readibility_ISO_Language_Set_3	363534478.5000143 ns/iter	357992033.4999542 ns/iter	1.02
Alterschema_Check_Readibility_OMC	89962543.12498309 ns/iter	83462247.87499069 ns/iter	1.08
Alterschema_Check_Readibility_KrakenD	1298203508.0001423 ns/iter	1156339056.0002062 ns/iter	1.12
Alterschema_Apply_Readibility_KrakenD	129723909171.99982 ns/iter	116731434122.0001 ns/iter	1.11
Alterschema_Check_Invalid_External_Refs	269283573.3333444 ns/iter	297632297.4999448 ns/iter	0.90

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

JavaScript (linux/llvm)

Details

Benchmark suite	Current: a32ae02	Previous: 33c3ad0	Ratio
E2E_Evaluator_adaptivecard	119476 ns	120396 ns	0.99
E2E_Evaluator_ansible-meta	549977 ns	517937 ns	1.06
E2E_Evaluator_aws-cdk	131210 ns	148755 ns	0.88
E2E_Evaluator_babelrc	366245 ns	332177 ns	1.10
E2E_Evaluator_cerebrum-criminal-case	2673918 ns	2432333 ns	1.10
E2E_Evaluator_clang-format	275634 ns	255591 ns	1.08
E2E_Evaluator_cmake-presets	18897437 ns	18184259 ns	1.04
E2E_Evaluator_code-climate	510210 ns	526487 ns	0.97
E2E_Evaluator_cql2	1250540 ns	1274960 ns	0.98
E2E_Evaluator_cspell	1702333 ns	1488677 ns	1.14
E2E_Evaluator_cypress	920344 ns	757635 ns	1.21
E2E_Evaluator_deno	1108676 ns	1046891 ns	1.06
E2E_Evaluator_dependabot	1126175 ns	1112660 ns	1.01
E2E_Evaluator_draft-04	21817792 ns	22305123 ns	0.98
E2E_Evaluator_fabric-mod	2542040 ns	2326560 ns	1.09
E2E_Evaluator_geojson	26691946 ns	24734222 ns	1.08
E2E_Evaluator_gitpod-configuration	764146 ns	702180 ns	1.09
E2E_Evaluator_helm-chart-lock	1020960 ns	1039534 ns	0.98
E2E_Evaluator_importmap	273527 ns	266580 ns	1.03
E2E_Evaluator_jasmine	288855 ns	295431 ns	0.98
E2E_Evaluator_jsconfig	1489528 ns	1233948 ns	1.21
E2E_Evaluator_jshintrc	2978147 ns	2708688 ns	1.10
E2E_Evaluator_krakend	641749 ns	669966 ns	0.96
E2E_Evaluator_lazygit	478349 ns	501303 ns	0.95
E2E_Evaluator_lerna	512214 ns	460524 ns	1.11
E2E_Evaluator_nest-cli	627439 ns	584273 ns	1.07
E2E_Evaluator_omc	114814 ns	122939 ns	0.93
E2E_Evaluator_omnisharp	1515421 ns	1367603 ns	1.11
E2E_Evaluator_openapi	43809443 ns	42546429 ns	1.03
E2E_Evaluator_pre-commit-hooks	2088986 ns	1964651 ns	1.06
E2E_Evaluator_pulumi	2104632 ns	1882994 ns	1.12
E2E_Evaluator_semantic-release	259399 ns	257197 ns	1.01
E2E_Evaluator_stale	418003 ns	409782 ns	1.02
E2E_Evaluator_stylecop	1302093 ns	1191598 ns	1.09
E2E_Evaluator_tmuxinator	239119 ns	225457 ns	1.06
E2E_Evaluator_ui5	2240654 ns	2220210 ns	1.01
E2E_Evaluator_ui5-manifest	10850831 ns	11014023 ns	0.99
E2E_Evaluator_unreal-engine-uproject	1643201 ns	1505521 ns	1.09
E2E_Evaluator_users-array	3012075 ns	3064284 ns	0.98
E2E_Evaluator_vercel	1051148 ns	971971 ns	1.08
E2E_Evaluator_yamllint	24634 ns	25665 ns	0.96

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

Benchmark (linux/gcc)

Details

Benchmark suite	Current: a32ae02	Previous: 692cb64	Ratio
Alterschema_Check_Readibility_ISO_Language_Set_3	366757591.0000096 ns/iter	383570380.4999184 ns/iter	0.96
Alterschema_Check_Readibility_OMC	98026153.57141674 ns/iter	97822325.57144459 ns/iter	1.00
Alterschema_Check_Readibility_KrakenD	1313401010.000007 ns/iter	1334062600.000152 ns/iter	0.98
Alterschema_Apply_Readibility_KrakenD	125318490847.00024 ns/iter	126268289353.99998 ns/iter	0.99
Alterschema_Check_Invalid_External_Refs	327015151.99994445 ns/iter	313453176.4999338 ns/iter	1.04
TestSuite_Parse_WoT	24472491.999996953 ns/iter	24592556.535716474 ns/iter	1.00
Micro_2020_12_Dynamic_Ref	842.7607644906742 ns/iter	845.6767605421542 ns/iter	1.00
Micro_2020_12_Dynamic_Ref_Single	4.363199831292932 ns/iter	4.359901925497099 ns/iter	1.00
Micro_2020_12_Simple_Output_Mask	1117.9672187136246 ns/iter	1124.0996049882363 ns/iter	0.99
Micro_2020_12_Simple_Output_Annotations	81974.04364104787 ns/iter	81730.50097891541 ns/iter	1.00
Micro_2020_12_Simple_Output_Annotation_Dropping	1983132.4322038423 ns/iter	1939105.5983378768 ns/iter	1.02
Micro_2020_12_Compile_NonCircular_Shared_Refs	1081395.5963019764 ns/iter	1083801.3131782392 ns/iter	1.00
Micro_2020_12_Exhaustive_Deep_Numeric	8835.471071113263 ns/iter	8847.023522446563 ns/iter	1.00
Micro_2020_12_Exhaustive_Deep_Numeric_SimpleOutput	54795.14220544926 ns/iter	52373.548266388505 ns/iter	1.05
Micro_2020_12_Exhaustive_Deep_Numeric_TraceOutput	25728.909812095604 ns/iter	25702.897584399718 ns/iter	1.00
Micro_2020_12_Exhaustive_Deep_Numeric_Fail	153.2701678242368 ns/iter	152.53454478865223 ns/iter	1.00
Micro_2020_12_Exhaustive_Deep_Numeric_Fail_SimpleOutput	3330.1541661921256 ns/iter	3234.1717487548385 ns/iter	1.03
Micro_2019_09_Unevaluated_Properties	30.588120903278284 ns/iter	31.820377749667756 ns/iter	0.96
Micro_2019_09_Compile_Wrap	299947097.0000857 ns/iter	324266928.00004274 ns/iter	0.93
Micro_Draft7_If_Then_Else	24.026794942304498 ns/iter	24.02543412501401 ns/iter	1.00
Micro_Draft6_Property_Names	149.58351880196207 ns/iter	149.42957551992993 ns/iter	1.00
Micro_Draft6_Compile_FHIR	13802483303.000145 ns/iter	13869898902.000158 ns/iter	1.00
Micro_Draft4_Meta_1_No_Callback	173.55962337224346 ns/iter	173.43144507032972 ns/iter	1.00
Micro_Draft4_Required_Properties	890.765704469319 ns/iter	890.3860442001439 ns/iter	1.00
Micro_Draft4_Many_Optional_Properties_Minimal_Match	21.519620506973187 ns/iter	22.351361843512564 ns/iter	0.96
Micro_Draft4_Few_Optional_Properties_Minimal_Match	15.909649352597208 ns/iter	15.910657255000713 ns/iter	1.00
Micro_Draft4_Items_Schema	359.57334203036726 ns/iter	351.72257426607416 ns/iter	1.02
Micro_Draft4_Nested_Object	1.5590614561194214 ns/iter	1.6540119340947366 ns/iter	0.94
Micro_Draft4_Properties_Triad_Optional	410.93048534470296 ns/iter	461.7734010632871 ns/iter	0.89
Micro_Draft4_Properties_Triad_Closed	337.71678618883954 ns/iter	370.57279803476325 ns/iter	0.91
Micro_Draft4_Properties_Triad_Required	517.7191252044269 ns/iter	445.9646268718953 ns/iter	1.16
Micro_Draft4_Properties_Closed	66.54357867290366 ns/iter	66.42919397831542 ns/iter	1.00
Micro_Draft4_Non_Recursive_Ref	9.032755685734378 ns/iter	9.084409251241468 ns/iter	0.99
Micro_Draft4_Pattern_Properties_True	207.5070170153318 ns/iter	202.86938069165865 ns/iter	1.02
Micro_Draft4_Ref_To_Single_Property	16.27001907721118 ns/iter	16.23774715404177 ns/iter	1.00
Micro_Draft4_Additional_Properties_Type	14.330122946000493 ns/iter	14.746924844061585 ns/iter	0.97
Micro_Draft4_Nested_Oneof	63.629436249112096 ns/iter	63.542268574396026 ns/iter	1.00
Micro_Draft4_Short_Enum	14.90828318225588 ns/iter	14.66234522561511 ns/iter	1.02
Micro_Draft4_Long_Enum	12.817362578991194 ns/iter	12.880885466104933 ns/iter	1.00
Micro_Draft4_Long_Enum_Short_Strings	14.636444099884653 ns/iter	14.647373525158478 ns/iter	1.00
Micro_Draft4_Type_Object	4.364085637776178 ns/iter	4.3589731364780295 ns/iter	1.00
Micro_Draft4_Ref_Single_100	2456841.9260561657 ns/iter	2446190.6713292967 ns/iter	1.00
Micro_Draft4_Compile_Ref_Many_Nested	1784749.093908408 ns/iter	1778928.8527917147 ns/iter	1.00
Micro_Draft4_Compile_Wrap	2585686.4473684924 ns/iter	2558383.7472527944 ns/iter	1.01
JSONLD_Catalog_Simple	25687758.9259198 ns/iter	29665550.565220617 ns/iter	0.87
JSONLD_Catalog_Override_Agreeing	26295476.653847173 ns/iter	29010432.47618959 ns/iter	0.91
JSONLD_Catalog_Override_Shadowing	37336543.05263276 ns/iter	42699156.68750457 ns/iter	0.87
E2E_Compiler_adaptivecard	61572190.54545169 ns/iter	64469081.799984455 ns/iter	0.96
E2E_Compiler_ansible_meta	27978342.15999501 ns/iter	28432929.99999903 ns/iter	0.98
E2E_Compiler_aws_cdk	365585.3933997448 ns/iter	369951.3910289942 ns/iter	0.99
E2E_Compiler_babelrc	2505708.3892858955 ns/iter	2504402.32616548 ns/iter	1.00
E2E_Compiler_clang_format	17524897.674996965 ns/iter	17545356.47500006 ns/iter	1.00
E2E_Compiler_cmake_presets	26349086.148153827 ns/iter	26495195.346158724 ns/iter	0.99
E2E_Compiler_code_climate	2527357.326164822 ns/iter	2520370.1043161387 ns/iter	1.00
E2E_Compiler_cql2	13892086.440000638 ns/iter	14217171.020405028 ns/iter	0.98
E2E_Compiler_cspell	23264572.900006898 ns/iter	23261927.733331807 ns/iter	1.00
E2E_Compiler_cypress	4041340.1321835592 ns/iter	4025515.7586211 ns/iter	1.00
E2E_Compiler_deno	7833495.842696517 ns/iter	7851357.775280121 ns/iter	1.00
E2E_Compiler_dependabot	3780232.7043009037 ns/iter	3799517.63243298 ns/iter	0.99
E2E_Compiler_draft_04	2693989.718146795 ns/iter	2705496.3513512267 ns/iter	1.00
E2E_Compiler_fabric_mod	4603985.809210013 ns/iter	4600351.467104247 ns/iter	1.00
E2E_Compiler_geojson	22005883.09375462 ns/iter	22246680.870965414 ns/iter	0.99
E2E_Compiler_gitpod_configuration	4866751.145834137 ns/iter	4888548.538461705 ns/iter	1.00
E2E_Compiler_helm_chart_lock	670279.7351246752 ns/iter	665317.2789576966 ns/iter	1.01
E2E_Compiler_importmap	271848.68779158226 ns/iter	273514.26434991125 ns/iter	0.99
E2E_Compiler_jasmine	1321108.4564396674 ns/iter	1331249.8003805291 ns/iter	0.99
E2E_Compiler_jshintrc	4127730.082354099 ns/iter	4156114.8869039332 ns/iter	0.99
E2E_Compiler_jsconfig	20830112.823526125 ns/iter	21071261.666672096 ns/iter	0.99
E2E_Compiler_krakend	136703136.99999496 ns/iter	147449001.79998693 ns/iter	0.93
E2E_Compiler_lazygit	32024560.954544708 ns/iter	32599941.333327956 ns/iter	0.98
E2E_Compiler_lerna	1578162.6821344637 ns/iter	1571388.5325843305 ns/iter	1.00
E2E_Compiler_nest_cli	7162397.887754548 ns/iter	7182512.306121215 ns/iter	1.00
E2E_Compiler_omc	161540565.4999904 ns/iter	171443256.00003186 ns/iter	0.94
E2E_Compiler_omnisharp	6414805.715596665 ns/iter	6445072.037037234 ns/iter	1.00
E2E_Compiler_openapi	25580731.46427984 ns/iter	25953449.846156217 ns/iter	0.99
E2E_Compiler_pre_commit_hooks	5056680.811593573 ns/iter	5081802.181158257 ns/iter	1.00
E2E_Compiler_pulumi	3928278.82022461 ns/iter	3937702.0338985864 ns/iter	1.00
E2E_Compiler_semantic_release	1506803.9248926302 ns/iter	1512961.894849753 ns/iter	1.00
E2E_Compiler_stale	1499950.9443255651 ns/iter	1504504.4946237863 ns/iter	1.00
E2E_Compiler_stylecop	5702114.032520121 ns/iter	5728927.918032807 ns/iter	1.00
E2E_Compiler_tmuxinator	1868791.5187170645 ns/iter	1880336.1693547536 ns/iter	0.99
E2E_Compiler_ui5	48285458.35714847 ns/iter	50113820.92857275 ns/iter	0.96
E2E_Compiler_ui5_manifest	318669830.4999709 ns/iter	367046589.99997264 ns/iter	0.87
E2E_Compiler_unreal_engine_uproject	4421708.578616263 ns/iter	4428284.132911899 ns/iter	1.00
E2E_Compiler_users_array	875030.7734263116 ns/iter	871436.5927319895 ns/iter	1.00
E2E_Compiler_vercel	12718333.23636532 ns/iter	12690135.272726946 ns/iter	1.00
E2E_Compiler_yamllint	338973.3312319016 ns/iter	341256.11062375695 ns/iter	0.99
E2E_Evaluator_adaptivecard	25141.587707659728 ns/iter	25543.186110134327 ns/iter	0.98
E2E_Evaluator_ansible_meta	291287.7654921661 ns/iter	280776.59143658524 ns/iter	1.04
E2E_Evaluator_aws_cdk	63202.02182538979 ns/iter	63619.46712958357 ns/iter	0.99
E2E_Evaluator_babelrc	117535.4559826837 ns/iter	116623.17786358367 ns/iter	1.01
E2E_Evaluator_cerebrum_criminal_case	1138638.5834709872 ns/iter	1163858.4342763019 ns/iter	0.98
E2E_Evaluator_clang_format	108551.81214508471 ns/iter	111101.93577694114 ns/iter	0.98
E2E_Evaluator_cmake_presets	4853173.440559423 ns/iter	5805988.491524889 ns/iter	0.84
E2E_Evaluator_code_climate	229176.41926908065 ns/iter	228577.78148760146 ns/iter	1.00
E2E_Evaluator_cql2	225790.6801529164 ns/iter	225052.4024312233 ns/iter	1.00
E2E_Evaluator_cspell	622074.9919571838 ns/iter	613793.8475770006 ns/iter	1.01
E2E_Evaluator_cypress	190467.88402271442 ns/iter	189526.7949407753 ns/iter	1.00
E2E_Evaluator_deno	361736.21237541316 ns/iter	358730.3942798086 ns/iter	1.01
E2E_Evaluator_dependabot	376524.66666668066 ns/iter	377290.63481610996 ns/iter	1.00
E2E_Evaluator_draft_04	9609651.134327063 ns/iter	12558018.457623849 ns/iter	0.77
E2E_Evaluator_fabric_mod	785210.1959459818 ns/iter	788413.2979684008 ns/iter	1.00
E2E_Evaluator_geojson	18483489.789470073 ns/iter	20617379.392855845 ns/iter	0.90
E2E_Evaluator_gitpod_configuration	270081.2006319275 ns/iter	272133.1136719129 ns/iter	0.99
E2E_Evaluator_helm_chart_lock	323628.47625572863 ns/iter	330484.94842007186 ns/iter	0.98
E2E_Evaluator_importmap	55285.23523372531 ns/iter	53322.212546819006 ns/iter	1.04
E2E_Evaluator_jasmine	124059.64005136702 ns/iter	122604.63449728473 ns/iter	1.01
E2E_Evaluator_jshintrc	1096098.5320813586 ns/iter	1094264.1294850446 ns/iter	1.00
E2E_Evaluator_jsconfig	445022.516782855 ns/iter	441060.6666667418 ns/iter	1.01
E2E_Evaluator_krakend	232673.78559632887 ns/iter	221667.48965298064 ns/iter	1.05
E2E_Evaluator_lazygit	149545.61626425924 ns/iter	147773.6997434005 ns/iter	1.01
E2E_Evaluator_lerna	167585.52461012627 ns/iter	169039.97940614138 ns/iter	0.99
E2E_Evaluator_nest_cli	209524.1939898501 ns/iter	209808.30140255965 ns/iter	1.00
E2E_Evaluator_omc	23035.7496430138 ns/iter	23412.26652529517 ns/iter	0.98
E2E_Evaluator_omnisharp	598054.0543103432 ns/iter	596327.4905336373 ns/iter	1.00
E2E_Evaluator_openapi	16238108.883721165 ns/iter	19068729.540543467 ns/iter	0.85
E2E_Evaluator_pre_commit_hooks	553472.3240446432 ns/iter	561081.0879297395 ns/iter	0.99
E2E_Evaluator_pulumi	769563.3684794871 ns/iter	779801.7444690786 ns/iter	0.99
E2E_Evaluator_semantic_release	110631.02066503184 ns/iter	107735.18624459491 ns/iter	1.03
E2E_Evaluator_stale	175331.14013062397 ns/iter	182630.15665120955 ns/iter	0.96
E2E_Evaluator_stylecop	334670.77298311476 ns/iter	325873.65641515504 ns/iter	1.03
E2E_Evaluator_tmuxinator	111945.86979167073 ns/iter	109621.26795840767 ns/iter	1.02
E2E_Evaluator_ui5	608359.8791304252 ns/iter	596134.8025531597 ns/iter	1.02
E2E_Evaluator_ui5_manifest	3051698.6060613375 ns/iter	3184296.1666659885 ns/iter	0.96
E2E_Evaluator_unreal_engine_uproject	489313.7724089755 ns/iter	495977.66332389985 ns/iter	0.99
E2E_Evaluator_users_array	677890.7420901206 ns/iter	665990.9088262568 ns/iter	1.02
E2E_Evaluator_vercel	334866.3312378336 ns/iter	334960.35057477094 ns/iter	1.00
E2E_Evaluator_yamllint	13206.504494884251 ns/iter	12993.30324210202 ns/iter	1.02
Schema_Format_ISO_Language_To_JSON	112558618.0000179 ns/iter	126242455.39996083 ns/iter	0.89
Schema_Bundle_Meta_2020_12	1754547.392952431 ns/iter	1755061.4773800413 ns/iter	1.00
Schema_Frame_WoT_References	5243128.28571411 ns/iter	5264068.552239236 ns/iter	1.00
Schema_Frame_OMC_References	22334826.548389643 ns/iter	24225515.900002394 ns/iter	0.92
Schema_Frame_OMC_Locations	20723025.69696617 ns/iter	21740694.100003563 ns/iter	0.95
Schema_Frame_ISO_Language_Locations	104319197.5714178 ns/iter	111974558.14284955 ns/iter	0.93
Schema_Frame_KrakenD_References	39344764.23530535 ns/iter	44333388.43749368 ns/iter	0.89
Schema_Frame_KrakenD_Reachable	552362898.9999452 ns/iter	576550769.9998125 ns/iter	0.96
Schema_Frame_ISO_Language_Locations_To_JSON	131607881.39998658 ns/iter	141812946.20002517 ns/iter	0.93
Schema_Frame_Many_Resources_References	386525355.0000034 ns/iter	411122987.99998826 ns/iter	0.94
Schema_Iterator_ISO_Language	4072266.686045813 ns/iter	3966711.06285664 ns/iter	1.03
Schema_Tracker_ISO_Language	5122796.374999272 ns/iter	5494780.969465448 ns/iter	0.93
Schema_Tracker_ISO_Language_To_JSON	21322142.78787725 ns/iter	22257755.03124794 ns/iter	0.96

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

JavaScript (linux/gcc)

Details

Benchmark suite	Current: a32ae02	Previous: 692cb64	Ratio
E2E_Evaluator_adaptivecard	121381 ns	119624 ns	1.01
E2E_Evaluator_ansible-meta	530230 ns	549354 ns	0.97
E2E_Evaluator_aws-cdk	146196 ns	152431 ns	0.96
E2E_Evaluator_babelrc	362637 ns	364594 ns	0.99
E2E_Evaluator_cerebrum-criminal-case	2542644 ns	2535939 ns	1.00
E2E_Evaluator_clang-format	256505 ns	263359 ns	0.97
E2E_Evaluator_cmake-presets	19527062 ns	19802239 ns	0.99
E2E_Evaluator_code-climate	525823 ns	537735 ns	0.98
E2E_Evaluator_cql2	1316893 ns	1281507 ns	1.03
E2E_Evaluator_cspell	1471678 ns	1556844 ns	0.95
E2E_Evaluator_cypress	793674 ns	849331 ns	0.93
E2E_Evaluator_deno	1083626 ns	1079872 ns	1.00
E2E_Evaluator_dependabot	1122003 ns	1187198 ns	0.95
E2E_Evaluator_draft-04	22167353 ns	24529521 ns	0.90
E2E_Evaluator_fabric-mod	2357746 ns	2436300 ns	0.97
E2E_Evaluator_geojson	24277494 ns	26412116 ns	0.92
E2E_Evaluator_gitpod-configuration	730718 ns	737255 ns	0.99
E2E_Evaluator_helm-chart-lock	982472 ns	1014768 ns	0.97
E2E_Evaluator_importmap	274128 ns	279805 ns	0.98
E2E_Evaluator_jasmine	295038 ns	309457 ns	0.95
E2E_Evaluator_jsconfig	1265670 ns	1396317 ns	0.91
E2E_Evaluator_jshintrc	2749183 ns	2900685 ns	0.95
E2E_Evaluator_krakend	671642 ns	684594 ns	0.98
E2E_Evaluator_lazygit	521108 ns	538833 ns	0.97
E2E_Evaluator_lerna	455893 ns	468243 ns	0.97
E2E_Evaluator_nest-cli	591787 ns	622367 ns	0.95
E2E_Evaluator_omc	118800 ns	119219 ns	1.00
E2E_Evaluator_omnisharp	1407239 ns	1474937 ns	0.95
E2E_Evaluator_openapi	43049051 ns	46726897 ns	0.92
E2E_Evaluator_pre-commit-hooks	1966308 ns	2077898 ns	0.95
E2E_Evaluator_pulumi	1906466 ns	1947241 ns	0.98
E2E_Evaluator_semantic-release	249726 ns	263101 ns	0.95
E2E_Evaluator_stale	404369 ns	394602 ns	1.02
E2E_Evaluator_stylecop	1149304 ns	1175460 ns	0.98
E2E_Evaluator_tmuxinator	231366 ns	241974 ns	0.96
E2E_Evaluator_ui5	2227637 ns	2268763 ns	0.98
E2E_Evaluator_ui5-manifest	10336927 ns	12018178 ns	0.86
E2E_Evaluator_unreal-engine-uproject	1496779 ns	1522252 ns	0.98
E2E_Evaluator_users-array	3124757 ns	3176777 ns	0.98
E2E_Evaluator_vercel	990478 ns	1035171 ns	0.96
E2E_Evaluator_yamllint	26526 ns	25619 ns	1.04

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

Benchmark (macos/llvm)

Details

Benchmark suite	Current: a32ae02	Previous: 33c3ad0	Ratio
Schema_Iterator_ISO_Language	3013386.454182704 ns/iter	2493818.9295302522 ns/iter	1.21
Schema_Tracker_ISO_Language	4648139.166665714 ns/iter	4687082.116788588 ns/iter	0.99
Schema_Tracker_ISO_Language_To_JSON	19523153.571429115 ns/iter	18162913.866664592 ns/iter	1.07
Schema_Frame_WoT_References	8556833.851852063 ns/iter	8039518.834782386 ns/iter	1.06
Schema_Frame_OMC_References	30154920.30435187 ns/iter	34017893.954548776 ns/iter	0.89
Schema_Frame_OMC_Locations	26474557.111104902 ns/iter	34429730.26316347 ns/iter	0.77
Schema_Frame_ISO_Language_Locations	100244952.28568803 ns/iter	120408430.49999239 ns/iter	0.83
Schema_Frame_KrakenD_References	52119496.769224636 ns/iter	54672555.500000425 ns/iter	0.95
Schema_Frame_KrakenD_Reachable	561081875.9998893 ns/iter	519918041.99997205 ns/iter	1.08
Schema_Frame_ISO_Language_Locations_To_JSON	77385135.37501036 ns/iter	70696046.22223677 ns/iter	1.09
Schema_Frame_Many_Resources_References	374402708.50004256 ns/iter	452764083.5000284 ns/iter	0.83
Schema_Bundle_Meta_2020_12	1595946.8313240106 ns/iter	2267575.480143913 ns/iter	0.70
Schema_Format_ISO_Language_To_JSON	93380928.7142984 ns/iter	126306137.14294927 ns/iter	0.74
E2E_Compiler_adaptivecard	70896124.9999902 ns/iter	77580339.28572722 ns/iter	0.91
E2E_Compiler_ansible_meta	35019233.34999901 ns/iter	29699533.71428433 ns/iter	1.18
E2E_Compiler_aws_cdk	349823.5948803763 ns/iter	313434.3013638707 ns/iter	1.12
E2E_Compiler_babelrc	2562436.823104641 ns/iter	2539311.777777837 ns/iter	1.01
E2E_Compiler_clang_format	18898617.424245898 ns/iter	21285226.00000079 ns/iter	0.89
E2E_Compiler_cmake_presets	30529655.318177488 ns/iter	28880193.826086286 ns/iter	1.06
E2E_Compiler_code_climate	2753215.4075468597 ns/iter	2601718.70881203 ns/iter	1.06
E2E_Compiler_cql2	14251620.24999984 ns/iter	14312922.872341972 ns/iter	1.00
E2E_Compiler_cspell	27049613.63635696 ns/iter	26368452.185183574 ns/iter	1.03
E2E_Compiler_cypress	4852024.284768326 ns/iter	4497194.89516141 ns/iter	1.08
E2E_Compiler_deno	9185683.336842263 ns/iter	8352078.282828785 ns/iter	1.10
E2E_Compiler_dependabot	4991540.209790923 ns/iter	5207819.169997947 ns/iter	0.96
E2E_Compiler_draft_04	3294767.9512199475 ns/iter	2975474.736220306 ns/iter	1.11
E2E_Compiler_fabric_mod	5586542.910447264 ns/iter	5187704.710144079 ns/iter	1.08
E2E_Compiler_geojson	28866083.333336215 ns/iter	27996337.499996573 ns/iter	1.03
E2E_Compiler_gitpod_configuration	4827127.467106457 ns/iter	5306912.910000392 ns/iter	0.91
E2E_Compiler_helm_chart_lock	603304.2727272417 ns/iter	793878.9756345674 ns/iter	0.76
E2E_Compiler_importmap	251477.28243140312 ns/iter	363906.5610267769 ns/iter	0.69
E2E_Compiler_jasmine	1258278.8245902625 ns/iter	1537241.279069624 ns/iter	0.82
E2E_Compiler_jshintrc	3918946.0000000284 ns/iter	5799061.475410453 ns/iter	0.68
E2E_Compiler_jsconfig	23144541.66666893 ns/iter	27360760.416665167 ns/iter	0.85
E2E_Compiler_krakend	180221521.0000213 ns/iter	176522485.99999136 ns/iter	1.02
E2E_Compiler_lazygit	36201449.05556168 ns/iter	45100446.11764469 ns/iter	0.80
E2E_Compiler_lerna	1468698.1270714742 ns/iter	1766646.135265634 ns/iter	0.83
E2E_Compiler_nest_cli	6969027.52678495 ns/iter	7879791.661539807 ns/iter	0.88
E2E_Compiler_omc	163409385.50001737 ns/iter	191666319.6667135 ns/iter	0.85
E2E_Compiler_omnisharp	6251360.638654523 ns/iter	7460114.773583934 ns/iter	0.84
E2E_Compiler_openapi	21992394.357141618 ns/iter	27930586.956521753 ns/iter	0.79
E2E_Compiler_pre_commit_hooks	4241246.913579743 ns/iter	4703886.255474173 ns/iter	0.90
E2E_Compiler_pulumi	3182090.4082566034 ns/iter	3842306.4270839784 ns/iter	0.83
E2E_Compiler_semantic_release	1198675.9416962638 ns/iter	1860686.9693882095 ns/iter	0.64
E2E_Compiler_stale	1248579.235621737 ns/iter	1804334.2262999516 ns/iter	0.69
E2E_Compiler_stylecop	5278028.474821041 ns/iter	7500307.7247693995 ns/iter	0.70
E2E_Compiler_tmuxinator	1455045.0126851301 ns/iter	1797125.198564707 ns/iter	0.81
E2E_Compiler_ui5	49544943.49999549 ns/iter	77825189.88890135 ns/iter	0.64
E2E_Compiler_ui5_manifest	310991082.9999762 ns/iter	407760875.0000081 ns/iter	0.76
E2E_Compiler_unreal_engine_uproject	4329811.369565871 ns/iter	5243627.909999304 ns/iter	0.83
E2E_Compiler_users_array	833895.414062472 ns/iter	916284.2832279382 ns/iter	0.91
E2E_Compiler_vercel	12559415.210529424 ns/iter	16611526.240741046 ns/iter	0.76
E2E_Compiler_yamllint	253241.76166727362 ns/iter	351432.9916951566 ns/iter	0.72
E2E_Evaluator_adaptivecard	20406.94070828296 ns/iter	27878.46766555078 ns/iter	0.73
E2E_Evaluator_ansible_meta	171276.87354038263 ns/iter	278262.4479472063 ns/iter	0.62
E2E_Evaluator_aws_cdk	37038.55202651898 ns/iter	59807.29580001025 ns/iter	0.62
E2E_Evaluator_babelrc	85019.4966101596 ns/iter	112110.87454212549 ns/iter	0.76
E2E_Evaluator_cerebrum_criminal_case	767080.540425526 ns/iter	1017291.7947932632 ns/iter	0.75
E2E_Evaluator_clang_format	99248.56829751362 ns/iter	124454.02941670228 ns/iter	0.80
E2E_Evaluator_cmake_presets	3426996.064676114 ns/iter	4152818.8622744563 ns/iter	0.83
E2E_Evaluator_code_climate	132027.3357954489 ns/iter	187320.07707457728 ns/iter	0.70
E2E_Evaluator_cql2	182403.7205817931 ns/iter	227712.00675436304 ns/iter	0.80
E2E_Evaluator_cspell	478679.8744257578 ns/iter	644296.1028706318 ns/iter	0.74
E2E_Evaluator_cypress	144191.31328984417 ns/iter	193327.06926799525 ns/iter	0.75
E2E_Evaluator_deno	237621.16511548366 ns/iter	461940.48328067857 ns/iter	0.51
E2E_Evaluator_dependabot	258197.26388890998 ns/iter	292781.64148471697 ns/iter	0.88
E2E_Evaluator_draft_04	5390096.80000023 ns/iter	5655381.436362074 ns/iter	0.95
E2E_Evaluator_fabric_mod	485717.4633635627 ns/iter	517991.3749998377 ns/iter	0.94
E2E_Evaluator_geojson	9132503.472222375 ns/iter	10563978.618422784 ns/iter	0.86
E2E_Evaluator_gitpod_configuration	178664.49885229967 ns/iter	184526.40577507584 ns/iter	0.97
E2E_Evaluator_helm_chart_lock	251193.12286933398 ns/iter	277324.8538011408 ns/iter	0.91
E2E_Evaluator_importmap	38753.24209407363 ns/iter	45646.400910382145 ns/iter	0.85
E2E_Evaluator_jasmine	98325.94310920282 ns/iter	90351.89887214535 ns/iter	1.09
E2E_Evaluator_jshintrc	1030022.58620673 ns/iter	927338.7963737126 ns/iter	1.11
E2E_Evaluator_jsconfig	314788.5830325337 ns/iter	322559.03299373214 ns/iter	0.98
E2E_Evaluator_krakend	140022.3840094173 ns/iter	152093.71143563517 ns/iter	0.92
E2E_Evaluator_lazygit	97021.33619972912 ns/iter	111740.92175161843 ns/iter	0.87
E2E_Evaluator_lerna	124965.39053868294 ns/iter	118806.69417928261 ns/iter	1.05
E2E_Evaluator_nest_cli	136709.92553398325 ns/iter	133613.6873239292 ns/iter	1.02
E2E_Evaluator_omc	15602.43435415998 ns/iter	17812.85703383764 ns/iter	0.88
E2E_Evaluator_omnisharp	493367.9210702953 ns/iter	445111.62397372077 ns/iter	1.11
E2E_Evaluator_openapi	12042402.597400086 ns/iter	9071434.37500224 ns/iter	1.33
E2E_Evaluator_pre_commit_hooks	386152.88635006483 ns/iter	388768.7657417454 ns/iter	0.99
E2E_Evaluator_pulumi	547892.6279433186 ns/iter	582572.5859766708 ns/iter	0.94
E2E_Evaluator_semantic_release	73961.59602719641 ns/iter	75614.56705981288 ns/iter	0.98
E2E_Evaluator_stale	122829.62002892705 ns/iter	127768.68116802962 ns/iter	0.96
E2E_Evaluator_stylecop	203093.49554729086 ns/iter	206611.21914827093 ns/iter	0.98
E2E_Evaluator_tmuxinator	70352.99888899855 ns/iter	75368.55017578183 ns/iter	0.93
E2E_Evaluator_ui5	473039.41875404597 ns/iter	387529.5613477691 ns/iter	1.22
E2E_Evaluator_ui5_manifest	2433064.980609201 ns/iter	1832615.98746103 ns/iter	1.33
E2E_Evaluator_unreal_engine_uproject	499937.1207524296 ns/iter	473384.8544987698 ns/iter	1.06
E2E_Evaluator_users_array	562530.6619600886 ns/iter	652424.9448123517 ns/iter	0.86
E2E_Evaluator_vercel	276887.8313807203 ns/iter	328233.6950233366 ns/iter	0.84
E2E_Evaluator_yamllint	7775.827613780123 ns/iter	7955.109046700212 ns/iter	0.98
JSONLD_Catalog_Simple	28898268.319999263 ns/iter	28038640.300004166 ns/iter	1.03
JSONLD_Catalog_Override_Agreeing	35279183.896552145 ns/iter	24597107.346153278 ns/iter	1.43
JSONLD_Catalog_Override_Shadowing	44598817.476190515 ns/iter	30983364.13042647 ns/iter	1.44
Micro_Draft4_Meta_1_No_Callback	184.34790725727927 ns/iter	156.05716656879233 ns/iter	1.18
Micro_Draft4_Required_Properties	918.0353613096316 ns/iter	747.6373749699919 ns/iter	1.23
Micro_Draft4_Many_Optional_Properties_Minimal_Match	18.597432194501167 ns/iter	23.790743471362543 ns/iter	0.78
Micro_Draft4_Few_Optional_Properties_Minimal_Match	10.188568920982625 ns/iter	10.914446126455502 ns/iter	0.93
Micro_Draft4_Items_Schema	292.4851655019794 ns/iter	460.64208053678544 ns/iter	0.63
Micro_Draft4_Nested_Object	1.366437475761113 ns/iter	1.1560208637740754 ns/iter	1.18
Micro_Draft4_Properties_Triad_Optional	231.49120511966706 ns/iter	233.25364598350598 ns/iter	0.99
Micro_Draft4_Properties_Triad_Closed	199.11644342302174 ns/iter	187.36615576023388 ns/iter	1.06
Micro_Draft4_Properties_Triad_Required	249.57013967411842 ns/iter	249.61803113018593 ns/iter	1.00
Micro_Draft4_Properties_Closed	52.24167500000476 ns/iter	51.05524159998822 ns/iter	1.02
Micro_Draft4_Non_Recursive_Ref	13.674556761260531 ns/iter	18.78178855474865 ns/iter	0.73
Micro_Draft4_Pattern_Properties_True	179.37845626664588 ns/iter	180.18228942867225 ns/iter	1.00
Micro_Draft4_Ref_To_Single_Property	10.303360302258138 ns/iter	9.202323918700534 ns/iter	1.12
Micro_Draft4_Additional_Properties_Type	34.60630164634828 ns/iter	28.543097263716593 ns/iter	1.21
Micro_Draft4_Nested_Oneof	43.30187574453949 ns/iter	36.42809044567002 ns/iter	1.19
Micro_Draft4_Short_Enum	6.820766819036745 ns/iter	5.656479299263307 ns/iter	1.21
Micro_Draft4_Long_Enum	11.245371411579177 ns/iter	10.390956256170497 ns/iter	1.08
Micro_Draft4_Long_Enum_Short_Strings	8.276212283180591 ns/iter	7.76309579109419 ns/iter	1.07
Micro_Draft4_Type_Object	3.5031156927098324 ns/iter	2.4313064452174564 ns/iter	1.44
Micro_Draft4_Ref_Single_100	2551969.792857075 ns/iter	1947553.6045845149 ns/iter	1.31
Micro_Draft4_Compile_Ref_Many_Nested	1505111.6016598088 ns/iter	1382185.6218813616 ns/iter	1.09
Micro_Draft4_Compile_Wrap	2866911.496350413 ns/iter	2138794.41317416 ns/iter	1.34
Micro_Draft6_Property_Names	111.34373483766034 ns/iter	101.1093344646047 ns/iter	1.10
Micro_Draft6_Compile_FHIR	14528030707.99996 ns/iter	24990771374.999893 ns/iter	0.58
Micro_Draft7_If_Then_Else	18.89710592527042 ns/iter	18.456868660348846 ns/iter	1.02
Micro_2019_09_Unevaluated_Properties	32.39870690769923 ns/iter	28.895509906315198 ns/iter	1.12
Micro_2019_09_Compile_Wrap	327129270.9999898 ns/iter	315280416.4999452 ns/iter	1.04
Micro_2020_12_Dynamic_Ref	730.9050300215438 ns/iter	670.1489914158085 ns/iter	1.09
Micro_2020_12_Dynamic_Ref_Single	2.7542417264478005 ns/iter	2.7083357608065297 ns/iter	1.02
Micro_2020_12_Simple_Output_Mask	945.2480225909799 ns/iter	1025.2417875602757 ns/iter	0.92
Micro_2020_12_Simple_Output_Annotations	63135.7428878261 ns/iter	81327.20523587812 ns/iter	0.78
Micro_2020_12_Simple_Output_Annotation_Dropping	1605163.5514021034 ns/iter	2026600.171511296 ns/iter	0.79
Micro_2020_12_Compile_NonCircular_Shared_Refs	825038.5794504237 ns/iter	1096713.752747 ns/iter	0.75
Micro_2020_12_Exhaustive_Deep_Numeric	5112.194979938045 ns/iter	5900.767290600117 ns/iter	0.87
Micro_2020_12_Exhaustive_Deep_Numeric_SimpleOutput	37202.46658495358 ns/iter	53922.92465460417 ns/iter	0.69
Micro_2020_12_Exhaustive_Deep_Numeric_TraceOutput	14895.568355454317 ns/iter	23724.615952211338 ns/iter	0.63
Micro_2020_12_Exhaustive_Deep_Numeric_Fail	93.87574456365682 ns/iter	120.90153007367137 ns/iter	0.78
Micro_2020_12_Exhaustive_Deep_Numeric_Fail_SimpleOutput	3498.0662715281233 ns/iter	6140.586250000979 ns/iter	0.57
TestSuite_Parse_WoT	31503723.129030585 ns/iter	81567773.11109585 ns/iter	0.39
Alterschema_Check_Readibility_ISO_Language_Set_3	274225520.99994844 ns/iter	516204290.9998945 ns/iter	0.53
Alterschema_Check_Readibility_OMC	78656708.29999089 ns/iter	115676453.12501895 ns/iter	0.68
Alterschema_Check_Readibility_KrakenD	787229792.000062 ns/iter	1133285624.9998713 ns/iter	0.69
Alterschema_Apply_Readibility_KrakenD	81106265958.99976 ns/iter	103559353875.00017 ns/iter	0.78
Alterschema_Check_Invalid_External_Refs	217469625.0000352 ns/iter	207017722.3334516 ns/iter	1.05

This comment was automatically generated by workflow using github-action-benchmark.

[github-actions]

JavaScript (macos/llvm)

Details

Benchmark suite	Current: a32ae02	Previous: 33c3ad0	Ratio
E2E_Evaluator_adaptivecard	119522 ns	169468 ns	0.71
E2E_Evaluator_ansible-meta	399224 ns	518752 ns	0.77
E2E_Evaluator_aws-cdk	131006 ns	151658 ns	0.86
E2E_Evaluator_babelrc	285099 ns	332514 ns	0.86
E2E_Evaluator_cerebrum-criminal-case	2069382 ns	1957812 ns	1.06
E2E_Evaluator_clang-format	226865 ns	217648 ns	1.04
E2E_Evaluator_cmake-presets	17651921 ns	17685365 ns	1.00
E2E_Evaluator_code-climate	503030 ns	531705 ns	0.95
E2E_Evaluator_cql2	1370242 ns	1341636 ns	1.02
E2E_Evaluator_cspell	1477555 ns	1664767 ns	0.89
E2E_Evaluator_cypress	737254 ns	1062939 ns	0.69
E2E_Evaluator_deno	833766 ns	1216629 ns	0.69
E2E_Evaluator_dependabot	810786 ns	1093230 ns	0.74
E2E_Evaluator_draft-04	16822528 ns	17293130 ns	0.97
E2E_Evaluator_fabric-mod	1658637 ns	1482074 ns	1.12
E2E_Evaluator_geojson	16888507 ns	18570097 ns	0.91
E2E_Evaluator_gitpod-configuration	521635 ns	588640 ns	0.89
E2E_Evaluator_helm-chart-lock	1036818 ns	995359 ns	1.04
E2E_Evaluator_importmap	264942 ns	292867 ns	0.90
E2E_Evaluator_jasmine	246379 ns	286663 ns	0.86
E2E_Evaluator_jsconfig	1117356 ns	1067130 ns	1.05
E2E_Evaluator_jshintrc	2615371 ns	2533398 ns	1.03
E2E_Evaluator_krakend	454684 ns	519325 ns	0.88
E2E_Evaluator_lazygit	302940 ns	329252 ns	0.92
E2E_Evaluator_lerna	339261 ns	359586 ns	0.94
E2E_Evaluator_nest-cli	501937 ns	463743 ns	1.08
E2E_Evaluator_omc	98991 ns	121734 ns	0.81
E2E_Evaluator_omnisharp	1049785 ns	1205438 ns	0.87
E2E_Evaluator_openapi	32691426 ns	28744792 ns	1.14
E2E_Evaluator_pre-commit-hooks	1405242 ns	1442535 ns	0.97
E2E_Evaluator_pulumi	1573705 ns	1625251 ns	0.97
E2E_Evaluator_semantic-release	191285 ns	201834 ns	0.95
E2E_Evaluator_stale	319323 ns	323129 ns	0.99
E2E_Evaluator_stylecop	739787 ns	832082 ns	0.89
E2E_Evaluator_tmuxinator	173203 ns	290480 ns	0.60
E2E_Evaluator_ui5	1860705 ns	2153906 ns	0.86
E2E_Evaluator_ui5-manifest	8697183 ns	10171104 ns	0.86
E2E_Evaluator_unreal-engine-uproject	1219685 ns	1357454 ns	0.90
E2E_Evaluator_users-array	2538320 ns	3466217 ns	0.73
E2E_Evaluator_vercel	799053 ns	968496 ns	0.83
E2E_Evaluator_yamllint	24893 ns	28941 ns	0.86

This comment was automatically generated by workflow using github-action-benchmark.