Exanite's Tracking Issue

This tracks a bunch of miscellaneous tasks that I accumulated while working on Silk 3.
I'm open for anyone to work on these tasks, but please discuss with me in the Silk Discord first.

A task being here on this tracker does not mean it is guaranteed to be completed.
This tracker's primary goal is so that we do not accidentally forget about tasks and that any tasks not completed are intentionally left uncompleted.

In-progress

Tasks in progress will be moved to the corresponding pull request.

Uncategorized

This section contains informal reports, suggestions, or complaints about Silk 3.
I'm logging them here so they don't get forgotten about until I either open a formal issue or address them.

From Aqua:

• libSDL3.so is humongous (should not be nearly 14 MB)
  • Exanite: I checked my system version and that one is only 3 MB.
  • Edit - Exanite: Looks running file libSDL3.so says libSDL3.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, with debug_info, not stripped, so definitely because the binary was not stripped
• Silk 3 does not prefer its own native binaries? I.e., it prefers the system ones instead of the ones shipped by Silk.
• Go through the thread Aqua has made: https://discord.com/channels/521092042781229087/1512914692853465088

From Ethereal:

• SilkTouch generator takes an immense amount of RAM: https://discord.com/channels/521092042781229087/607634593201520651/1513592461493145601
  • I need to also check with Curin to see how much RAM their device has. If their device doesn't have enough RAM, this might explain why the Microsoft job takes so long (aside from it already processing a lot of data).
  • Run the Microsoft job locally and see how much RAM it takes on my device.

From Nolram (https://discord.com/channels/521092042781229087/607634593201520651/1524213012402147348):

• Add Imgui bindings
  • Nolram recommended https://github.com/dearimgui/dear_bindings over cimgui (https://github.com/cimgui/cimgui)
• Investigate whether direct P/Invoke and static linking works in Silk 3 and how we want to support it.
  • See: I'm not familiar with what direct P/Invoke means though
  • Specifically, it involves some configuration, providing static builds, and maybe switching over to LibraryImport over DllImport.

From dfkeenan:

• Add SDL shadercross support: https://discord.com/channels/521092042781229087/587346162802229298/1543100984270520340
  • My opinion on this: Shadercross is worth doing, worth considering the rest of the SDL addons as well, since Silk 3 leans much more heavily on SDL.
• "Trim the fat" from the generated bindings. Stuff like SDL's DUMMY_ENUM_VALUE should be removed
  • See https://discord.com/channels/521092042781229087/587346162802229298/1545694628643340328

From Curin:

• For D3D12, the Agility SDK is loaded from the System32 DLL instead of the working directory's D3D12 subdirectory. This means that Silk 2.x (report was for 2.x) and maybe Silk 3.0 (needs verification) cannot access Agility SDK features. See https://discord.com/channels/521092042781229087/587346162802229298/1549273531668373566
  • My comment: seems to be another search path issue. This seems similar to my issue in Silk 2 where system binaries are preferred over the vendored/local ones. I'm personally very interested in changing Silk so that it always prefers local binaries whenever possible.

From me (Exanite):

• OpenCL GetPlatformIDs, GetDeviceIDs, and likely more functions do not have singular GetPlatformID equivalents. This should be handled by our function transformer so I'm guessing the XML metadata is missing. See [3.0] Add OpenCL and OpenXR bindings #2590 (comment).
• Test projects should be added for all bindings. See [3.0] Add OpenCL and OpenXR bindings #2590 (comment)
• InlineArray structs should have implicit casts from strings/arrays, i.e., should behave like Ref. See [3.0] Add OpenCL and OpenXR bindings #2590 (comment)
• OpenXR types should be integrated with Silk.NET Maths
  • See for interface approach (relevant for certain types where ABI is different): [3.0] Add OpenCL and OpenXR bindings #2590 (comment)
  • See for remapping approach (relevant for types like vectors where ABI is the same): Use Silk.NET.Maths in OpenXR #506
  • See for my initial list of relevant types: [3.0] Add OpenCL and OpenXR bindings #2590 (comment)
• API object devirtualization:
  • See Perksey's comment and my comment here on potentially returning the concrete API class instead of the instance: [3.0] Add OpenCL and OpenXR bindings #2590 (comment)
    • These comments were in response to my todo comments in the OpenCL example program
  • Also see Perksey's idea to change the return something like GL<T> where T is an INativeContext to remove one level of callvirt: https://discord.com/channels/521092042781229087/587346162802229298/1541090497185980478

From me (Exanite), as part of my OpenCL example program:

Original comments:

• Expose INativeContext from the concrete API object where applicable (but not the interface)
• Discuss INativeContext API. Silk 3 is currently LoadFunction(functionName, libraryNameHint), but I liked how Silk 2 only required the function name.

High priority

Generator:

• ExtractEnumConstants does not preserve attributes placed on extracted constants (needs verification).

Bindings usage:

• Why don't we use the InlineArrayN<T> types provided by .NET? Should we use them?
  • Someone in the C# discord mentioned that these were added in .NET 10, which is what led to me add this task. This might be the reason we don't use them?
• OpenAL context should be disposable
  • This is noted by Perksey in the OpenAL.Tutorial001.HelloSound project. I need to figure out what this Dispose method should do exactly.
  • Check other API contexts as well for the same issue
    channels/521092042781229087/1514735149915439274/1519099432442794150
• The magic nullptr constant does not work. It fails to resolve to anything.
  • Apparently it works in the Silk 3 examples. Probably because it uses a ProjectReference instead of a PackageReference.
• The Vulkan bindings are double registering the LoaderInterface hook: https://discord.com/channels/521092042781229087/607634593201520651/1518220269519638599
  • I'm going to implement a temporary hack fix, but I don't believe it to be the correct fix.
    • We need a way to initialize an assembly consistently.
    • Currently there are multiple entrypoints and they aren't guaranteed to be called.
  • In Vulkan, we call RegisterHook in both Vk (defined in a manual file) and Vk.DllImport (defined in a generated file).
    • This causes a multiple registration error, which admittedly is my fault since I didn't realize OpenAL didn't do the same pattern.
    • Removing the duplicate call in Vk is the "fix" I mentioned above, but does not solve the underlying issue.
    • Static constructors are only invoked if the static constructor's type or members are accessed.
    • Notably, accessing Vk.DllImport does not trigger the static constructor for Vk, and vice versa.
    • Because we assume/require both static constructors to be called for proper behavior (Vk.DllImport registers the hook and is not extensible, so we have to register alternative shared library names in Vk), having only one of them be called is a problem.
• Add option to configure which vendor suffixes KhronosNonExclusiveVendor can identify. Eg: For OpenGL, this should be only ARB.
  • See conversation here: https://discord.com/channels/521092042781229087/587346162802229298/1531416150565453924
• Consider making function pointer structs work with nullptr. Seems like a nice QOL feature.
• OpenXR/CL/Vulkan: Add CurrentInstance/Device/Platform properties to interface partial.

Enums and other integral typedefs:

These have been proving to be consistently annoying to work with...
Vulkan and OpenXR have some special handling so this problem is less problematic, but still exists.

• I don't think the OpenGL bindings specify the method parameter types properly
  • Eg: void DrawElements(uint mode, uint count, uint type, void* indices);
  • This should use the enum type instead of uint.
• Vulkan: Some types are incorrect. uint is used instead of MaybeBool<uint>.
• OpenCL: Same issue. Tracked in the smoke test project as todos.

Documentation:

• Add Vulkan smoke test project
• Add bindings usage docs
  • Explain that the native docs for each API can be used since Silk doesn't really change how you use the API
  • Explain difference between bindings and high level APIs
    • Ie, high level APIs do not represent bindings so usually the native docs don't apply for those
  • Explain the different DSL types and magical constants (eg: nullptr)

Maintenance:

• Add SilkConstants class to generator
  • Ideally all magic numbers and constants are consolidated here and documented.
• Add <auto-generated/> comment to start of each generated file. This has the effect of disabling IDE/linter/etc checks, which means less noise and performance cost of working in the Silk repo.

Medium priority

Bindings:

• Use CLong where possible in the bindings.
  • For enums, use Constant<CLong, EnumHere>, and theoretically it should work.
  • See conversation here: [3.0] Add OpenCL and OpenXR bindings #2590 (comment)
• Consider implementing a IsProbablyABitmask function based on BuildTool's IsProbablyABitmask for identifying flags enums
  • Original comment: [3.0] Implement name-related improvements identified in previous Vulkan PR #2503 (comment)

Quality:

• Add some way of verifying our bindings are well polished.
  • These would mainly involve simple automated sanity checks that ensure our bindings our consistent. Could be done as a separate CLI project from SilkTouch.
  • Check for missing namespaces.
    • This was an issue discovered during [3.0] Split up ExtractNestedTyping and add tests for TransformHandles #2587. I've fixed one case of these, but haven't verified the rest of the generated files (since there's way too many to manually verify).
  • Check for spelling/casing issues? Identifiers like Sdl.PropGpuDeviceCreateDebugmodeBoolean are technically correct (native name is SDL_PROP_GPU_DEVICE_CREATE_DEBUGMODE_BOOLEAN, so Debugmode is "one" word), but we could have an audit tool that checks for these non-ideal cases so we can add manual overrides. Worth the effort? Probably not unless we care about perfecting the bindings.

Maintenance:

• ClangSharp switched from System.CommandLine to a custom command line argument parser. I noticed that we had verbatim copied code from ClangSharp for handling arguments in Silk 3, so we should update that.
  • This also lets us update System.CommandLine, which we are still using a beta version of.

Low priority

CI:

• Prevent native builds from triggering unnecessarily when the PR description is edited
  • Native builds should only trigger once. If the last commit in the PR is from the native builds workflow, it should not run again.
• Figure out how to make our CI workflows work with forks.
  • I did some testing here and got a proof-of-concept working: https://github.com/Exanite-Dev/GithubActionTesting
  • Note that we need to be careful about security here. This involves escalation of privileges.

Maintenance:

• Cross reference changelog and past issues for Silk 2 to ensure that we don't have the same bugs in Silk 3

High level utilities:

• Reimplement the Vulkan struct chaining API
• Add image format utils for Vulkan/etc

IDE / Metadata:

• Ensure SupportedApiProfiles attributes are correct
  • Properly resolve API profiles for Flags/FlagBits types
  • Prefer name from NativeName for resolving API profiles
• Consider optimizing NativeName attributes.
  • This should involve renaming StripAttributes to CleanupAttributes and adding an option in its config to optimize NativeName attributes.
  • Also see: [3.0] Implement name-related improvements identified in previous Vulkan PR #2503 (comment)