docs: Internals Book improvements (01)

[NickSdot]

This is switching the internals book to Markdown. The PR is formatting only and deliberately:

• makes no content changes
• leaves file extension untouched

Why Markdown?

• not yet another syntax contributors have to learn
• rendered correctly by default everywhere; including GitHub and IDEs
• keep diffs when moving/merging docs-old clean
• because we can; there is an official plugin for exactly that
• we can easily extend it later if we want to

This is a preflight in preparation for:

• docs: Internals Book improvements (02) #23369
• docs: Internals Book improvements (03) #23370
• docs: Internals Book improvements (04) #23371
• docs: Internals Book improvements (05) #23372
  until here it was more or less mechanical changes; they should be seen as preparation for the following 3 that change actual old contents.
• docs: Internals Book improvements (06) #23373
• docs: Internals Book improvements (07) #23374
• docs: Internals Book improvements (08) #23375

The work is split into multiple PRs to keep diffs comparable and focused; without splitting them git would mark quite literally everything as deleted and new files.

Preview

Branch: https://nicksdot.github.io/docs-restructure-01/
Final: https://nicksdot.github.io/docs-restructure-final/

Aside

I got multiple times asked to split my PRs into multiple smaller PRs. But I also got approached and told that it is annoying. Hence, I'd like to address that quick. Here splitting is essential because with the many moves and content changes in one PR the diff would simply not be reviewable. My serial PRs were needed because things either depended strongly on each other or touched too many files/extensions for one PR. That said, I consider the work that I have been doing over the last few weeks as "groundwork"; after this one I should be done with the bigger changes. So please kindly bear with me. :)

Related

#13338 (#13338 (comment))
#15939
#15940
#17761
#18409

[iluuu1994]

We've explicitly chosen reST over markdown. I don't see a point in switching to markdown. reST has more flexibility, e.g. see the horribly formatted markdown tables.

[NickSdot]

We've explicitly chosen reST over markdown. I don't see a point in switching to markdown. reST has more flexibility, e.g. see the horribly formatted markdown tables.

Alright, got it - that's not a biggie. It allows Markdown in rst; means I can just flip the file extensions back to rst so that we can use Markdown for all the standard stuff and whatever md is short of we can fall back to the non-Markdown syntax.

Could you please point me to the discussion where that was decided? Keen to adjust/revert whichever things were deemed not flexible (eg tables; what else?) enough. 🫡

Apart from that? First impression of everything else good?

Edit:
Which horribly formatted tables are you referring to? Spot checked some pages and the tables look actually the same.

[iluuu1994]

Could you please point me to the discussion where that was decided?

https://externals.io/message/122357#122361, basically.

Apart from that? First impression of everything else good?

I cannot identify a usable diff for review in this PR.

Which horribly formatted tables are you referring to? Spot checked some pages and the tables look actually the same.

This is referring to the code, not the rendered result. Markdown tables are a PITA to maintain by-hand, but also result in a huge diffs if the column width changes for all rows. Maybe a bit less of a problem with LLMs nowadays. But I do also remember having formatting issues with them, e.g. trying to insert breaks for better code readability, which were then unintentionally rendered in the HTML.

[NickSdot]

https://externals.io/message/122357#122361, basically.

Thanks! So I am taking it's mainly the tables and extendability, which I get.

I cannot identify a usable diff for review in this PR.

Brought the diff here down from ~3,500 to ~1,100 (actually easy to digest) lines. For the following PRs please find in each PR desc:

• the diff link for each step; I made it really so that each step has a clean diff.
• the link to GH pages for each branch

This is referring to the code, not the rendered result. Markdown tables are a PITA to maintain by-hand, but also result in a huge diffs if the column width changes for all rows. Maybe a bit less of a problem with LLMs nowadays. But I do also remember having formatting issues with them, e.g. trying to insert breaks for better code readability, which were then unintentionally rendered in the HTML.

That's a very fair point, you are right. I reverted the tables back; now using the MyST to keep the more convenient syntax. Gives us the best out of both worlds: the simpler Markdown for titles, inline code, prose and such, while keeping tables as you want them. 🫡

[iluuu1994]

I'm not at all convinced a switch is necessary. reST itself is very popular and well known. Much more so than this Markdown/reST hybrid. If you really want to push this forward, please discuss on the list to see how others feel. For me this is code churn.