Cranelift modernize docs

cranelift/docs/add-a-clif-instruction.md

@@ -0,0 +1,104 @@
+# How to Add a New Instruction to CLIF
+
+This is a walkthrough of what's involved in adding a brand new opcode to
+Cranelift's target-independent IR. It doesn't cover adding a new instruction
+to a specific machine backend (the ISA-specific `MachInst`s);
+
+Before adding a new instruction, it's worth checking whether the semantics you want can already be expressed as a combination of existing instructions. CLIF tries to keep its instruction set fairly small; new opcodes are for cases where composing existing ones would be either impossible or noticeably worse for codegen.
+
+## 1. Declare the instruction
+
+Instructions are declared in the meta-language, not directly in Rust. The
+shared (ISA-independent) instructions are in
+`cranelift/codegen/meta/src/shared/instructions.rs`. A declaration looks like
+this:
+
+```rust
+ig.push(
+    Inst::new(
+        "iadd",
+        r#"
+    Wrapping integer addition: `a := x + y \pmod{2^B}`.
+
+    This instruction does not depend on the signed/unsigned interpretation
+    of the operands.
+    "#,
+        &formats.binary,
+    )
+    .operands_in(vec![Operand::new("x", Int), Operand::new("y", Int)])
+    .operands_out(vec![Operand::new("a", Int)])
+    .inst_builder_imm_method(true),
+);
+```
+
+The third argument to `Inst::new` is an instruction format, chosen from
+`cranelift/codegen/meta/src/shared/formats.rs` (`unary`, `binary`, `ternary`,
+`call`, `load`, `store`, `atomic_rmw`, and so on). The format determines what
+shape of operands the generated `InstructionData` variant has. That is to say,reuse an
+existing format if your instruction fits one, rather than adding a new one.
+
+If your instruction has effects beyond producing a result
+say so on the `Inst` builder with methods like `.can_trap()`, `.can_load()`,
+`.can_store()`, `.other_side_effects()`, `.branches()`, or `.call()` (see
+`cranelift/codegen/meta/src/cdsl/instructions.rs` for the full list). 
+
+## 2. Let the build regenerate the Rust and ISLE glue
+
+`cranelift/codegen/build.rs` runs the meta crate as part of the normal
+`cargo build`. From your instruction declaration it generates (into
+`OUT_DIR`, under `target/`):
+
+- A variant of the `Opcode` enum and of `InstructionData`.
+- Support code for the format you used (encoding, decoding, `Display`).
+- A method on the `InstBuilder` trait, so frontends can write
+  `builder.ins().your_new_instr(...)`.
+- `extern` declarations for the instruction in the auto-generated
+  `clif_lower.isle` and `clif_opt.isle` files, so it's immediately usable
+  as an ISLE pattern from lowering and mid-end rules. (See
+  [How ISLE is Integrated with Cranelift](isle-integration.md) for more on
+  those generated files.)
+
+You don't write any of this by hand, by running `cargo check` (or `build`)
+the compiler will tell you where you still need to put the new opcode in.
+
+## 3. Fill in the remaining match arms
+
+Because `Opcode` gained a new variant, several exhaustive `match`es elsewhere
+in the codebase will stop compiling until you handle it. In practice this
+means the compiler walks you through most of the checklist, but the usual
+places are:
+
+- **The interpreter**, `cranelift/interpreter/src/step.rs`, which matches
+  on `inst.opcode()` to give every instruction an execution semantics.
+  Without this, `test interpret` and `test run` won't be able to execute the
+  instruction.
+- **The verifier**, `cranelift/codegen/src/verifier/mod.rs`, if the new instruction introduces type rules or operand constraints that aren't already enforced by its format or type variables.
+- **Alias analysis and the egraph mid-end**
+  (`cranelift/codegen/src/alias_analysis.rs`,
+  `cranelift/codegen/src/egraph/`) if the instruction touches memory or
+  needs special handling to be optimized correctly. In practice, this is often an extension of the side-effect annotations defined in the first step.
+
+Pure instructions (no side effects, standard type variables) often
+need nothing beyond steps 1 and 2 plus a lowering rule in each backend that
+should support them.
+
+## 4. Add a lowering rule per backend
+
+A CLIF instruction with no backend that can lower it will hit ISLE's "no rule
+matched" panic the moment it's used. At a minimum, you should add a (rule (lower (your_instr ...)) ...) definition to each cranelift/codegen/src/isa/<arch>/lower.isle backend you intend to support. 
+
+## 5. Test it
+
+Add file tests under `cranelift/filetests/filetests/`. Use `test interpret` and
+`test run` to verify the execution semantics implemented in step 3, `test optimize`
+to validate any mid-end rewrites, and `test compile` (placed under
+`filetests/isa/<arch>/` for each target architecture)to test the backend lowering implemented in step 4. See
+[Testing Cranelift](testing.md) for how these test commands work.
+
+## Worked example
+
+[#12101](https://github.com/bytecodealliance/wasmtime/pull/12101), which
+added the `patchable_call` instruction, is a reasonably self-contained recent
+example that touches most of the above: the instruction declaration in
+`instructions.rs`, and lowering rules in both the x64 and aarch64
+`lower.isle` files.

cranelift/docs/isle-integration.md

@@ -108,8 +108,7 @@ Instead, these things should be handled by some combination of
 `cranelift/codegen/src/isa/<arch>/lower/isle.rs` or elsewhere).
 
 When an instruction modifies a register, both reading from it and writing to it,
-we should build an SSA view of that instruction that gets legalized via "move
-mitosis" by splitting a move out from the register.
+we should build an SSA view of that instruction that gets legalized by splitting a move out from the register.
 
 For example, on x86 the `add` instruction reads and writes its first operand:
 
@@ -125,18 +124,16 @@ Then, as an implementation detail of the facade, we emit moves as necessary:
 
     add a, b, c    ==>    mov a, b; add b, c
 
-We call the process of emitting these moves "move mitosis". For ISAs with
+For ISAs with
 ubiquitous use of modified registers and instructions in two-operand form, like
-x86, we implement move mitosis with methods on the ISA's `MachInst`. For other
+x86, we implement this with methods methods on the ISA's `MachInst`. For other
 ISAs that are RISCier and where modified registers are pretty rare, such as
-aarch64, we implement the handful of move mitosis special cases at the
+aarch64, we handle the handful of required cases at the
 `inst.isle` layer. Either way, the important thing is that the lowering rules
 remain pure.
 
 Finally, note that these moves are generally cleaned up by the register
-allocator's move coalescing, and move mitosis will eventually go away completely
-once we switch over to `regalloc2`, which takes instructions in SSA form
-directly as input.
+allocator's move coalescing.
 
 Instructions that implicitly operate on specific registers, or which require
 that certain operands be in certain registers, are handled similarly: the

cranelift/docs/testing.md

@@ -22,42 +22,44 @@ easier to provide substantial input functions for the compiler tests.
 
 File tests are `*.clif` files in the `filetests/` directory
 hierarchy. Each file has a header describing what to test followed by a number
-of input functions in the :doc:`Cranelift textual intermediate representation
-<ir>`:
+of input functions in the [Cranelift textual intermediate representation](ir.md):
 
-.. productionlist::
-    test_file     : test_header `function_list`
-    test_header   : test_commands (`isa_specs` | `settings`)
-    test_commands : test_command { test_command }
-    test_command  : "test" test_name { option } "\n"
+```
+test_file     : test_header function_list
+test_header   : test_commands (isa_specs | settings)
+test_commands : test_command { test_command }
+test_command  : "test" test_name { option } "\n"
+```
 
 The available test commands are described below.
 
 Many test commands only make sense in the context of a target instruction set
 architecture. These tests require one or more ISA specifications in the test
 header:
 
-.. productionlist::
-    isa_specs     : { [`settings`] isa_spec }
-    isa_spec      : "isa" isa_name { `option` } "\n"
+```
+isa_specs     : { [settings] isa_spec }
+isa_spec      : "isa" isa_name { option } "\n"
+```
 
 The options given on the `isa` line modify the ISA-specific settings defined in
-`cranelift-codegen/meta-python/isa/*/settings.py`.
+`cranelift/codegen/meta/src/isa/<arch>.rs`.
 
 All types of tests allow shared Cranelift settings to be modified:
 
-.. productionlist::
-    settings      : { setting }
-    setting       : "set" { option } "\n"
-    option        : flag | setting "=" value
+```
+settings      : { setting }
+setting       : "set" { option } "\n"
+option        : flag | setting "=" value
+```
 
 The shared settings available for all target ISAs are defined in
-`cranelift-codegen/meta-python/base/settings.py`.
+`cranelift/codegen/meta/src/shared/settings.rs`.
 
 The `set` lines apply settings cumulatively:
 
 ```
-    test legalizer
+    test optimize
     set opt_level=best
     set is_pic=1
     target riscv64
@@ -67,7 +69,7 @@ The `set` lines apply settings cumulatively:
     function %foo() {}
 ```
 
-This example will run the legalizer test twice. Both runs will have
+This example will run the optimize test twice. Both runs will have
 `opt_level=best`, but they will have different `is_pic` settings. The 32-bit
 run will also have the RISC-V specific flag `supports_m` disabled.
 
@@ -87,7 +89,7 @@ $ CRANELIFT_FILETESTS_THREADS=1 clif-util test path/to/file.clif
 
 Many of the test commands described below use *filecheck* to verify their
 output. Filecheck is a Rust implementation of the LLVM tool of the same name.
-See the `documentation <https://docs.rs/filecheck/>`_ for details of its syntax.
+See the [documentation](https://docs.rs/filecheck/) for details of its syntax.
 
 Comments in `.clif` files are associated with the entity they follow.
 This typically means an instruction or the whole function. Those tests that
@@ -212,71 +214,16 @@ both correct and complete.
 
 This test also sends the computed CFG post-order through filecheck.
 
-### `test legalizer`
-
-Legalize each function for the specified target ISA and run the resulting
-function through filecheck. This test command can be used to validate the
-encodings selected for legal instructions as well as the instruction
-transformations performed by the legalizer.
-
-### `test regalloc`
-
-Test the register allocator.
-
-First, each function is legalized for the specified target ISA. This is
-required for register allocation since the instruction encodings provide
-register class constraints to the register allocator.
-
-Second, the register allocator is run on the function, inserting spill code and
-assigning registers and stack slots to all values.
-
-The resulting function is then run through filecheck.
-
-
-### `test simple-gvn`
+### `test optimize`
 
-Test the simple GVN pass.
-
-The simple GVN pass is run on each function, and then results are run
-through filecheck.
-
-### `test licm`
-
-Test the LICM pass.
-
-The LICM pass is run on each function, and then results are run
-through filecheck.
-
-### `test dce`
-
-Test the DCE pass.
-
-The DCE pass is run on each function, and then results are run
-through filecheck.
-
-### `test shrink`
-
-Test the instruction shrinking pass.
-
-The shrink pass is run on each function, and then results are run
-through filecheck.
-
-### `test simple_preopt`
-
-Test the preopt pass.
-
-The preopt pass is run on each function, and then results are run
-through filecheck.
+Runs each function through the optimization passes (egraph-based rewriting,
+GVN, LICM, etc.), but not lowering or register allocation, and runs the
+resulting function through filecheck. This requires a target ISA, since some
+of the rewrites are ISA-specific.
 
 ### `test compile`
 
-Test the whole code generation pipeline.
-
-Each function is passed through the full `Context::compile()` function
-which is normally used to compile code. This type of test often depends
-on assertions or verifier errors, but it is also possible to use
-filecheck directives which will be matched against the final form of the
-Cranelift IR right before binary machine code emission.
+Tests the whole code generation pipeline. Each function is passed through the full `Context::compile()` function which is normally used to compile code. This type of test often depends on assertions or verifier errors, but it is also possible to use filecheck directives which will be matched against the final form of the Cranelift IR right before binary machine code emission.
 
 ### `test run`