Skip to content

Rewatch: in-process bsc prototype #8659

Description

@cknitt

Based on the OCaml rewatch reimplementation from #8653, we can prototype integrating the compiler directly into the rewatch executable to avoid having to spawn a bsc binary for each compilation step.

@cristianoc fed the prompt below into Fable, and I fed the same one into Astra.

I'll post the responses we got as separate comments. These can serve as a basis for further discussion.

We are porting Rewatch to OCaml as a drop-in for the Rust version. Next I want to see whether in-process bsc is worth it: stop spawning a bsc process per module, and ideally stop writing cmj/cmt (and maybe cmi) to disk at all.

Do not implement yet. Think this through with me.

Context:

  • A lot of build time is process creation and artifact I/O, not typechecking.
  • Integration might be >2x. I want a dirty prototype that measures that, even if it is incorrect and single-threaded.
  • The compiler has lots of global/shared mutable state (Clflags, Config.load_path, Env cache, Ident, Js_config, typechecker mutation, etc.). There are essentially no tests that reuse one compiler process across modules.
  • Getting rid of global state is probably much easier than the Rewatch port, but only if we know what to reset/isolate.
  • No multicore required for the prototype. Sequential in-process with no cm* writes may already be faster.
  • A real compiler service / parallel-from-one-process comes later.

Please:

  1. Survey how bsc is invoked today (parse vs compile), what it reads/writes (ast, cmi, cmj, cmt, js), and which globals would leak across compilations.
  2. Propose the smallest prototype that can give a real speedup signal on stdlib + tests (or a real project). What can stay on disk? What must be in memory? What must be reset between modules?
  3. Argue whether that prototype is the right next step, or whether something else is better (reset-and-reuse API, keep writing cmi only, compiler service still spawning-free but still on disk, etc.).
  4. Explicitly list what is likely to go well, what is likely to go wrong, and how we would get fooled by a misleading benchmark.
  5. Recommend a concrete goal we could later implement with /goal.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions