close
Skip to content

Js_of_ocaml: share standalone runtimes#13621

Merged
rgrinberg merged 3 commits into
ocaml:mainfrom
vouillon:jsoo-shared-runtime
Mar 17, 2026
Merged

Js_of_ocaml: share standalone runtimes#13621
rgrinberg merged 3 commits into
ocaml:mainfrom
vouillon:jsoo-shared-runtime

Conversation

@vouillon
Copy link
Copy Markdown
Member

At the moment, in separate compilation mode, we are building one custom runtime per executable, even though the same runtime could be used by the different executable.

The first commit implement the sharing at the level of a stanza, while the second commit extends it to workspace-wide sharing.

I'm not completely sure the added complexity is worth it. A clean run of the Js_of_ocaml test suite go from 3 minutes down to 2m40 with this change, but it's kind of extreme.

@hhugo What do you think?

@hhugo
Copy link
Copy Markdown
Collaborator

hhugo commented Feb 12, 2026

I wanted to explore another direction where we would emit a minimal runtime based on requirement from individual units. Either having jsoo link do the job of computing the runtime or relying on an intermediate runtime file like now. But we need to make sure the overhead is acceptable (We need incremental rebuilds to be as fast as possible)

That said, I like the current PR.
In the jsoo repo, the size of all runtime files under _build/ goes from ~280M to ~9M with this PR.
We can always revisit later if needed.

@vouillon vouillon marked this pull request as ready for review February 12, 2026 13:32
Comment thread test/blackbox-tests/test-cases/jsoo/jsoo-config.t/run.t Outdated
@vouillon vouillon force-pushed the jsoo-shared-runtime branch 3 times, most recently from 42ab11e to 8025dad Compare March 3, 2026 13:27
@vouillon vouillon force-pushed the jsoo-shared-runtime branch from 8025dad to 8f9ac22 Compare March 12, 2026 13:12
@hhugo
Copy link
Copy Markdown
Collaborator

hhugo commented Mar 13, 2026

@rgrinberg, I think this is ready

@hhugo hhugo added the jsoo label Mar 13, 2026
@hhugo
Copy link
Copy Markdown
Collaborator

hhugo commented Mar 14, 2026

@vouillon, this needs to be rebased

When dune builds `(executables (names a b c))` in separate compilation
mode, it previously ran `js_of_ocaml build-runtime` once per executable.
Since all executables in the same stanza share the same compilation
context, the runtimes are identical. This change builds one shared
runtime per stanza per mode, eliminating redundant work.

Signed-off-by: Jérôme Vouillon <jerome.vouillon@gmail.com>
Phase 1 shared one runtime per (executables ...) stanza. This commit
eliminates remaining duplication by sharing runtimes across all stanzas
in the workspace that link the same set of runtime-contributing libraries.

Following the PPX driver sharing pattern (Key module + on-demand rule
generation), a Runtime_key module encodes (mode, lib_names, project_root)
into a digest. Stanzas with no user javascript_files/wasm_files get a
Shared digest; others keep per-stanza behavior.

Shared runtimes are built on-demand at .js/<config>/.runtime/<digest>/
when first requested, with the config resolved dynamically at build time
to ensure correctness across directories with different env configs.

Signed-off-by: Jérôme Vouillon <jerome.vouillon@gmail.com>
Use dune trace cat | jq to extract deterministic sorted target lists
instead of --display short which produces unstable build step ordering.
For install tests, use find | sort instead of install log output.

Signed-off-by: Jérôme Vouillon <jerome.vouillon@gmail.com>
@vouillon vouillon force-pushed the jsoo-shared-runtime branch from 8f9ac22 to 88620d2 Compare March 17, 2026 10:41
@rgrinberg rgrinberg merged commit b0bfad0 into ocaml:main Mar 17, 2026
29 checks passed
avsm pushed a commit to ocaml/opam-repository that referenced this pull request May 5, 2026
CHANGES:

### Fixed

- Auto-inject `"menhir" {>= "20180523"}` into generated opam files when the
  menhir extension is used. Dune's menhir rules rely on `--infer-write-query`
  and `--infer-read-reply`, which require at least this version; without the
  lower bound, builds fail with older menhir installations.
  (ocaml/dune#10707, @robinbb)

- Fix `--display=quiet` not suppressing "Entering directory" and "Leaving
  directory" messages when using `--root`. These messages are now deferred
  until there is actual output, so silent builds produce no noise.
  (ocaml/dune#12974, fixes ocaml/dune#12854, @Alizter)

- Fix an internal error when a module is shared between a `rocq.theory` and
  `rocq.extraction` stanza. The error now includes a hint pointing to the
  conflicting stanza. (ocaml/dune#13733, @Durbatuluk1701)

- Fix cookies defined on `ppx_rewriter` being lost when that rewriter was used
  as a dependency of another `ppx_rewriter`. (ocaml/dune#13737, fixes ocaml/dune#3426, @Alizter)

- Improve opam file generation for packages that set `(dir ..)`. Such packages
  will now have a test target that is limited to this directory. (ocaml/dune#13778, @rgrinberg)

- Stop duplicating dune diagnostics to subscribers over RPC (ocaml/dune#13816,
  @rgrinberg)

- Fix dependency cycle when using the `package` with a library
  conditionally enabled via `enabled_if`. (ocaml/dune#13833, @toots)

- Fix underspecification of dependencies in the sandbox for modules with
  interfaces. (ocaml/dune#13842, @anmonteiro)

- Honor sandbox settings specified inside `(:include ..)` expressions in the `deps` field
  (ocaml/dune#13898, @rgrinberg)

- Fix `dune pkg lock` failing when a `pin` stanza contains a `file://` URL with
  a relative path outside the workspace (ocaml/dune#13915, fixes ocaml/dune#10254, @shunueda)

- Use all stat attributes to detect patch back source tree changes (ocaml/dune#13986, @rgrinberg)

- Use device and inode number for invalidating cached digests (ocaml/dune#13991, @rgrinberg)

- Stop trying to set SO_REUSEADDR on unix sockets when setting up dune rpc
  (ocaml/dune#14056, @rgrinberg)

- Fix autolocking to correctly detect changes to `(depends)` in watch mode
  (ocaml/dune#14066, fixes ocaml/dune#13234, @Alizter)

- Fix incremental builds for libraries using `(wrapped (transition ...))`.
  The compat shim modules were never recompiled when the inner module's
  interface changed, causing "inconsistent assumptions" errors.
  (ocaml/dune#14090, fixes ocaml/dune#14089, @Alizter)

- Fix Rocq configuration detection to use `rocq c --config` subcommand
  instead of `rocq --config` (ocaml/dune#14093, fixes ocaml/dune#13774, @Durbatuluk1701)

- Bump dune rpc's request pending request limit 10 to 100 (ocaml/dune#14094, @rginberg)

- Invalidate stale cached digests in all build commands (ocaml/dune#14126, @rgrinberg)

- Fix `root_module` generating duplicate module definitions when a
  findlib sub-package shares a directory with its parent (e.g.,
  `logs.lwt` alongside `logs`). (ocaml/dune#14135, fixes ocaml/dune#6148, @robinbb)

- Fix `dune subst` prepending a duplicate `version:` field to opam
  files that already contain one, instead of replacing it in place.
  (ocaml/dune#14136, fixes ocaml/dune#878, @robinbb)

- Fix `@ocaml-index` alias being generated for all contexts, causing build
  errors in multi-context workspaces where some libraries are disabled in
  non-default contexts. The alias is now only generated for the merlin context.
  (ocaml/dune#14137, fixes ocaml/dune#12007, @robinbb)

- Remove the warning about the deprecation of the coq stanza for
  dune lang before 3.21 since it has been introduced in this
  version (ocaml/dune#14187, @bobot)

- `$ dune init` now generates projects with correct .opam files (ocaml/dune#14192, @rgrinberg)

- Send SIGTERM before SIGKILL when cancelling child processes in watch
  mode, giving cleanup handlers a chance to run before escalating.
  (ocaml/dune#14224, ocaml/dune#14170, fixes ocaml/dune#2445, @robinbb)

- Correctly specify dependencies for `generate_runner` in inline tests (ocaml/dune#14276, @rgrinberg)

- Fix duplicate dune version bounds in generated opam files. When users
  declare `(dune (>= X.Y))` matching or exceeding the `(lang dune X.Y)`
  version, the generated opam `depends` no longer contains redundant
  constraints like `{>= "2.7" & >= "2.7"}`. (ocaml/dune#3916, ocaml/dune#11106, @robinbb)

- Fix inline tests not being executed for byte-only libraries. When a library
  has `(modes byte)`, the inline test runner is now linked in byte mode so the
  library's test code is included. (ocaml/dune#9757, @robinbb)

### Added

- Add support for `c_library_flags` in foreign_stubs (ocaml/dune#13484, @madroach)

- Move the management of Jsoo config details out of dune (ocaml/dune#13613, @vouillon)

- Js_of_ocaml: share standalone runtimes (ocaml/dune#13621, @vouillon)

- Allow multiple `(dirs ..)` stanzas in the same Dune file. Starting with
  `(lang dune 3.23)`, Dune takes the union of the specified directories
  (ocaml/dune#13734, fixes ocaml/dune#6249, @anmonteiro)

- Sandboxed rules are now allowed to produce diff promotions when `(corrections
  produce)` is set on the rules. Whenever such an action produces `foo.corrected`, it will
  be automatically registered as a promotion for `foo` (ocaml/dune#13813, @rgrinberg)

- `$ dune clean` now accepts arguments to only remove certain targets from the
  _build directory (ocaml/dune#13875, @rgrinberg)

- `$ dune promote` now promotes all paths that are a prefix of the path provided.
  For example, `$ dune promote foo` will promote `foo`, `foo/bar`, `foo/bar/baz`
  (but not `foobar`). (ocaml/dune#13876, @rgrinberg)

- Allow for the `diff` action to diff entire directories (ocaml/dune#13880, fixes ocaml/dune#3567, @rgrinberg)

- A hint is now emitted when trying to build a target inside a directory that
  was excluded by a `(dirs ...)` stanza (ocaml/dune#13919, @mefyl).

- Add --debug-backtraces to `$ dune {promote,trace,cache}` (ocaml/dune#13933, @rgrinberg)

- Actions are now able to observe their project directory via
  `DUNE_PROJECT_ROOT` (ocaml/dune#13934, @rgrinberg)

- Support hardlink sandboxing on Windows. (ocaml/dune#13987, addresses part of ocaml/dune#4362, @Alizter)

- `rocq.extraction`: Add `extracted_files` field in `(rocq 0.13)`, replacing
  `extracted_modules`, supporting extraction to languages other than OCaml
  (ocaml/dune#13997, @Durbatuluk1701).

- Infer source directories as dependencies when they're as reference
  directories in `diff` actions.

- Add trace events for build start/stop/restarts (ocaml/dune#14163, ocaml/dune#14166, @rgrinberg)

- Generate `compile_commands.json` for C/C++ foreign stubs when building
  `@check` with `(lang dune 3.23)`. This enables `clangd`, `ccls`, and other
  tools that consume compilation databases. (ocaml/dune#14185, fixes ocaml/dune#3531, @Alizter)

- Add trace events for accepting clients and shutting down RPC (ocaml/dune#14232, @rgrinberg)

- Back up the default trace file as `trace.csexp.old` before each build,
  preserving the previous trace for comparison. This does not apply when
  `--trace-file` is used. (ocaml/dune#14269, @Alizter)

### Changed

- Dune digests target files and directories in background threads (ocaml/dune#13341,
  @rgrinbreg)

- `Dyn.pp` now prints valid OCaml literals for more constructors:
  - `Char` is quoted (e.g., `'a'`)
  - `Int32`/`Int64`/`Nativeint` include literal suffixes (`l`/`L`/`n`)
  - `Float` handles `infinity`, `neg_infinity`, and `nan`
  (ocaml/dune#13394, grants ocaml/dune#13378, @Alizter)

- Bump the minimal OCaml version required to build Dune from 4.08 to 4.14.
  (ocaml/dune#13533, @Alizter)

- Starting from 3.23, user rules are sandboxed by default (ocaml/dune#13805, @rgrinberg)

- Dune will no longer stat paths in the _build directory. This was done as protection
  from user rules accidentally touching things in the _build directory. This is forbidden,
  and dune offers rule sandboxing to prevent this (ocaml/dune#13875, @rgrinberg)

- Improve mtime precision by no longer approximating them with floats (ocaml/dune#13962,
  @rgrinberg).

- Remove `--makefile` (or `-m`) in `$ dune rules`. The sexp output is the only
  supported way of reflecting dune rules (ocaml/dune#14069, @rgrinberg)

- Do not automatically promote files. Now, opam files must be manually
  promoted with `$ dune promote`. Their generation is triggered by `@install`,
  `@runtest`, and `@opam`. In release mode, .opam files aren't generated at all
  and whatever is in the source is used (ocaml/dune#14108, @rgrinberg)

- Inline test runner generation is now sandboxed (ocaml/dune#14257, @rgrinberg)
@hhugo hhugo mentioned this pull request May 7, 2026
20 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants