JS-to-TS Converter Benchmarks
Most converter comparisons are vibes. This one is methodology-first. We run identical inputs through different converters and report what we measured — including where our own engine has weaknesses.
Honesty note
We built one of the tools being benchmarked. Numbers below come from running each tool against the same fixtures on the same hardware, with output reviewed for correctness. We've published the fixtures so you can reproduce. We have not paid for premium LLM tiers; the cloud-AI numbers reflect the free / default tier.
1. Determinism
Same input. Run 10 times. Count how many distinct outputs you get.
| Converter | Distinct outputs / 10 runs | Notes |
|---|---|---|
| JavaScriptConverter (AST) | 1 | Byte-for-byte identical by design. |
| Cloud LLM converter A | 6–9 | Different whitespace, type names, and occasionally different inferred types. |
| Cloud LLM converter B | 4–7 | More stable than A but still produces semantically different output between runs. |
Fixture: 50 small functions covering CommonJS imports, classes, async/await, and JSDoc comments.
2. Throughput
How long to convert a representative 100-file batch (~30k LOC) end-to-end, single thread.
| Converter | Total time | Per-file (mean) |
|---|---|---|
| JavaScriptConverter (desktop) | ~18s | ~180ms |
| JavaScriptConverter (online) | ~25s | ~250ms (incl. network) |
| Cloud LLM converter A | ~6 min | ~3.6s (rate-limited) |
| Hand conversion (1 dev) | ~16h | ~10 min (varies wildly) |
Fixture: an Express + React mixed monorepo slice; M2 MacBook, single threaded.
3. Type accuracy
For each file, count: (a) parameters/returns correctly typed from JSDoc, (b) class fields detected from constructor assignments, (c) types invented that were not present in source.
| Converter | JSDoc → type accuracy | Class fields | Hallucinated types |
|---|---|---|---|
| JavaScriptConverter | 100% | All detected | 0 (by design) |
| Cloud LLM converter A | ~85% | Most detected | 2–3 per 100 files |
| Cloud LLM converter B | ~78% | Most detected | 4–6 per 100 files |
"Hallucinated types" means: the converter emitted a type that wasn't derivable from JSDoc, defaults, or class assignments — it guessed.
4. Where we lose
Deterministic transforms don't beat LLMs at everything. We honestly underperform on:
- Naming. LLMs often suggest better type names than we infer from local context. We use the property path; they use intuition.
- Refactoring suggestions. We translate; they sometimes also restructure. Whether that's a feature depends on your code review style.
- Inferring types from heuristics. If a variable is named
userIdand assigned a number, we type itnumber; an LLM might type itUserIdwith a branded type.
Reproducibility
The fixtures and a runner script live alongside the desktop installer (look for
benchmark-fixtures/ inside the ZIP). Send us your own results
if they differ; we'll update this page.
Try it on your own code
Benchmarks aren't your codebase. Run the converter on something you actually ship.
