plum

#treesitter#compiler#wasm

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

bed1cc5 — Peter John 2026-11-05T19:07:24+05:30 
plans: cleanup phase 4 (node ids, spans, typed codegen); backlog: FBIP behind a flag
Files changed (2) hide show
  1. docs/backlog.md +1 -1
  2. docs/plans/2026-11-05-compiler-cleanup-phase4.md +105 -0
docs/backlog.md CHANGED
@@ -14,7 +14,7 @@ Open work, in the order the user queued it, plus smaller items found during buil
14
14
  - **Unbox iterators / niche Option layout** (user 2026-11-05: later, here, not before generics). Run D made `for` loops slower: for-over-List 1M 28 → 49 ms (+75%), for-over-Range 6 → 8 ms (+33%), Map.get +17% (docs/plans/2026-11-05-language-simplification-d.benchmark.md). Cause: `next() -> Option[Pair[T, Self]]` allocates a `Some` cell per step on top of the Pair (the old `Pair?` was a nullable ref). Options: `Option[R]` for a GC ref R as a nullable ref (niche), and/or fusion/drive unboxing the `Some(step) := it.next()` loop. Bisect 961bd21 vs d7658cc back-to-back first.
15
15
  - **Run C game A/B** still owed (stress_scene/stress_world vs pre-run-C). D2 found closure-spec hints were empty since run C (fixed), which may explain part of it.
16
16
  - **Build-speed** done separately right after run D (2026-11-05).
17
- 4. **FBIP: no heap mutation** — persistent std collections updated in place when unique, Koka-style `fip` functions as a checked in-place guarantee, a reference count used only as a uniqueness check if static proof falls short. Worklist = the `mut` fields left today (see below). (Decision record §9.)
17
+ 4. **FBIP: no heap mutation** — persistent std collections updated in place when unique, Koka-style `fip` functions as a checked in-place guarantee, a reference count used only as a uniqueness check if static proof falls short. Worklist = the `mut` fields left today (see below). (Decision record §9.) User 2026-11-05: FBIP sits behind a flag like fusion (a `PassOptions` switch / `--no-fbip`), so it can be turned off and A/B'd.
18
18
  5. **One product type, `struct`** — after FBIP, merge `record` into an immutable `struct` with compiler-chosen boxing (optional hint, shown by `--explain`). (Decision record §9b.)
19
19
  - **Decide: do we still need traits?** (user 2026-11-04, decide with this item and item 2, since traits are lowered through the same hidden-union machinery). Today traits are used only by `std/game/imperative/Scene` (`trait Actor`, 14 fields), `std/game/pure/PureScene` (field-less `trait Actor` for `Custom` nodes) and three examples. The functional scene, the kids' games plan and `../kam_plum` use none. Their unique job is letting std hold user-defined kinds (a scene tree of the game's own `Ship`/`Orb`) with static dispatch.
20
20
  - **User's proposal:** make `Actor` a plain struct/record and let embedding give inheritance-like behaviour: a record that embeds another *first* is a subtype of it (`Walker` is an `Actor`, so it can sit in a `Slice[Actor]`), with `override fun` dispatched on the concrete kind. Today record embedding only copies (`a World is not a Stage`), so this means embedding becomes subtyping. That is the box-free trait layout (wasm-gc base struct + subtypes, `isSubtypedTrait`) applied to records, which would retire trait fields, `__get_`/`__set_` accessors and the boxed-trait path. Open: whether behaviour-only traits stay (Rust-style, no fields) or abstract methods on the base record cover them; what `==` means on a subtyped value (identity today); whether only first-position embedding subtypes (the rule today).
docs/plans/2026-11-05-compiler-cleanup-phase4.md ADDED
@@ -0,0 +1,105 @@
1
+ # Compiler cleanup, phase 4: node ids, spans and typed codegen
2
+
3
+ **Review plan:** docs/plans/2026-11-04-compiler-cleanup.review.md (phase 4 rows "Node ids", "Typed codegen")
4
+ **Goal:** every `Expr` and `Stmt` carries a stable `NodeId` and a source `Span`; the checker's type tables (`expr_types`, `inst_expr_types`, `call_inst`, `op_calls`) are keyed by `NodeId`, not memory address; diagnostics point at the exact sub-expression; codegen never re-infers a type (no `Int` fallback). No language change; example wasm byte-identical. Approved by the user 2026-11-05 ("do it, it simplifies the compiler").
5
+ **Architecture:** compiler/core (ast, parser, every pass), compiler/checker, compiler/driver, compiler/wasm-codegen, compiler/lsp.
6
+ **Tech stack:** Rust 2021.
7
+ **Detail:** contracts
8
+ **Target-min:** 60
9
+
10
+ ## Global constraints
11
+ - Work on `master` in the main checkout, default shared `target/`. Don't commit. Never run `plum format` on real files. No `sed -i -E`.
12
+ - No `cargo test` (hook): `cargo build --tests`; per-crate test binaries in `target/debug/deps` (`core-*`, `checker-*`, `cli-*`, `wasm_codegen-*`, `driver_tests-*`; newest by mtime, `/bin/ls -t`). Sweep `cargo build -p plum-cli && target/debug/plum test std examples` (662).
13
+ - Byte-identical wasm: before T1, hash every examples/**/*.plum compile (`scripts/wasm-hashes.sh` or the scratch script); each task keeps them identical.
14
+ - Design (coordinator ruling): `pub struct Expr { pub id: NodeId, pub span: Span, pub kind: ExprKind }` and the same for `Stmt`/`StmtKind`. `impl Expr` gets constructor fns named like the old variants (`#[allow(non_snake_case)] pub fn Int(v: i64) -> Expr`, ..., and `pub const Self_: Expr` for payload-free ones) so construction sites `Expr::Int(0)` compile unchanged; only match sites change to `match &e.kind { ExprKind::Int(n) => .. }`. `NodeId(0)` = unassigned; `ast::numberIds(&mut Source)` gives every unassigned node a fresh id; clones keep ids; a pass that DUPLICATES code (simplify inlining, drive specialisation, closure_spec copies, trmc, ctors) calls `ast::refreshIds(&mut subtree)` on its copies.
15
+ - Each task ends with workspace build zero warnings, every binary, sweep 662, wasm hashes identical.
16
+
17
+ ## Task graph
18
+ ```mermaid
19
+ flowchart LR
20
+ T1[AST: Expr/Stmt structs + ids + spans]
21
+ T2[checker + driver + passes on NodeId]
22
+ T3[codegen + uniqueness on NodeId]
23
+ T4[spans in diagnostics + LSP]
24
+ T5[typed codegen: no re-inference]
25
+ T6[verify]
26
+ %% taskgraph
27
+ T1 --> T2
28
+ T2 --> T3
29
+ T3 --> T4
30
+ T4 --> T5
31
+ T5 --> T6
32
+ ```
33
+ Ruling: sequential [chain] - the AST change breaks every crate at once; table keys must switch before codegen and spans can use them - cost: one task at a time.
34
+
35
+ ## Package A: ids
36
+ **Probe entry:** run=plum test std examples
37
+
38
+ ### T1 AST: Expr/Stmt structs with id and span · Complexity 6 · Est. context 70k
39
+ **Owns:** compiler/core/src/*.rs, compiler/checker/src/*.rs, compiler/driver/src/*.rs, compiler/wasm-codegen/src/*.rs, compiler/lsp/src/*.rs, compiler/cli/src/*.rs, compiler/test-support/src/*.rs, compiler/*/tests/**/*.rs
40
+ **Components:** 2
41
+ **LOC owned:** 300
42
+ **Tests:** fast
43
+ **Model:** sonnet
44
+ **Produces:** `ast::Expr { id: NodeId, span: Span, kind: ExprKind }`, `ast::Stmt { id, span, kind: StmtKind }`, `ast::numberIds(&mut Source)`, `ast::refreshIds(&mut Expr)`; parser fills spans (start/end line+column from the tree-sitter node) and calls numberIds
45
+ e.g. parsing `fun f() -> Int = 1 + 2` -> the Binary node has span 1:18-1:23 and a non-zero id distinct from its operands; `src.clone()` keeps every id
46
+ **Notes:** behaviour unchanged: the tables stay address-keyed in this task. Mechanical; let compiler errors drive the match-site edits. Construction sites keep compiling via the variant-named constructors.
47
+ **Test code:** core test `ids_unique_and_kept_by_clone`; span e.g. case; all binaries + sweep + wasm hashes.
48
+
49
+ ### T2 checker, driver and passes on NodeId · Complexity 5 · Est. context 60k
50
+ **Owns:** compiler/checker/src/*.rs, compiler/driver/src/*.rs, compiler/core/src/*.rs, compiler/checker/tests/**/*.rs, compiler/driver/tests/*.rs, compiler/core/tests/core/*.rs
51
+ **Components:** 2
52
+ **LOC owned:** 200
53
+ **Tests:** fast
54
+ **Consumes:** `ast::Expr { id: NodeId, span: Span, kind: ExprKind }`, `ast::Stmt { id, span, kind: StmtKind }`, `ast::numberIds(&mut Source)`, `ast::refreshIds(&mut Expr)`; parser fills spans (start/end line+column from the tree-sitter node) and calls numberIds (T1)
55
+ **Produces:** `TypedInfo.expr_types: HashMap<NodeId, PlumType>` (and inst_expr_types/call_inst/op_calls keyed by NodeId); checker numbers ids before checking; duplicating passes refresh ids
56
+ e.g. closure-spec hints read directly off `pre` by NodeId (the checked-program twin mapping from D2 is deleted); a `None` param default's type found by NodeId (the D2 `defaultSingleton` workaround can go)
57
+ **Test code:** regression tests from D2 (closure_hint_test, none_as_param_default) still green without their workarounds; all binaries + sweep + wasm hashes.
58
+
59
+ ### T3 codegen and uniqueness on NodeId · Complexity 4 · Est. context 50k
60
+ **Owns:** compiler/wasm-codegen/src/*.rs, compiler/core/src/uniqueness.rs, compiler/wasm-codegen/tests/wasm_codegen/**/*.rs
61
+ **Components:** 1
62
+ **LOC owned:** 150
63
+ **Tests:** fast
64
+ **Consumes:** `TypedInfo.expr_types: HashMap<NodeId, PlumType>` (and inst_expr_types/call_inst/op_calls keyed by NodeId); checker numbers ids before checking; duplicating passes refresh ids (T2)
65
+ **Produces:** no `as *const` address keys left in compiler/*/src
66
+ e.g. `grep -rn 'as \*const' compiler/*/src` -> no hits
67
+ **Test code:** the grep; all binaries + sweep + wasm hashes.
68
+
69
+ ## Package B: spans and typed codegen
70
+ **Probe entry:** run=plum test std examples
71
+
72
+ ### T4 spans in diagnostics and LSP · Complexity 4 · Est. context 50k
73
+ **Owns:** compiler/checker/src/*.rs, compiler/lsp/src/*.rs, compiler/checker/tests/**/*.rs, compiler/cli/tests/cli/*.rs
74
+ **Components:** 2
75
+ **LOC owned:** 150
76
+ **Tests:** fast
77
+ **Consumes:** no `as *const` address keys left in compiler/*/src (T3)
78
+ **Produces:** `inferExpr` errors are `Diagnostic`s at the failing node's span; the 3b registered-message classification is deleted; LSP ranges use span start..end
79
+ e.g. `fun f() -> Int = 1 + "a"` -> diagnostic at the `"a"` (1:22), not the fn; LSP underlines exactly `"a"`
80
+ **Test code:** checker + lsp tests for the e.g.; message wording unchanged; all binaries + sweep.
81
+
82
+ ### T5 typed codegen: no re-inference · Complexity 5 · Est. context 55k
83
+ **Owns:** compiler/wasm-codegen/src/*.rs, compiler/core/src/*.rs, compiler/checker/src/*.rs, compiler/driver/src/*.rs, compiler/wasm-codegen/tests/wasm_codegen/**/*.rs
84
+ **Components:** 2
85
+ **LOC owned:** 200
86
+ **Tests:** fast
87
+ **Consumes:** `inferExpr` errors are `Diagnostic`s at the failing node's span; the 3b registered-message classification is deleted; LSP ranges use span start..end (T4)
88
+ **Produces:** codegen reads every type from TypedInfo by NodeId; codegen's own synthetic nodes are created before checking (a lowering step in core/driver); `inferLocalType`'s re-inference and the `Int` fallback are gone (a missing type is an internal Codegen diagnostic)
89
+ e.g. `grep -n 'inferLocalType' compiler/wasm-codegen/src` -> no hits (or only a table lookup)
90
+ **Notes:** any real type gap this exposes gets fixed or listed in the report, not papered over (review plan risk).
91
+ **Test code:** all binaries + sweep + wasm hashes identical.
92
+
93
+ ### T6 verify · Complexity 2 · Est. context 30k
94
+ **Owns:** docs/plans/2026-11-05-compiler-cleanup-phase4.results.md
95
+ **Components:** 1
96
+ **LOC owned:** 30
97
+ **Tests:** slow
98
+ **Consumes:** codegen reads every type from TypedInfo by NodeId; codegen's own synthetic nodes are created before checking (a lowering step in core/driver); `inferLocalType`'s re-inference and the `Int` fallback are gone (a missing type is an internal Codegen diagnostic) (T5)
99
+ **Produces:** phase 4 results
100
+ **Notes:** workspace build zero warnings; every binary; sweep; fixtures; browser (stress_scene + one game per style via compiler/game/web-test); wasm hashes; LOC delta `git diff --stat <start>`.
101
+ **Test code:** results doc; report red, don't fix.
102
+
103
+ ## Review Focus
104
+ - A: state — ids kept by clone, refreshed by duplication; no two nodes of one program share an id after numbering.
105
+ - B: encoding — span columns 1-based in Diagnostic, 0-based in LSP.