plum

#treesitter#compiler#wasm

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

ada2b0a — Peter John 2026-11-04T17:08:16+05:30 
docs: phase 2a plan split (rename, generics, host tasks)
Files changed (1) hide show
  1. docs/plans/2026-11-04-compiler-cleanup-phase2a.md +83 -37
docs/plans/2026-11-04-compiler-cleanup-phase2a.md CHANGED
@@ -39,8 +39,11 @@ flowchart LR
39
39
  T5[forms union_lower]
40
40
  T6[uniqueness reuse tail]
41
41
  T7[trmc queries]
42
+ T14[rename module]
42
- T8[generics by scope]
43
+ T8[generics module]
44
+ T15[passes ask generics]
43
45
  T10[plum-host crate]
46
+ T16[hosts use plum-host]
44
47
  T13[test helpers LSP]
45
48
  T9[verify 2a]
46
49
  %% taskgraph
@@ -49,21 +52,26 @@ flowchart LR
49
52
  T1 --> T5
50
53
  T1 --> T6
51
54
  T1 --> T7
52
- T2 --> T3
53
- T2 --> T4
55
+ T2 --> T14
54
- T2 --> T5
55
56
  T2 --> T6
56
57
  T2 --> T7
58
+ T14 --> T3
59
+ T14 --> T4
60
+ T14 --> T5
57
- T3 --> T8
61
+ T1 --> T8
58
- T5 --> T8
62
+ T14 --> T8
63
+ T8 --> T15
64
+ T3 --> T15
65
+ T5 --> T15
59
- T7 --> T8
66
+ T7 --> T15
67
+ T10 --> T16
60
68
  T4 --> T9
61
69
  T6 --> T9
62
- T8 --> T9
63
- T10 --> T9
70
+ T15 --> T9
71
+ T16 --> T9
64
72
  T13 --> T9
65
73
  ```
66
- Ruling: sequential - T8 edits drive/forms/trmc after their conversions; same files - cost: longer chain.
74
+ Ruling: sequential - T15 edits drive/forms/trmc/union_lower after their conversions, T8 edits checker lib.rs after T1 and core lib.rs after T14; same files - cost: longer chain.
67
75
 
68
76
  ## Package A: compiler core
69
77
  **Probe entry:** run=bash scripts/wasm-hashes.sh
@@ -82,7 +90,7 @@ Ruling: sequential - T8 edits drive/forms/trmc after their conversions; same fil
82
90
  ### T2 visit module · Complexity 5 · Est. context 40k
83
91
  **Owns:** plum-core/src/visit.rs, plum-core/src/lib.rs, plum-core/tests/visit_test.rs
84
92
  **Components:** 1
85
- **LOC owned:** 260
93
+ **LOC owned:** 150
86
94
  **Tests:** fast
87
95
  **Model:** sonnet
88
96
  **Produces:**
@@ -93,18 +101,32 @@ Ruling: sequential - T8 edits drive/forms/trmc after their conversions; same fil
93
101
  - visit::visitBlock(b: &mut Block, f: &mut dyn FnMut(&mut Expr))
94
102
  - visit::visitExpr(e: &mut Expr, f: &mut dyn FnMut(&mut Expr))
95
103
  - visit::anyExpr(e: &Expr, pred: &mut dyn FnMut(&Expr) -> bool) -> bool
96
- - visit::Name enum { Decl, Pat, Plain, Ref }
97
- - visit::renameStmts(stmts: &mut [Stmt], rn: &mut dyn FnMut(&mut String, Name))
98
- - visit::substType(t: &Type, map: &HashMap<String, Type>) -> Type
99
104
  - visit::patternBinders(p: &CasePattern, out: &mut Vec<String>)
100
105
  - visit::unparen(e: &Expr) -> &Expr
101
106
  - visit::countVar(body: &FnBody, name: &str) -> usize
102
107
  e.g. anyExpr(&parse_expr("f(Try(1))"), &mut |e| matches!(e, Expr::Call{..})) -> true
103
108
  e.g. anyExpr(&parse_expr("\"Try(\""), &mut |e| matches!(e, Expr::Call{..})) -> false
104
- e.g. substType(&Type{name:"List", generics:[T]}, {T: Int}) -> List[Int]
105
- **Review Focus:** `rename_shadowing`: a closure param shadowing `x` is renamed as Decl and stops outer renames in its body (copy drive.rs behaviour exactly); `pattern_names`: constructor names in patterns are not binders.
106
- **Test code:** `plum-core/tests/visit_test.rs`: the e.g. lines, one rename test per Name kind, patternBinders on `Cons(head: h, tail: t)` → ["h","t"].
107
- **Notes:** copy `drive.rs` 1884-2260 (walkers + rename) as the reference behaviour; add only `anyExpr`. Generic `substType` must not decide what is generic — it substitutes exactly the map's keys.
109
+ e.g. patternBinders(`Cons(head: h, tail: t)`) -> ["h", "t"]
110
+ **Review Focus:** `pattern_names`: constructor names in patterns are not binders.
111
+ **Test code:** `plum-core/tests/visit_test.rs`: the e.g. lines plus childExprs/stmtBlocks on an if/match/for sample.
112
+ **Notes:** copy `drive.rs` ≈1884-2040 and 2160-2260 (walkers) as the reference behaviour; add only `anyExpr`.
113
+
114
+ ### T14 shared rename and type substitution · Complexity 4 · Est. context 35k
115
+ **Owns:** plum-core/src/rename.rs, plum-core/src/lib.rs, plum-core/tests/rename_test.rs
116
+ **Components:** 1
117
+ **LOC owned:** 110
118
+ **Tests:** fast
119
+ **Model:** sonnet
120
+ **Consumes:** visit::childExprs(e: &mut Expr) -> Vec<&mut Expr> (T2)
121
+ **Produces:**
122
+ - rename::Name enum { Decl, Pat, Plain, Ref }
123
+ - rename::renameStmts(stmts: &mut [Stmt], rn: &mut dyn FnMut(&mut String, Name))
124
+ - rename::substType(t: &Type, map: &HashMap<String, Type>) -> Type
125
+ e.g. renameStmts(`x := 1; y := x`, x→x2) -> `x2 := 1; y := x2` (Decl then Ref)
126
+ e.g. substType(&List[T], {T: Int}) -> List[Int]
127
+ **Review Focus:** `rename_shadowing`: a closure param shadowing `x` is reported as Decl and outer renames stop in its body (copy drive.rs ≈2041-2160 exactly); `field_assign_target`: `self.x = v` target is not renamed as a variable.
128
+ **Test code:** `plum-core/tests/rename_test.rs`: e.g. lines, one test per Name kind, the two focus tests.
129
+ **Notes:** `substType` substitutes exactly the map's keys; it never decides what is generic.
108
130
 
109
131
  ### T3 drive and simplify use visit · Complexity 4 · Est. context 45k
110
132
  **Owns:** plum-core/src/drive.rs, plum-core/src/simplify.rs
@@ -112,7 +134,7 @@ e.g. substType(&Type{name:"List", generics:[T]}, {T: Int}) -> List[Int]
112
134
  **LOC owned:** 60
113
135
  **Tests:** slow
114
136
  **Model:** sonnet
115
- **Consumes:** visit::childExprs(e: &mut Expr) -> Vec<&mut Expr> (T2) · visit::renameStmts(stmts: &mut [Stmt], rn: &mut dyn FnMut(&mut String, Name)) (T2) · visit::childRefs(e: &Expr) -> Vec<&Expr> (T2) · visit::unparen(e: &Expr) -> &Expr (T2)
137
+ **Consumes:** visit::childExprs(e: &mut Expr) -> Vec<&mut Expr> (T2) · rename::renameStmts(stmts: &mut [Stmt], rn: &mut dyn FnMut(&mut String, Name)) (T14) · visit::childRefs(e: &Expr) -> Vec<&Expr> (T2) · visit::unparen(e: &Expr) -> &Expr (T2)
116
138
  **Produces:** none
117
139
  **Test code:** plum-core drive_test + simplify_test binaries pass; wasm hashes identical to T1.
118
140
  **Steps:** delete the local walker/rename copies and `isTag`/`isTagName` duplicates, call visit; keep `isGen`/`substType` for T8.
@@ -123,7 +145,7 @@ e.g. substType(&Type{name:"List", generics:[T]}, {T: Int}) -> List[Int]
123
145
  **LOC owned:** 50
124
146
  **Tests:** slow
125
147
  **Model:** sonnet
126
- **Consumes:** visit::childExprs(e: &mut Expr) -> Vec<&mut Expr> (T2) · visit::renameStmts(stmts: &mut [Stmt], rn: &mut dyn FnMut(&mut String, Name)) (T2) · visit::countVar(body: &FnBody, name: &str) -> usize (T2)
148
+ **Consumes:** visit::childExprs(e: &mut Expr) -> Vec<&mut Expr> (T2) · rename::renameStmts(stmts: &mut [Stmt], rn: &mut dyn FnMut(&mut String, Name)) (T14) · visit::countVar(body: &FnBody, name: &str) -> usize (T2)
127
149
  **Produces:** none
128
150
  **Review Focus:** `pattern_decl`: closure_spec's own Name has no `Pat` and renames pattern names as Decl — map its call sites so behaviour is unchanged (pin with a test in `plum-core/tests/closure_spec_test.rs` first); `debug_key`: the `format!("{:?}")` key (≈322) becomes a structural `Expr` equality (ast derives PartialEq).
129
151
  **Test code:** closure_spec_test binary passes plus the pinning test; hashes identical.
@@ -134,10 +156,10 @@ e.g. substType(&Type{name:"List", generics:[T]}, {T: Int}) -> List[Int]
134
156
  **LOC owned:** 60
135
157
  **Tests:** slow
136
158
  **Model:** sonnet
137
- **Consumes:** visit::childExprs(e: &mut Expr) -> Vec<&mut Expr> (T2) · visit::stmtExprs(s: &mut Stmt) -> Vec<&mut Expr> (T2) · visit::stmtBlocks(s: &mut Stmt) -> Vec<&mut Block> (T2) · visit::substType(t: &Type, map: &HashMap<String, Type>) -> Type (T2) · visit::patternBinders(p: &CasePattern, out: &mut Vec<String>) (T2)
159
+ **Consumes:** visit::childExprs(e: &mut Expr) -> Vec<&mut Expr> (T2) · visit::stmtExprs(s: &mut Stmt) -> Vec<&mut Expr> (T2) · visit::stmtBlocks(s: &mut Stmt) -> Vec<&mut Block> (T2) · rename::substType(t: &Type, map: &HashMap<String, Type>) -> Type (T14) · visit::patternBinders(p: &CasePattern, out: &mut Vec<String>) (T2)
138
160
  **Produces:** none
139
161
  **Test code:** forms_* and union_lower_test binaries pass; hashes identical.
140
- **Notes:** forms' rename uses a bool, union_lower's `rn*` renames pairs: express both through `visit::renameStmts`; `forms::mapExprsIn`/`walkStmts` stay public as thin wrappers.
162
+ **Notes:** forms' rename uses a bool, union_lower's `rn*` renames pairs: express both through `rename::renameStmts`; `forms::mapExprsIn`/`walkStmts` stay public as thin wrappers.
141
163
 
142
164
  ### T6 uniqueness, reuse, tail share helpers · Complexity 3 · Est. context 40k
143
165
  **Owns:** plum-core/src/uniqueness.rs, plum-core/src/reuse.rs, plum-core/src/tail.rs
@@ -162,32 +184,56 @@ e.g. substType(&Type{name:"List", generics:[T]}, {T: Int}) -> List[Int]
162
184
  **Test code:** those two tests in trmc_test.rs (fail before, pass after); all trmc_test pass; hashes identical.
163
185
  **Steps:** replace `bannedText(format!("{:?}"..))` (≈127, 188) and `mentions` (≈255) with `anyExpr` matches on the real variants.
164
186
 
165
- ### T8 generics by scope · Complexity 5 · Est. context 50k
166
- **Owns:** plum-core/src/generics.rs, plum-core/src/lib.rs, plum-core/src/drive.rs, plum-core/src/forms.rs, plum-core/src/trmc.rs, plum-core/src/union_lower.rs, plum-checker/src/lib.rs, plum-core/tests/generics_test.rs
167
- **Components:** 2
168
- **LOC owned:** 120
187
+ ### T8 generics module and checker · Complexity 4 · Est. context 45k
188
+ **Owns:** plum-core/src/generics.rs, plum-core/src/lib.rs, plum-checker/src/lib.rs, plum-core/tests/generics_test.rs
189
+ **Components:** 1
190
+ **LOC owned:** 70
169
191
  **Tests:** slow
170
192
  **Model:** sonnet
171
193
  **Produces:** generics::isGenericName(n: &str) -> bool · generics::fnGenerics(f: &Fn, owner: &[String]) -> Vec<String>
172
194
  e.g. fnGenerics(`fun map(self, f: fn(T) -> B) -> List[B]` in `List[T]`, &["T"]) -> ["T","B"]
173
195
  e.g. fnGenerics(`fun len(self) -> Int`, &[]) -> []
174
- **Review Focus:** `no_spelling_checks_left`: `grep -rn "is_ascii_uppercase" plum-core/src plum-checker/src` shows only generics.rs (and unrelated casing checks, each justified in the report); `enum_fallback_gone`: checker `enumGenericParams` uses only declared generics.
175
- **Test code:** generics_test.rs for the e.g. lines; all crate tests pass; hashes identical.
176
- **Steps:** drive `isGen`, forms `isGenericName`, trmc `open`, checker `isGenericName` → call `generics::isGenericName`, or better, check membership in the fn/owner generic set where the set is at hand; delete the checker's single-letter-field fallback in `enumGenericParams`; leave the grammar alone.
196
+ e.g. isGenericName("T") -> true; isGenericName("Int") -> false
197
+ **Review Focus:** `enum_fallback_gone`: checker `enumGenericParams` uses only declared generics (delete the single-letter-field fallback); checker `isGenericName` becomes a re-export/call of `generics::isGenericName`.
198
+ **Test code:** generics_test.rs for the e.g. lines; checker test binaries pass; hashes identical.
199
+
200
+ ### T15 passes ask generics · Complexity 3 · Est. context 40k
201
+ **Owns:** plum-core/src/drive.rs, plum-core/src/forms.rs, plum-core/src/trmc.rs, plum-core/src/union_lower.rs
202
+ **Components:** 1
203
+ **LOC owned:** 50
204
+ **Tests:** slow
205
+ **Model:** sonnet
206
+ **Consumes:** generics::isGenericName(n: &str) -> bool (T8) · generics::fnGenerics(f: &Fn, owner: &[String]) -> Vec<String> (T8) · rename::substType(t: &Type, map: &HashMap<String, Type>) -> Type (T14)
207
+ **Produces:** none
208
+ **Review Focus:** `no_spelling_checks_left`: `grep -rn "is_ascii_uppercase" plum-core/src plum-checker/src` shows only generics.rs plus unrelated casing checks, each justified in the report.
209
+ **Test code:** drive/forms/trmc/union_lower test binaries pass; hashes identical.
210
+ **Steps:** drive `isGen`+`substType`, forms `isGenericName`+`substType`, trmc `open`+`subst`, union_lower `substType` → `generics::*` and `rename::substType`; prefer membership in the fn/owner generic set where it is at hand.
177
211
 
178
212
  ## Package B: hosts and tests
179
213
  **Probe entry:** run=cargo build --workspace --tests
180
214
 
181
- ### T10 plum-host crate · Complexity 5 · Est. context 55k
182
- **Owns:** plum-host/Cargo.toml, plum-host/src/lib.rs, Cargo.toml, plum-cli/Cargo.toml, plum-cli/src/main.rs, plum-runtime/Cargo.toml, plum-runtime/src/main.rs, plum-game/Cargo.toml, plum-game/src/host.rs
215
+ ### T10 plum-host crate · Complexity 4 · Est. context 45k
216
+ **Owns:** plum-host/Cargo.toml, plum-host/src/lib.rs, Cargo.toml
217
+ **Components:** 1
218
+ **LOC owned:** 140
219
+ **Tests:** fast
220
+ **Model:** sonnet
221
+ **Produces:** plum_host::engine() -> wasmtime::Engine · plum_host::strBytes(store: &mut impl AsContextMut, s: &Rooted<AnyRef>) -> Result<Vec<u8>> · plum_host::newStr(store: &mut impl AsContextMut, bytes: &[u8]) -> Result<Val> · plum_host::hostImports(store: &mut Store<()>, module: &Module, caps: Caps) -> Result<Vec<Extern>> · plum_host::Caps { fs: bool, http: bool, clock: bool, rng: bool }
222
+ e.g. strBytes(newStr("héllo")) -> b"h\xc3\xa9llo"
223
+ e.g. hostImports for a module importing ("plum","printLn") with Caps::default() -> 1 extern
224
+ **Test code:** plum-host unit tests: Str round trip, hostImports on a tiny wat module.
225
+ **Notes:** port from plum-cli/src/main.rs 284-835 (the superset); keep the 64 MiB GC heap where the CLI uses it today; workspace member.
226
+
227
+ ### T16 hosts use plum-host · Complexity 4 · Est. context 45k
228
+ **Owns:** plum-cli/Cargo.toml, plum-cli/src/main.rs, plum-runtime/Cargo.toml, plum-runtime/src/main.rs, plum-game/Cargo.toml, plum-game/src/host.rs
183
229
  **Components:** 2
184
- **LOC owned:** 200
230
+ **LOC owned:** 80
185
231
  **Tests:** slow
186
232
  **Model:** sonnet
187
- **Produces:** plum_host::engine() -> wasmtime::Engine · plum_host::strBytes(store, s) -> Result<Vec<u8>> · plum_host::newStr(store, bytes) -> Result<Val> · plum_host::hostImports(store: &mut Store<()>, module: &Module, caps: Caps) -> Result<Vec<Extern>> · plum_host::Caps { fs: bool, http: bool, clock: bool, rng: bool }
188
- **Review Focus:** `runtime_build`: `plum build` (cmdBuild ≈1120) still produces a working runtime binary with the new path dep; `game_host`: plum-game host tests pass (`cargo build --tests` in plum-game, run its binaries).
189
- **Test code:** unit tests in plum-host for strBytes/newStr round trip; cli compile_tests/crash_test/host_stub_test pass; plum-game host_test passes.
190
- **Notes:** the CLI copy (main.rs 284-835) is the superset; runtime keeps only what it enables via Caps. Keep the 64 MiB GC heap setting where it is used today.
233
+ **Consumes:** plum_host::hostImports(store: &mut Store<()>, module: &Module, caps: Caps) -> Result<Vec<Extern>> (T10) · plum_host::strBytes(store: &mut impl AsContextMut, s: &Rooted<AnyRef>) -> Result<Vec<u8>> (T10) · plum_host::newStr(store: &mut impl AsContextMut, bytes: &[u8]) -> Result<Val> (T10) · plum_host::engine() -> wasmtime::Engine (T10)
234
+ **Produces:** none
235
+ **Review Focus:** `runtime_build`: `plum build` (cmdBuild ≈1120) still builds a working runtime with the path dep; `game_host`: plum-game host_test passes (`cargo build --tests` in plum-game, run its binaries).
236
+ **Test code:** cli compile_tests/crash_test/host_stub_test pass; plum-game host_test passes; plum-runtime builds.
191
237
 
192
238
  ### T13 test helpers and LSP generic fix · Complexity 3 · Est. context 40k
193
239
  **Owns:** plum-test-support/src/lib.rs, plum-core/tests/{drive,simplify,uniqueness,record_embed,record_init,record_init_parse,trait_parse,traits,closure_types_parse,match_body_parse,override_parse,whistle,purity,forms_list_range,parser}_test.rs, plum-checker/tests/traits_tests.rs, plum-checker/tests/trait_fields_tests.rs, plum-checker/tests/features/mutability.rs, plum-lsp/src/completion.rs
@@ -217,4 +263,4 @@ e.g. fnGenerics(`fun len(self) -> Int`, &[]) -> []
217
263
  - Package B (host/tests): protocol, file — risks: host import names/signatures, Str ABI, runtime build path.
218
264
 
219
265
  ## Self-review
220
- Covers phase 2 rows: determinism (T1), tree walker (T2-T6), real tree queries (T4, T7), generics by scope (T8), host crate (T10), test helpers + LSP fix (T13). Front end, compile call and Paren are phase 2b.
266
+ Covers phase 2 rows: determinism (T1), tree walker (T2, T14, T3-T6), real tree queries (T4, T7), generics by scope (T8, T15), host crate (T10, T16), test helpers + LSP fix (T13). Front end, compile call and Paren are phase 2b.