plum

#treesitter#compiler#wasm

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

567dd39 — Peter John 2026-11-04T18:08:00+05:30 
docs: compiler cleanup phase 2b plan
Files changed (1) hide show
  1. docs/plans/2026-11-04-compiler-cleanup-phase2b.md +163 -0
docs/plans/2026-11-04-compiler-cleanup-phase2b.md ADDED
@@ -0,0 +1,163 @@
1
+ # Compiler cleanup, phase 2b: one front end, one compile call, no Paren
2
+
3
+ **Review plan:** docs/plans/2026-11-04-compiler-cleanup.review.md
4
+ **Goal:** One front-end lowering entry used by the loader and single-file parsing; `test`-block desugaring moved out of codegen; one compile entry with an options value (fusion options no longer a thread-local); the last walker helpers in `visit.rs`; `Expr::Paren` removed. No behaviour change.
5
+ **Architecture:** Rust workspace (plum-core, plum-checker, plum-wasm-codegen, plum-cli, plum-lsp, plum-host, plum-test-support); plum-runtime and plum-game are separate cargo projects.
6
+ **Tech stack:** Rust 2021, cargo, wasmtime 47, tree-sitter-plum.
7
+ **Detail:** contracts
8
+ **Target-min:** 25
9
+
10
+ ## Global constraints
11
+ - Agents cannot run `cargo test` (hook). Build with `cargo build -p <crate> --tests`, run the binaries in `target/debug/deps` (newest by mtime). `touch` edited .rs files first.
12
+ - No behaviour change: all 25 examples compile to sha256-identical wasm vs the start sha (`.superlite/progress.md`), via `bash scripts/wasm-hashes.sh <sha> OUT` — the start sha builds cleanly. Keep scratch files in your own scratchpad subdirectory.
13
+ - camelCase fns, `#![allow(non_snake_case)]`. Never run `plum format` on real files. Don't commit.
14
+
15
+ ## Main diagram
16
+ ```mermaid
17
+ flowchart LR
18
+ P["parser: parse only"] --> L["lower::frontEnd"]
19
+ Ld["loader"] --> L
20
+ L --> TL["lower tests"]
21
+ TL --> C["checker typedSourceWith(opts)"]
22
+ C --> G["codegen compile(opts)"]
23
+ V["visit.rs helpers"] --> X["Paren removal (last)"]
24
+ G --> X
25
+ ```
26
+
27
+ ## File structure
28
+ - `plum-core/src/lower.rs` — the front-end lowering sequence (traits, ctors, inline-variant checks, member resolution, forms prep).
29
+ - `plum-core/src/tests_lower.rs` — `test` blocks and `assert` into ordinary fns (was in codegen).
30
+
31
+ ## Task graph
32
+ ```mermaid
33
+ flowchart LR
34
+ T1[front-end entry]
35
+ T2[parseSource uses it]
36
+ T3[tests lowering]
37
+ T4[compile options]
38
+ T5[compile callers]
39
+ T6[visit helpers]
40
+ T7[Paren removal]
41
+ T8[verify 2b]
42
+ %% taskgraph
43
+ T1 --> T2
44
+ T1 --> T3
45
+ T3 --> T4
46
+ T4 --> T5
47
+ T2 --> T7
48
+ T5 --> T7
49
+ T6 --> T7
50
+ T7 --> T8
51
+ ```
52
+ Ruling: sequential - codegen lib.rs (T3, T4, T7) and parser.rs (T2, T7) are edited in turn; Paren touches every crate - cost: longer chain.
53
+
54
+ ## Package A: front end
55
+ **Probe entry:** run=bash scripts/wasm-hashes.sh
56
+
57
+ ### T1 front-end entry · Complexity 3 · Est. context 35k
58
+ **Owns:** plum-core/src/lower.rs, plum-core/src/lib.rs, plum-core/src/loader.rs, plum-core/tests/lower_test.rs
59
+ **Components:** 1
60
+ **LOC owned:** 70
61
+ **Tests:** fast
62
+ **Model:** sonnet
63
+ **Produces:** lower::frontEnd(source: &mut Source) -> Result<(), String>
64
+ e.g. frontEnd on a program with `trait Actor(x: Int)` + a record embedding it -> Ok, traits lowered (same AST the loader produced before)
65
+ e.g. frontEnd on a program using an unknown union member -> Err containing the old resolveMembers message
66
+ **Test code:** lower_test.rs: the e.g. lines; loader_test binary unchanged.
67
+ **Steps:** move loader.rs ≈84-100 (lowerTraits, lowerCtors, inlineVariantTypeErrors, resolveMembers, prepareForms, errors joined with "\n") into `frontEnd`; the loader calls it.
68
+
69
+ ### T2 parseSource runs the front end · Complexity 4 · Est. context 45k
70
+ **Owns:** plum-core/src/parser.rs, plum-core/tests/*.rs except lower_test.rs, plum-checker/tests/*.rs except fusion_pipeline_test.rs, plum-checker/tests/features/*, plum-lsp/src/main.rs, plum-lsp/src/completion.rs
71
+ **Components:** 2
72
+ **LOC owned:** 80
73
+ **Tests:** fast
74
+ **Model:** sonnet
75
+ **Consumes:** lower::frontEnd(source: &mut Source) -> Result<(), String> (T1)
76
+ **Produces:** none
77
+ **Review Focus:** `single_file_equals_loader`: for every plum-examples file without imports, `parseSource` gives the same AST as `loadAndMerge` (add this as a test in plum-core/tests/loader_test.rs); errors stay discarded in parseSource as today (callers that need them call frontEnd).
78
+ **Test code:** all plum-core, plum-checker, plum-lsp test binaries pass; any test whose expectation relied on an unlowered AST switches to `parseSourceRaw` (list them).
79
+
80
+ ### T3 tests lowering out of codegen · Complexity 4 · Est. context 45k
81
+ **Owns:** plum-core/src/tests_lower.rs, plum-core/src/lib.rs, plum-wasm-codegen/src/lib.rs, plum-core/tests/tests_lower_test.rs
82
+ **Components:** 1
83
+ **LOC owned:** 260
84
+ **Tests:** slow
85
+ **Model:** sonnet
86
+ **Consumes:** lower::frontEnd(source: &mut Source) -> Result<(), String> (T1)
87
+ **Produces:** tests_lower::lowerTests(source: &Source) -> TestProgram
88
+ e.g. lowerTests(src with `test "a" = expect 1 == 1`) -> TestProgram { names: ["a"], source with one generated fn }
89
+ **Review Focus:** `assert_names_per_compile`: generated temp names start at 0 for each call (no process-global ASSERT_TMP); `plum test` output unchanged.
90
+ **Test code:** tests_lower_test.rs for the e.g. line + two calls give identical output; cli assert_once_test/test_scope_test pass; `plum test plum-std plum-examples` 678 passed.
91
+ **Steps:** move desugarTestsToFns, desugarAssertStmt, assertFailureMessage, TestProgram (codegen lib.rs ≈1812-2076) to plum-core; codegen re-exports `TestProgram`/`desugarTests` for now (T4 removes them).
92
+
93
+ ## Package B: one compile call
94
+ **Probe entry:** run=bash scripts/wasm-hashes.sh
95
+
96
+ ### T4 compile options · Complexity 5 · Est. context 50k
97
+ **Owns:** plum-wasm-codegen/src/lib.rs, plum-checker/src/lib.rs
98
+ **Components:** 2
99
+ **LOC owned:** 120
100
+ **Tests:** slow
101
+ **Model:** sonnet
102
+ **Produces:** plum_wasm_codegen::compile(source: &Source, opts: &CompileOptions) -> Result<Output, String> · CompileOptions { tests: bool, fusion: FusionOptions } · Output { wasm: Vec<u8>, fn_keys: Vec<String>, test_names: Vec<String> } · plum_checker::typedSourceWith(source: &Source, fusion: FusionOptions) -> CheckResult<TypedInfo>
103
+ e.g. compile(src, &CompileOptions::default()) -> Ok(Output{wasm, test_names: [], ..}) same bytes as compileSource(src)
104
+ e.g. compile(src_with_tests, &CompileOptions{tests: true, ..}) -> test_names ["a"]
105
+ **Review Focus:** `no_fusion_thread_local`: `FUSION_OPTIONS`, `setFusionOptions`, `withFusionOptions` removed from plum-checker; `typedSource(s)` == `typedSourceWith(s, FusionOptions::default())`.
106
+ **Test code:** unit tests for the e.g. lines in codegen lib.rs `#[cfg(test)]`; codegen test binaries pass.
107
+ **Steps:** keep `compileSource(source)` as a one-line wrapper over `compile`; delete compileTyped, compileSourceWithFunctions, compileTestSource, compileTestTyped and the desugarTests re-export once T5 moves their callers (mark them `#[deprecated]` thin wrappers in this task so callers still build).
108
+
109
+ ### T5 compile callers · Complexity 3 · Est. context 40k
110
+ **Owns:** plum-cli/src/main.rs, plum-wasm-codegen/tests/codegen_tests.rs, plum-checker/tests/fusion_pipeline_test.rs, plum-game/tests/host_test.rs
111
+ **Components:** 1
112
+ **LOC owned:** 60
113
+ **Tests:** slow
114
+ **Model:** sonnet
115
+ **Consumes:** plum_wasm_codegen::compile(source: &Source, opts: &CompileOptions) -> Result<Output, String> (T4) · plum_checker::typedSourceWith(source: &Source, fusion: FusionOptions) -> CheckResult<TypedInfo> (T4)
116
+ **Produces:** none
117
+ **Review Focus:** `cli_flags`: `--no-fusion` and `--explain-fusion` still work (run `plum run` on an example with each and compare to master's output).
118
+ **Test code:** cli test binaries, codegen_tests, fusion_pipeline_test, plum-game host_test pass; no `#[deprecated]` warnings left; then delete the deprecated wrappers from codegen lib.rs is T7's job (it owns lib.rs next).
119
+
120
+ ## Package C: tidy and remove Paren
121
+ **Probe entry:** run=bash scripts/wasm-hashes.sh
122
+
123
+ ### T6 last walker helpers into visit · Complexity 2 · Est. context 30k
124
+ **Owns:** plum-core/src/visit.rs, plum-core/src/simplify.rs, plum-core/src/drive.rs
125
+ **Components:** 1
126
+ **LOC owned:** 50
127
+ **Tests:** fast
128
+ **Model:** sonnet
129
+ **Produces:** visit::walkRef(e: &Expr, f: &mut dyn FnMut(&Expr)) · visit::walkBlockRef(b: &Block, f: &mut dyn FnMut(&Expr)) · visit::argRef(a: &Arg) -> &Expr · visit::isTag(n: &str) -> bool
130
+ **Test code:** visit_test, drive_test, simplify_test pass; hashes identical.
131
+ **Steps:** move simplify's pub(crate) walkRef/walkBlockRef/argRef/isTag into visit.rs; drive and simplify import them.
132
+
133
+ ### T7 remove Expr::Paren and deprecated wrappers · Complexity 4 · Est. context 50k
134
+ **Owns:** plum-core/src/*.rs, plum-checker/src/*.rs, plum-wasm-codegen/src/lib.rs, plum-lsp/src/*.rs, plum-core/tests/parser_test.rs
135
+ **Components:** 2
136
+ **LOC owned:** 90
137
+ **Tests:** slow
138
+ **Model:** sonnet
139
+ **Produces:** none
140
+ **Review Focus:** `paren_gone`: `grep -rn "Paren" plum-*/src` has no `Expr::Paren`; precedence still right: `(1 + 2) * 3` evaluates to 9 (add to e2e via an existing example or parser_test AST check); tail position through parentheses still marked (tail_test passes).
141
+ **Test code:** all workspace binaries pass; hashes identical.
142
+ **Steps:** parser returns the inner expression for parenthesized_expression (parser.rs ≈1389); delete the variant and every arm (~46); delete codegen's deprecated compile wrappers from T4.
143
+
144
+ ## Package D: verify
145
+ **Probe entry:** run=bash scripts/wasm-hashes.sh
146
+
147
+ ### T8 verify 2b · Complexity 2 · Est. context 30k
148
+ **Owns:** docs/plans/2026-11-04-compiler-cleanup-phase2b.results.md
149
+ **Components:** 1
150
+ **LOC owned:** 40
151
+ **Tests:** slow
152
+ **Model:** sonnet
153
+ **Produces:** none
154
+ **Test code:** all workspace test binaries, plum-game host_test, `plum build` runtime, `plum test plum-std plum-examples` 678; hashes at start sha == final tree.
155
+ **Steps:** results file: hash diff, test totals, Rust LOC and test counts vs the start sha.
156
+
157
+ ## Review Focus
158
+ - Package A: deep, state — single-file vs loader divergence, test name generation.
159
+ - Package B: protocol — CLI flags, public API of plum-wasm-codegen used by cli/lsp/game tests.
160
+ - Package C: deep — operator precedence and tail position without Paren.
161
+
162
+ ## Self-review
163
+ Covers the phase 2 rows left from 2a: one front end (T1, T2, T3), one compile call (T4, T5), Paren removal (T7, moved from phase 1), visit helpers (T6, from a 2a ruling).