plum

#treesitter#compiler#wasm

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

1ccf9bd — Peter John 2026-07-20T12:59:58+05:30 
docs: add design spec for grammar trailing-statement and tail-position gaps
Files changed (1) hide show
  1. docs/superpowers/specs/2026-07-20-tail-position-and-grammar-gaps-design.md +116 -0
docs/superpowers/specs/2026-07-20-tail-position-and-grammar-gaps-design.md ADDED
@@ -0,0 +1,116 @@
1
+ # Fix two pre-existing gaps: grammar trailing-statement limitation, codegen tail-position value drop
2
+
3
+ ## Problem
4
+
5
+ Two real, pre-existing defects were discovered (and worked around, not fixed) while implementing
6
+ general enum support:
7
+
8
+ 1. **Grammar**: a multi-line indented function/method body's `_statement` rule
9
+ (`tooling/tree-sitter-plum/grammar.js:166-179`) only accepts `$.primary_expression` as one of
10
+ its alternatives, not the full `$.expression`. A bare comparison (`a == b`), boolean operator
11
+ (`a && b`), or ternary (`a ? b : c`) used as a statement — most commonly the final line of an
12
+ indented body — fails to parse: the operator is dropped and the parser emits an `ERROR` node,
13
+ silently splitting what should be one expression into two separate statements.
14
+
15
+ 2. **Codegen**: `plum-wasm-codegen`'s `compile_block_as_fn_body` (`plum-wasm-codegen/src/lib.rs`)
16
+ only preserves a function's return value when the body's *literal* last statement is a bare
17
+ `ast::Stmt::Expr`. A `Stmt::If` or `Stmt::Match` in that same tail position compiles each arm as
18
+ an ordinary statement block (`BlockType::Empty`), so a bare tail expression inside an arm is
19
+ `Drop`ped instead of left as the function's result. This produces wasm that fails
20
+ `wasmparser::validate` — a loud failure in the test suite, but `plum-cli`'s `compile` subcommand
21
+ (`plum-cli/src/main.rs`) writes `compile_source`'s output bytes to disk without validating them,
22
+ so a real user hitting this today gets a corrupt, unusable `.wasm` file with no error message at
23
+ all, violating the README's stated guarantee that codegen "reports a clear error rather than
24
+ silently producing wrong code."
25
+
26
+ Both were confirmed by direct reproduction during a prior session (parse-tree dump for gap 1;
27
+ `wasmparser::validate` failure plus a fixed reproduction using explicit `return` for gap 2).
28
+
29
+ ## Fix 1: grammar
30
+
31
+ `$.expression` (`grammar.js:261-269`) is already a strict superset of `$.primary_expression`
32
+ (`grammar.js:271-285`) — every existing alternative reachable through `primary_expression` remains
33
+ reachable through `expression`, and `expression` is already used without incident elsewhere in the
34
+ grammar (`assign`'s values, `assert`, `return`, `if`'s condition). The fix is a single-line change:
35
+
36
+ ```js
37
+ _statement: ($) =>
38
+ choice(
39
+ $.assign,
40
+ $.break,
41
+ $.continue,
42
+ $.assert,
43
+ $.for,
44
+ $.while,
45
+ $.if,
46
+ $.match,
47
+ $.return,
48
+ $.todo,
49
+ $.expression, // was: $.primary_expression
50
+ ),
51
+ ```
52
+
53
+ Regenerate the parser and run the full existing corpus suite to confirm zero regressions (every
54
+ prior `primary_expression`-shaped statement remains valid, since `expression` accepts it too), then
55
+ add new corpus cases proving a bare comparison, boolean-operator, and ternary expression now parse
56
+ as a single statement with no `ERROR` node when used as a body's trailing line.
57
+
58
+ ## Fix 2: codegen tail-position value propagation
59
+
60
+ Add a "value position" compile path that mirrors the existing `Stmt::Expr` special-case in
61
+ `compile_block_as_fn_body`, but recursively for `Stmt::If` and `Stmt::Match`:
62
+
63
+ - A `Stmt::If`/`Stmt::Match` in **value position** (the function body's literal last statement, or
64
+ recursively the last statement of an `if`/`else if`/`else` branch or `match` arm that is itself in
65
+ value position) compiles its condition/subject as today, but compiles each branch/arm using
66
+ `BlockType::Result(result_vt)` instead of `BlockType::Empty`, and compiles that branch's own last
67
+ statement through the same value-position path (recursing into further nested `If`/`Match`, or
68
+ terminating at a bare `Stmt::Expr` — left on the stack, not dropped — or `Stmt::Return`/`Stmt::Todo`
69
+ — already stack-polymorphic in wasm, since control never falls through past them).
70
+ - Every branch of a value-position `If` must have an `else` (a value can't be produced on a
71
+ path that doesn't exist); every `match` arm must resolve to one of the shapes above. If any
72
+ branch/arm's tail statement is some other shape (loop, assignment, etc.) that cannot yield a
73
+ value, `compile_source` returns a clear `Err` (e.g. `"codegen: function 'f' has a control-flow
74
+ path that doesn't produce a return value"`) — never silently falls through to today's
75
+ invalid-wasm behavior.
76
+ - Existing callers of `compile_block_as_fn_body` are unaffected: the recursive value-position
77
+ logic only activates when `has_return_value` is true and the function's tail statement actually
78
+ is `If`/`Match` (today it already special-cases plain `Stmt::Expr`; this generalizes the same
79
+ idea one level of control flow deeper). `If`/`Match` appearing anywhere *except* value position
80
+ (e.g. as a non-tail statement, or a tail statement in a `Unit`-returning function) keep their
81
+ existing, unchanged compilation path.
82
+
83
+ ## Testing plan
84
+
85
+ - **Grammar**: extend `tooling/tree-sitter-plum/test/corpus/` with cases for a bare comparison,
86
+ boolean-operator, and ternary expression as a body's trailing statement; run the full corpus
87
+ suite to confirm no regressions.
88
+ - **Checker**: no changes are needed to `plum-checker` for either fix (both are purely
89
+ parser/codegen concerns) — but re-run the full checker suite (including `examples_test.rs`) to
90
+ confirm nothing regresses, since the grammar change affects what `plum-core`'s parser produces.
91
+ - **Codegen**: add tests (`plum-wasm-codegen/tests/codegen_tests.rs`) executing (via `wasmtime`,
92
+ not just validating) functions whose tail statement is: a `match` with bare-expression arms and
93
+ no explicit `return` (the original `examples/match.plum` shape that motivated this work); an `if`
94
+ with bare-expression branches and no explicit `return`; a *nested* `if` inside a `match` arm
95
+ (recursion through both control-flow kinds); a mix of one arm using explicit `return` and
96
+ another using a bare tail expression (validates the stack-polymorphism claim); and a
97
+ deliberately malformed case (a `match` arm whose tail statement is e.g. a bare `Stmt::Assign`)
98
+ asserting `compile_source` returns the new clear `Err` rather than ever producing bytes.
99
+ - **Examples**: revert the `return`-adding workaround applied to `examples/match.plum`'s five
100
+ functions back to their natural bare-tail-expression form (proving the fix actually restores the
101
+ originally-intended, more idiomatic style) and confirm `plum-wasm-codegen/tests/examples_test.rs`
102
+ still passes.
103
+ - **README**: remove the now-fixed "final statement being a match/if without explicit return"
104
+ bullet from Known Gaps.
105
+
106
+ ## Out of scope
107
+
108
+ - General "does every code path return a value" checking beyond what's needed to make
109
+ value-position `If`/`Match` either compile correctly or fail with a clear error — this is not a
110
+ full control-flow/definite-assignment analysis in `plum-checker`, just a defensive check inside
111
+ the new codegen path.
112
+ - `plum-cli`'s `compile` command validating its own output before writing to disk — worth doing
113
+ separately, but this fix makes the underlying codegen bug (the actual source of the corrupt
114
+ output) go away rather than adding a downstream safety net. Not required for this fix to be
115
+ complete, since the root cause is resolved.
116
+ - Generics monomorphization — a separate, much larger effort, tracked independently.