plum

#treesitter#compiler#wasm

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python


docs/superpowers/specs/2026-07-20-tail-position-and-grammar-gaps-design.md
# Fix two pre-existing gaps: grammar trailing-statement limitation, codegen tail-position value drop

## Problem

Two real, pre-existing defects were discovered (and worked around, not fixed) while implementing
general enum support:

1. **Grammar**: a multi-line indented function/method body's `_statement` rule
   (`tooling/tree-sitter-plum/grammar.js:166-179`) only accepts `$.primary_expression` as one of
   its alternatives, not the full `$.expression`. A bare comparison (`a == b`), boolean operator
   (`a && b`), or ternary (`a ? b : c`) used as a statement — most commonly the final line of an
   indented body — fails to parse: the operator is dropped and the parser emits an `ERROR` node,
   silently splitting what should be one expression into two separate statements.

2. **Codegen**: `plum-wasm-codegen`'s `compile_block_as_fn_body` (`plum-wasm-codegen/src/lib.rs`)
   only preserves a function's return value when the body's *literal* last statement is a bare
   `ast::Stmt::Expr`. A `Stmt::If` or `Stmt::Match` in that same tail position compiles each arm as
   an ordinary statement block (`BlockType::Empty`), so a bare tail expression inside an arm is
   `Drop`ped instead of left as the function's result. This produces wasm that fails
   `wasmparser::validate` — a loud failure in the test suite, but `plum-cli`'s `compile` subcommand
   (`plum-cli/src/main.rs`) writes `compile_source`'s output bytes to disk without validating them,
   so a real user hitting this today gets a corrupt, unusable `.wasm` file with no error message at
   all, violating the README's stated guarantee that codegen "reports a clear error rather than
   silently producing wrong code."

Both were confirmed by direct reproduction during a prior session (parse-tree dump for gap 1;
`wasmparser::validate` failure plus a fixed reproduction using explicit `return` for gap 2).

## Fix 1: grammar

`$.expression` (`grammar.js:261-269`) is already a strict superset of `$.primary_expression`
(`grammar.js:271-285`) — every existing alternative reachable through `primary_expression` remains
reachable through `expression`, and `expression` is already used without incident elsewhere in the
grammar (`assign`'s values, `assert`, `return`, `if`'s condition). The fix is a single-line change:

```js
_statement: ($) =>
  choice(
    $.assign,
    $.break,
    $.continue,
    $.assert,
    $.for,
    $.while,
    $.if,
    $.match,
    $.return,
    $.todo,
    $.expression,   // was: $.primary_expression
  ),
```

Regenerate the parser and run the full existing corpus suite to confirm zero regressions (every
prior `primary_expression`-shaped statement remains valid, since `expression` accepts it too), then
add new corpus cases proving a bare comparison, boolean-operator, and ternary expression now parse
as a single statement with no `ERROR` node when used as a body's trailing line.

## Fix 2: codegen tail-position value propagation

Add a "value position" compile path that mirrors the existing `Stmt::Expr` special-case in
`compile_block_as_fn_body`, but recursively for `Stmt::If` and `Stmt::Match`:

- A `Stmt::If`/`Stmt::Match` in **value position** (the function body's literal last statement, or
  recursively the last statement of an `if`/`else if`/`else` branch or `match` arm that is itself in
  value position) compiles its condition/subject as today, but compiles each branch/arm using
  `BlockType::Result(result_vt)` instead of `BlockType::Empty`, and compiles that branch's own last
  statement through the same value-position path (recursing into further nested `If`/`Match`, or
  terminating at a bare `Stmt::Expr` — left on the stack, not dropped — or `Stmt::Return`/`Stmt::Todo`
  — already stack-polymorphic in wasm, since control never falls through past them).
- Every branch of a value-position `If` must have an `else` (a value can't be produced on a
  path that doesn't exist); every `match` arm must resolve to one of the shapes above. If any
  branch/arm's tail statement is some other shape (loop, assignment, etc.) that cannot yield a
  value, `compile_source` returns a clear `Err` (e.g. `"codegen: function 'f' has a control-flow
  path that doesn't produce a return value"`) — never silently falls through to today's
  invalid-wasm behavior.
- Existing callers of `compile_block_as_fn_body` are unaffected: the recursive value-position
  logic only activates when `has_return_value` is true and the function's tail statement actually
  is `If`/`Match` (today it already special-cases plain `Stmt::Expr`; this generalizes the same
  idea one level of control flow deeper). `If`/`Match` appearing anywhere *except* value position
  (e.g. as a non-tail statement, or a tail statement in a `Unit`-returning function) keep their
  existing, unchanged compilation path.

## Testing plan

- **Grammar**: extend `tooling/tree-sitter-plum/test/corpus/` with cases for a bare comparison,
  boolean-operator, and ternary expression as a body's trailing statement; run the full corpus
  suite to confirm no regressions.
- **Checker**: no changes are needed to `plum-checker` for either fix (both are purely
  parser/codegen concerns) — but re-run the full checker suite (including `examples_test.rs`) to
  confirm nothing regresses, since the grammar change affects what `plum-core`'s parser produces.
- **Codegen**: add tests (`plum-wasm-codegen/tests/codegen_tests.rs`) executing (via `wasmtime`,
  not just validating) functions whose tail statement is: a `match` with bare-expression arms and
  no explicit `return` (the original `examples/match.plum` shape that motivated this work); an `if`
  with bare-expression branches and no explicit `return`; a *nested* `if` inside a `match` arm
  (recursion through both control-flow kinds); a mix of one arm using explicit `return` and
  another using a bare tail expression (validates the stack-polymorphism claim); and a
  deliberately malformed case (a `match` arm whose tail statement is e.g. a bare `Stmt::Assign`)
  asserting `compile_source` returns the new clear `Err` rather than ever producing bytes.
- **Examples**: revert the `return`-adding workaround applied to `examples/match.plum`'s five
  functions back to their natural bare-tail-expression form (proving the fix actually restores the
  originally-intended, more idiomatic style) and confirm `plum-wasm-codegen/tests/examples_test.rs`
  still passes.
- **README**: remove the now-fixed "final statement being a match/if without explicit return"
  bullet from Known Gaps.

## Out of scope

- General "does every code path return a value" checking beyond what's needed to make
  value-position `If`/`Match` either compile correctly or fail with a clear error — this is not a
  full control-flow/definite-assignment analysis in `plum-checker`, just a defensive check inside
  the new codegen path.
- `plum-cli`'s `compile` command validating its own output before writing to disk — worth doing
  separately, but this fix makes the underlying codegen bug (the actual source of the corrupt
  output) go away rather than adding a downstream safety net. Not required for this fix to be
  complete, since the root cause is resolved.
- Generics monomorphization — a separate, much larger effort, tracked independently.