plum

#treesitter#compiler#wasm

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

4f1f2b6 — Peter John 2026-07-20T21:06:10+05:30 
docs: add design spec for closures
Files changed (1) hide show
  1. docs/superpowers/specs/2026-07-20-closures-design.md +112 -0
docs/superpowers/specs/2026-07-20-closures-design.md ADDED
@@ -0,0 +1,112 @@
1
+ # Closures (`|params| body`)
2
+
3
+ ## Problem
4
+
5
+ `closure` (`|params| body`) already exists as a tree-sitter grammar rule but is unreachable from
6
+ any parse path — it's referenced only in a comment inside `expression`'s choice list. There is
7
+ also no function-value type annotation syntax at all (something like `cb: fn(v: a) -> Bool`,
8
+ as used in the aspirational `libs/std/list.plum`, doesn't parse today). Without closures, any
9
+ higher-order method (`each`, `map`, `filter`, `reduce`, `sort`, ...) on a future `List`/`Map` has
10
+ no way to accept a callback, and `list.plum`'s actual usage (`this.each() |v| { res.write(v.to_str(),
11
+ sep) }`, capturing `res`/`sep` from the enclosing method) can't be expressed at all.
12
+
13
+ ## Scope
14
+
15
+ Full, capturing closures — not just non-capturing function values. `list.plum`'s real usage reads
16
+ enclosing-scope variables from inside a closure literal, so a "closures" feature that can't
17
+ capture wouldn't actually unblock the motivating use case.
18
+
19
+ **Capture semantics: snapshot by value**, not live/shared mutable capture. A captured variable's
20
+ *current value* is copied into the closure's environment at the moment the closure literal is
21
+ evaluated; a later mutation to that variable in the enclosing scope is not visible inside an
22
+ already-created closure. This is simpler (no heap-boxed shared cells needed for ordinary locals)
23
+ and matches every known real usage (reading `res`/`sep`, not reassigning them from inside the
24
+ closure). Live/shared capture is explicitly out of scope — flagged here as a real semantic choice,
25
+ not an oversight, in case a future use case needs it.
26
+
27
+ ## Design
28
+
29
+ ### 1. Grammar (`tooling/tree-sitter-plum`)
30
+
31
+ - Uncomment `$.closure` in `expression`'s choice list (the rule itself — `"|" params "|" body` —
32
+ already exists and needs no changes).
33
+ - Add a new function-value type rule for param type annotations, using **positional types only**
34
+ (no param names in the annotation — types don't need names, and this stays consistent with the
35
+ existing type-list convention `List(a)`/`Pair(a, b)`): `fn(Int) -> Bool`, `fn(a) -> b`. Usable
36
+ wherever a param type currently appears (`each(cb: fn(a))`).
37
+
38
+ ### 2. AST (`plum-core`)
39
+
40
+ - New `ast::Closure { params: Vec<String>, body: Block }`, added as `Expr::Closure(Box<Closure>)`.
41
+ - New `ast::ParamType::Fn(Vec<Type>, Option<Box<Type>>)` variant (param types, param positions
42
+ only — a class field storing a callback value is out of scope, not needed by any current
43
+ example or by `list.plum`'s actual usage, which only ever passes closures as call arguments).
44
+
45
+ ### 3. Checker (`plum-checker`)
46
+
47
+ - `PlumType::TFun(Vec<PlumType>, Box<PlumType>)` already exists and models a closure's type
48
+ directly — no new `PlumType` variant needed.
49
+ - `infer_expr`'s new `Closure` arm: bind each closure param to a fresh `TVar` (matching this
50
+ checker's existing permissive style — it already never really unifies generic parameters, just
51
+ accepts them), infer the body against that environment, return `TFun(param_types, body_type)`.
52
+ - Calling a closure-typed parameter by name (`cb(x)`) already type-checks via the **existing**
53
+ `FnCall` inference path unmodified — `lookup(env, &call.name)` resolves any `TFun`-typed binding
54
+ in scope, regardless of whether it came from a top-level function or a local closure-typed
55
+ param. No changes needed there.
56
+
57
+ ### 4. Codegen (`plum-wasm-codegen`)
58
+
59
+ This is the substantial part — wasm has no closures natively, so it needs:
60
+
61
+ - **New wasm sections**: a function table and an elements segment. `WasmModule` needs
62
+ `add_table`/element-segment support, which doesn't exist yet.
63
+ - **Compiling a closure literal**: every closure literal becomes its own real wasm function,
64
+ registered as a table element, with an **implicit first parameter** — the boxed
65
+ captured-environment pointer — exactly mirroring how a method already receives `self` as an
66
+ implicit first parameter. Inside the closure's compiled body, each captured variable is loaded
67
+ from a fixed offset within that env struct (same load mechanics already used for class fields).
68
+ - **Free-variable analysis**: at the point a closure literal appears in source, walk its body and
69
+ collect every referenced name that is NOT one of the closure's own declared params, a global, or
70
+ a function name — these are the captured variables, snapshotted by value.
71
+ - **Constructing the closure value**: at the closure literal's site, bump-allocate an env struct
72
+ sized to hold the captured variables' current values (same bump-allocator convention as class
73
+ instances), then bump-allocate a small `{table_index: i32, env_pointer: i32}` pair. The pointer
74
+ to that pair **is** the closure's runtime value — a single `i32`, consistent with every other
75
+ reference-shaped value in this compiler (class instances, payload-carrying enum variants).
76
+ - **Calling a closure value**: load `table_index` and `env_pointer` from the pointer, push
77
+ `env_pointer` then the real call arguments, then `call_indirect`. The wasm type index for
78
+ `call_indirect` is resolved from the **caller's** own already-concrete (monomorphized) call
79
+ site — a closure's internal capture layout is opaque to its caller, so calling a higher-order
80
+ function like `each` needs no new monomorphization mechanism: `each`'s own concrete
81
+ instantiation (via the existing generics pass) already fixes what signature any closure passed
82
+ to it must have, and the checker already enforces that at the call site.
83
+
84
+ ## Testing plan
85
+
86
+ - **Grammar**: corpus tests for a closure literal parsing in expression position, and a
87
+ `fn(...)`-typed param annotation parsing correctly (both bracket-free positional-type forms:
88
+ `fn(Int) -> Bool`, `fn(a) -> b`).
89
+ - **Checker**: a closure's inferred `TFun` type flows correctly into a `TFun`-typed binding;
90
+ calling a closure-typed param type-checks via the unmodified `FnCall` path.
91
+ - **Codegen** (compiled and executed via `wasmtime`, matching existing style):
92
+ - A non-capturing closure passed to a function and called.
93
+ - A closure capturing one enclosing local, called after that local's value has changed in the
94
+ enclosing scope — proving the capture is a value snapshot at creation time, not a live
95
+ reference (the closure must see the OLD value, not the new one).
96
+ - A closure passed through an already-generic higher-order function (proving the two features
97
+ compose — the outer function is monomorphized per its own generic param as usual, and any
98
+ closure passed to it is called correctly via `call_indirect` regardless).
99
+ - **Examples**: a minimal, real end-to-end usage (e.g. a small `each`-like helper function taking
100
+ a `fn(a)` callback, called with both a non-capturing and a capturing closure).
101
+
102
+ ## Out of scope
103
+
104
+ - A class field storing a closure value.
105
+ - Live/shared mutable capture (a closure seeing a later mutation to a captured variable) — see
106
+ "Capture semantics" above; this is a deliberate simplification, not a deferred requirement.
107
+ - Multiple closures with genuinely different captured-variable layouts needing different wasm
108
+ function *signatures* at the same `call_indirect` site — not expected to arise given the design
109
+ (the env pointer is always a single opaque `i32` regardless of what it points to), but noted as
110
+ an edge to watch during implementation.
111
+ - Rewriting `libs/std/list.plum`/`map.plum` to actually use this — a separate, subsequent
112
+ follow-up once closures exist.