plum
git clone https://git.pyrossh.dev/plum
A statically typed, imperative programming language inspired by rust, python
docs/superpowers/plans/2026-07-20-closures.md
| 038eebd | 1 | # Closures Implementation Plan |
| 038eebd | 2 | |
| 038eebd | 3 | > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. |
| 038eebd | 4 | |
| 038eebd | 5 | **Goal:** Make `|params| body` closure literals — full, capturing closures with snapshot-by-value semantics, usable as an ordinary call argument — parse, type-check, and compile to working wasm. |
| 038eebd | 6 | |
| 038eebd | 7 | **Architecture:** Grammar wires the already-existing (but unreachable) `closure` rule into expression position and adds a new `fn(...)` function-value type annotation. The AST gains `Expr::Closure` and `ParamType::Fn`. The checker's existing `PlumType::TFun` models a closure's type directly, and calling a closure-typed binding by name already works through the unmodified `FnCall` inference path. Codegen is the substantial part: wasm has no native closures, so every closure literal compiles to its own real wasm function (registered in a new function table, with an implicit first parameter — the captured-environment pointer, mirroring how a method already receives `self`), and a closure *value* at runtime is a single `i32` pointer to a heap-allocated `{table_index, env_pointer}` pair. Calling a closure value loads both fields and does `call_indirect`. |
| 038eebd | 8 | |
| 038eebd | 9 | **Tech Stack:** Rust (workspace: `plum-core`, `plum-checker`, `plum-wasm-codegen`), tree-sitter grammar, `wasm-encoder` (already a dependency; this plan uses its `TableSection`/`ElementSection`/`Elements`/`Instruction::CallIndirect`, none of which this codebase uses yet). |
| 038eebd | 10 | |
| 038eebd | 11 | ## Global Constraints |
| 038eebd | 12 | |
| 038eebd | 13 | - **Scope**: closures as expressions, passable as an ordinary call argument (`each(|v| ...)`) — NOT the trailing-closure calling convention (`each() |v| ...`) shown in `libs/std/list.plum`'s aspirational draft; NOT a class field storing a closure value. Capture is snapshot-by-value (a captured variable's value at closure-creation time), not live/shared mutation. |
| 038eebd | 14 | - Function-type annotations use **positional types only** — `fn(Int) -> Bool`, `fn(a) -> b` — no param names inside the annotation. |
| 038eebd | 15 | - Task 5 (closure discovery + codegen) is substantial new algorithmic code, not a modification of existing tested logic. Its design is sound but should be validated primarily through that task's own tests (TDD), the same posture the generics-monomorphization plan took for its hardest task — expect to need judgment calls during implementation, and report BLOCKED/NEEDS_CONTEXT rather than papering over a genuine design gap, exactly as that plan's precedent established. |
| 038eebd | 16 | - Follow existing code style: terse one-line "why" comments only where non-obvious; error messages use the existing `"codegen: ..."` / `"monomorphize: ..."` prefixes as appropriate. |
| 038eebd | 17 | - Every task must leave `cargo test --workspace` and `npx --yes tree-sitter-cli test` (from `tooling/tree-sitter-plum/`) green before moving to the next task. |
| 038eebd | 18 | |
| 038eebd | 19 | --- |
| 038eebd | 20 | |
| 038eebd | 21 | ### Task 1: Grammar — wire closures into expression position, add function-value type syntax |
| 038eebd | 22 | |
| 038eebd | 23 | **Files:** |
| 038eebd | 24 | - Modify: `tooling/tree-sitter-plum/grammar.js` |
| 038eebd | 25 | - Test: `tooling/tree-sitter-plum/test/corpus/` (new cases) |
| 038eebd | 26 | |
| 038eebd | 27 | **Interfaces:** |
| 038eebd | 28 | - Consumes: nothing from other tasks. |
| 038eebd | 29 | - Produces: a `closure` node reachable from `expression`, and a new `fn_value_type` node reachable from `param`'s type field. Neither `plum-core`'s parser nor its AST need any changes yet — that's Task 2. |
| 038eebd | 30 | |
| 038eebd | 31 | - [ ] **Step 1: Uncomment `$.closure` in `expression`'s choice list** |
| 038eebd | 32 | |
| 038eebd | 33 | Find (in `grammar.js`): |
| 038eebd | 34 | |
| 038eebd | 35 | ```js |
| 038eebd | 36 | expression: ($) => |
| 038eebd | 37 | choice( |
| 038eebd | 38 | $.comparison_operator, |
| 038eebd | 39 | $.not_operator, |
| 038eebd | 40 | $.boolean_operator, |
| 038eebd | 41 | // $.closure, |
| 038eebd | 42 | $.primary_expression, |
| 038eebd | 43 | $.ternary_expression, |
| 038eebd | 44 | ), |
| 038eebd | 45 | ``` |
| 038eebd | 46 | |
| 038eebd | 47 | Change to: |
| 038eebd | 48 | |
| 038eebd | 49 | ```js |
| 038eebd | 50 | expression: ($) => |
| 038eebd | 51 | choice( |
| 038eebd | 52 | $.comparison_operator, |
| 038eebd | 53 | $.not_operator, |
| 038eebd | 54 | $.boolean_operator, |
| 038eebd | 55 | $.closure, |
| 038eebd | 56 | $.primary_expression, |
| 038eebd | 57 | $.ternary_expression, |
| 038eebd | 58 | ), |
| 038eebd | 59 | ``` |
| 038eebd | 60 | |
| 038eebd | 61 | - [ ] **Step 2: Add the `fn_value_type` rule and wire it into `param`** |
| 038eebd | 62 | |
| 038eebd | 63 | Find: |
| 038eebd | 64 | |
| 038eebd | 65 | ```js |
| 038eebd | 66 | param: ($) => |
| 038eebd | 67 | seq( |
| 038eebd | 68 | field("name", $.var_identifier), |
| 038eebd | 69 | ":", |
| 038eebd | 70 | field("type", choice($.type, $.variadic_type)), |
| 038eebd | 71 | optional(seq("=", field("value", $.expression))), |
| 038eebd | 72 | ), |
| 038eebd | 73 | ``` |
| 038eebd | 74 | |
| 038eebd | 75 | Change to: |
| 038eebd | 76 | |
| 038eebd | 77 | ```js |
| 038eebd | 78 | param: ($) => |
| 038eebd | 79 | seq( |
| 038eebd | 80 | field("name", $.var_identifier), |
| 038eebd | 81 | ":", |
| 038eebd | 82 | field("type", choice($.type, $.variadic_type, $.fn_value_type)), |
| 038eebd | 83 | optional(seq("=", field("value", $.expression))), |
| 038eebd | 84 | ), |
| 038eebd | 85 | |
| 038eebd | 86 | fn_value_type: ($) => |
| 038eebd | 87 | seq( |
| 038eebd | 88 | "fn", |
| 038eebd | 89 | "(", |
| 038eebd | 90 | field("params", optional(commaSep1($.type))), |
| 038eebd | 91 | ")", |
| 038eebd | 92 | optional(seq("->", field("returns", $.type))), |
| 038eebd | 93 | ), |
| 038eebd | 94 | ``` |
| 038eebd | 95 | |
| 038eebd | 96 | (Place `fn_value_type` near `variadic_type`'s definition, a few lines above `param`. The explicit `"params"`/`"returns"` field names — the same idiom `fn`'s own `field("returns", ...)` already uses — let `parser.rs` (Task 2) distinguish the return type from the param types unambiguously by field, rather than by counting/positional-guessing among same-kind `"type"` children.) |
| 038eebd | 97 | |
| 038eebd | 98 | - [ ] **Step 3: Regenerate and run the existing corpus suite** |
| 038eebd | 99 | |
| 038eebd | 100 | ```bash |
| 038eebd | 101 | cd tooling/tree-sitter-plum && npx --yes tree-sitter-cli generate && npx --yes tree-sitter-cli test |
| 038eebd | 102 | ``` |
| 038eebd | 103 | |
| 038eebd | 104 | Expected: generation succeeds with no unresolved-conflict errors, and all pre-existing corpus cases still pass. (The `closure` rule's own body — `"|" params "|" body` — already exists unchanged; only its reachability and the new `fn_value_type` rule are new. If `generate` reports a conflict — e.g. between `|` as the closure delimiter and `|` as the bitwise-or binary operator in expression position — read the reported example carefully; a `conflicts: [[$.closure, ...]]` entry or precedence adjustment may be needed. Don't guess a fix blindly; the existing `conflicts` array in `grammar.js` (already used for the `fn_call`/`class_call` ambiguity from prior work) is the established idiom for this.) |
| 038eebd | 105 | |
| 038eebd | 106 | - [ ] **Step 4: Add corpus cases** |
| 038eebd | 107 | |
| 038eebd | 108 | Append to `tooling/tree-sitter-plum/test/corpus/function.txt` (input halves — the next step fills in expected trees): |
| 038eebd | 109 | |
| 038eebd | 110 | ``` |
| 038eebd | 111 | ================================================================================ |
| 038eebd | 112 | function - closure literal in expression position |
| 038eebd | 113 | ================================================================================ |
| 038eebd | 114 | |
| 038eebd | 115 | useClosure() -> Bool = |
| 038eebd | 116 | cb = |v| |
| 038eebd | 117 | True |
| 038eebd | 118 | cb(5) |
| 038eebd | 119 | |
| 038eebd | 120 | -------------------------------------------------------------------------------- |
| 038eebd | 121 | ================================================================================ |
| 038eebd | 122 | function - closure literal with no params |
| 038eebd | 123 | ================================================================================ |
| 038eebd | 124 | |
| 038eebd | 125 | useClosure() -> Bool = |
| 038eebd | 126 | cb = || |
| 038eebd | 127 | True |
| 038eebd | 128 | cb() |
| 038eebd | 129 | |
| 038eebd | 130 | -------------------------------------------------------------------------------- |
| 038eebd | 131 | ================================================================================ |
| 038eebd | 132 | function - function-value type param annotation |
| 038eebd | 133 | ================================================================================ |
| 038eebd | 134 | |
| 038eebd | 135 | each(cb: fn(Int)) -> Bool = |
| 038eebd | 136 | True |
| 038eebd | 137 | |
| 038eebd | 138 | -------------------------------------------------------------------------------- |
| 038eebd | 139 | ================================================================================ |
| 038eebd | 140 | function - function-value type param annotation with generic types and return |
| 038eebd | 141 | ================================================================================ |
| 038eebd | 142 | |
| 038eebd | 143 | each(cb: fn(a) -> b) -> Bool = |
| 038eebd | 144 | True |
| 038eebd | 145 | |
| 038eebd | 146 | -------------------------------------------------------------------------------- |
| 038eebd | 147 | ``` |
| 038eebd | 148 | |
| 038eebd | 149 | - [ ] **Step 5: Generate expected trees and verify** |
| 038eebd | 150 | |
| 038eebd | 151 | ```bash |
| 038eebd | 152 | cd tooling/tree-sitter-plum && npx --yes tree-sitter-cli test -u -f "closure literal" && npx --yes tree-sitter-cli test -u -f "function-value type" |
| 038eebd | 153 | ``` |
| 038eebd | 154 | |
| 038eebd | 155 | Open `test/corpus/function.txt` and confirm each new case's generated tree has no `ERROR`/`MISSING` node, and that the closure literal appears as a single `closure` node (not split), and `fn_value_type` appears as a single node containing its param types and optional return type. |
| 038eebd | 156 | |
| 038eebd | 157 | - [ ] **Step 6: Run the full corpus suite and the Rust workspace suite** |
| 038eebd | 158 | |
| 038eebd | 159 | ```bash |
| 038eebd | 160 | cd tooling/tree-sitter-plum && npx --yes tree-sitter-cli test |
| 038eebd | 161 | cargo test --workspace |
| 038eebd | 162 | ``` |
| 038eebd | 163 | |
| 038eebd | 164 | Expected: both green. (The workspace suite should be unaffected — no Rust code changes yet — but confirm nothing regresses via the parser's generic wrapper-node handling, the same way past grammar-only changes this session were verified.) |
| 038eebd | 165 | |
| 038eebd | 166 | - [ ] **Step 7: Commit** |
| 038eebd | 167 | |
| 038eebd | 168 | ```bash |
| 038eebd | 169 | git add tooling/tree-sitter-plum/grammar.js tooling/tree-sitter-plum/test/corpus/function.txt |
| 038eebd | 170 | git add tooling/tree-sitter-plum/src # generated parser.c etc, only if tracked — check git status first |
| 038eebd | 171 | git commit -m "feat(tree-sitter-plum): wire closure literals into expression position; add fn(...) type syntax" |
| 038eebd | 172 | ``` |
| 038eebd | 173 | |
| 038eebd | 174 | --- |
| 038eebd | 175 | |
| 038eebd | 176 | ### Task 2: AST + Parser — `Expr::Closure`, `ParamType::Fn` |
| 038eebd | 177 | |
| 038eebd | 178 | **Files:** |
| 038eebd | 179 | - Modify: `plum-core/src/ast.rs` |
| 038eebd | 180 | - Modify: `plum-core/src/parser.rs` |
| 038eebd | 181 | - Test: `plum-core/tests/` (a new or extended integration test parsing a closure and a `fn(...)` param — check existing test file conventions, e.g. `formatter_test.rs`, for how this crate structures its own tests; there is no dedicated parser test file today, so add one, `plum-core/tests/parser_test.rs`, following the same `AstParser::new(src)` + `tree_sitter::Parser` pattern already used in `plum-checker`/`plum-wasm-codegen`'s own test helpers) |
| 038eebd | 182 | |
| 038eebd | 183 | **Interfaces:** |
| 038eebd | 184 | - Consumes: the `closure`/`fn_value_type` grammar nodes from Task 1. |
| 038eebd | 185 | - Produces: |
| 038eebd | 186 | ```rust |
| 038eebd | 187 | pub struct Closure { pub params: Vec<String>, pub body: Block } |
| 038eebd | 188 | // added to Expr: |
| 038eebd | 189 | Closure(Box<Closure>), |
| 038eebd | 190 | // added to ParamType: |
| 038eebd | 191 | Fn(Vec<Type>, Option<Box<Type>>), |
| 038eebd | 192 | ``` |
| 038eebd | 193 | Task 3 (checker) and Task 5 (codegen) both match on these. |
| 038eebd | 194 | |
| 038eebd | 195 | - [ ] **Step 1: Write failing tests** |
| 038eebd | 196 | |
| 038eebd | 197 | Create `plum-core/tests/parser_test.rs`: |
| 038eebd | 198 | |
| 038eebd | 199 | ```rust |
| 038eebd | 200 | use plum_core::ast::*; |
| 038eebd | 201 | use plum_core::AstParser; |
| 038eebd | 202 | |
| 038eebd | 203 | fn parse(src: &str) -> Source { |
| 038eebd | 204 | let mut parser = tree_sitter::Parser::new(); |
| 038eebd | 205 | parser.set_language(&tree_sitter_plum::LANGUAGE.into()).unwrap(); |
| 038eebd | 206 | let tree = parser.parse(src, None).unwrap(); |
| 038eebd | 207 | assert!(!tree.root_node().has_error(), "parse error:\n{}", tree.root_node().to_sexp()); |
| 038eebd | 208 | let ap = AstParser::new(src); |
| 038eebd | 209 | ap.parse_source(tree.root_node()) |
| 038eebd | 210 | } |
| 038eebd | 211 | |
| 038eebd | 212 | fn only_fn(source: &Source) -> &Fn { |
| 038eebd | 213 | source.items.iter().find_map(|i| match i { Item::Fn(f) => Some(f), _ => None }).expect("expected a Fn item") |
| 038eebd | 214 | } |
| 038eebd | 215 | |
| 038eebd | 216 | #[test] |
| 038eebd | 217 | fn closure_literal_parses_with_params_and_body() { |
| 038eebd | 218 | let src = "\ |
| 038eebd | 219 | useClosure() -> Bool = |
| 038eebd | 220 | cb = |v| |
| 038eebd | 221 | True |
| 038eebd | 222 | cb(5) |
| 038eebd | 223 | "; |
| 038eebd | 224 | let source = parse(src); |
| 038eebd | 225 | let f = only_fn(&source); |
| 038eebd | 226 | let FnBody::Block(block) = &f.body else { panic!("expected a block body") }; |
| 038eebd | 227 | let Stmt::Assign(assign) = &block.stmts[0] else { panic!("expected an assign statement") }; |
| 038eebd | 228 | let Expr::Closure(closure) = &assign.values[0] else { panic!("expected a closure expression, got {:?}", assign.values[0]) }; |
| 038eebd | 229 | assert_eq!(closure.params, vec!["v".to_string()]); |
| 038eebd | 230 | assert_eq!(closure.body.stmts.len(), 1); |
| 038eebd | 231 | } |
| 038eebd | 232 | |
| 038eebd | 233 | #[test] |
| 038eebd | 234 | fn closure_literal_parses_with_no_params() { |
| 038eebd | 235 | let src = "\ |
| 038eebd | 236 | useClosure() -> Bool = |
| 038eebd | 237 | cb = || |
| 038eebd | 238 | True |
| 038eebd | 239 | cb() |
| 038eebd | 240 | "; |
| 038eebd | 241 | let source = parse(src); |
| 038eebd | 242 | let f = only_fn(&source); |
| 038eebd | 243 | let FnBody::Block(block) = &f.body else { panic!("expected a block body") }; |
| 038eebd | 244 | let Stmt::Assign(assign) = &block.stmts[0] else { panic!("expected an assign statement") }; |
| 038eebd | 245 | let Expr::Closure(closure) = &assign.values[0] else { panic!("expected a closure expression") }; |
| 038eebd | 246 | assert!(closure.params.is_empty()); |
| 038eebd | 247 | } |
| 038eebd | 248 | |
| 038eebd | 249 | #[test] |
| 038eebd | 250 | fn fn_value_type_param_parses_with_positional_types_and_return() { |
| 038eebd | 251 | let src = "each(cb: fn(Int) -> Bool) -> Bool =\n True\n"; |
| 038eebd | 252 | let source = parse(src); |
| 038eebd | 253 | let f = only_fn(&source); |
| 038eebd | 254 | let ParamType::Fn(param_types, ret) = &f.params[0].ty else { panic!("expected ParamType::Fn, got {:?}", f.params[0].ty) }; |
| 038eebd | 255 | assert_eq!(param_types.len(), 1); |
| 038eebd | 256 | assert_eq!(param_types[0].name, "Int"); |
| 038eebd | 257 | assert_eq!(ret.as_ref().map(|t| t.name.clone()), Some("Bool".to_string())); |
| 038eebd | 258 | } |
| 038eebd | 259 | |
| 038eebd | 260 | #[test] |
| 038eebd | 261 | fn fn_value_type_param_parses_with_no_return() { |
| 038eebd | 262 | let src = "each(cb: fn(Int)) -> Bool =\n True\n"; |
| 038eebd | 263 | let source = parse(src); |
| 038eebd | 264 | let f = only_fn(&source); |
| 038eebd | 265 | let ParamType::Fn(param_types, ret) = &f.params[0].ty else { panic!("expected ParamType::Fn") }; |
| 038eebd | 266 | assert_eq!(param_types.len(), 1); |
| 038eebd | 267 | assert!(ret.is_none()); |
| 038eebd | 268 | } |
| 038eebd | 269 | ``` |
| 038eebd | 270 | |
| 038eebd | 271 | - [ ] **Step 2: Run to see them fail** |
| 038eebd | 272 | |
| 038eebd | 273 | Run: `cargo test -p plum-core --test parser_test` |
| 038eebd | 274 | Expected: fails to compile — `Expr::Closure`/`ParamType::Fn` don't exist yet. |
| 038eebd | 275 | |
| 038eebd | 276 | - [ ] **Step 3: Add the AST nodes** |
| 038eebd | 277 | |
| 038eebd | 278 | In `plum-core/src/ast.rs`, find the `Expr` enum (it currently ends with variants like `Var(String)`, `TypeName(String)`) and add a new variant: |
| 038eebd | 279 | |
| 038eebd | 280 | ```rust |
| 038eebd | 281 | /// `|params| body` |
| 038eebd | 282 | Closure(Box<Closure>), |
| 038eebd | 283 | ``` |
| 038eebd | 284 | |
| 038eebd | 285 | Add the `Closure` struct near `Block`'s definition: |
| 038eebd | 286 | |
| 038eebd | 287 | ```rust |
| 038eebd | 288 | #[derive(Debug, Clone, PartialEq)] |
| 038eebd | 289 | pub struct Closure { |
| 038eebd | 290 | pub params: Vec<String>, |
| 038eebd | 291 | pub body: Block, |
| 038eebd | 292 | } |
| 038eebd | 293 | ``` |
| 038eebd | 294 | |
| 038eebd | 295 | Find the `ParamType` enum (`Type(Type)`, `Variadic(Type)`) and add: |
| 038eebd | 296 | |
| 038eebd | 297 | ```rust |
| 038eebd | 298 | /// `fn(Int, Str) -> Bool` — a function-value type annotation. Positional types |
| 038eebd | 299 | /// only, no param names (types don't need names). |
| 038eebd | 300 | Fn(Vec<Type>, Option<Box<Type>>), |
| 038eebd | 301 | ``` |
| 038eebd | 302 | |
| 038eebd | 303 | - [ ] **Step 4: Parse `closure` and `fn_value_type` nodes** |
| 038eebd | 304 | |
| 038eebd | 305 | In `plum-core/src/parser.rs`, find `parse_primary_expression`'s match (it currently has arms like `"binary_operator" => ...`, `"fn_call" => ...`) — but `closure` is NOT under `primary_expression` in the grammar, it's a direct alternative of `expression` itself, so it needs handling in `parse_expression` instead. Find: |
| 038eebd | 306 | |
| 038eebd | 307 | ```rust |
| 038eebd | 308 | pub fn parse_expression(&self, node: Node) -> Expr { |
| 038eebd | 309 | let node = self.unwrap_expr_node(node); |
| 038eebd | 310 | match node.kind() { |
| 038eebd | 311 | "comparison_operator" => self.parse_compare(node), |
| 038eebd | 312 | "not_operator" => { |
| 038eebd | 313 | let arg = node.named_child(0) |
| 038eebd | 314 | .map(|n| { let u = self.unwrap_expr_node(n); self.parse_expression(u) }) |
| 038eebd | 315 | .unwrap_or(Expr::Int(0)); |
| 038eebd | 316 | Expr::Not(Box::new(arg)) |
| 038eebd | 317 | } |
| 038eebd | 318 | "boolean_operator" => self.parse_bool_op(node), |
| 038eebd | 319 | "ternary_expression" => self.parse_ternary(node), |
| 038eebd | 320 | _ => self.parse_primary_expression(node), |
| 038eebd | 321 | } |
| 038eebd | 322 | } |
| 038eebd | 323 | ``` |
| 038eebd | 324 | |
| 038eebd | 325 | Change to: |
| 038eebd | 326 | |
| 038eebd | 327 | ```rust |
| 038eebd | 328 | pub fn parse_expression(&self, node: Node) -> Expr { |
| 038eebd | 329 | let node = self.unwrap_expr_node(node); |
| 038eebd | 330 | match node.kind() { |
| 038eebd | 331 | "comparison_operator" => self.parse_compare(node), |
| 038eebd | 332 | "not_operator" => { |
| 038eebd | 333 | let arg = node.named_child(0) |
| 038eebd | 334 | .map(|n| { let u = self.unwrap_expr_node(n); self.parse_expression(u) }) |
| 038eebd | 335 | .unwrap_or(Expr::Int(0)); |
| 038eebd | 336 | Expr::Not(Box::new(arg)) |
| 038eebd | 337 | } |
| 038eebd | 338 | "boolean_operator" => self.parse_bool_op(node), |
| 038eebd | 339 | "ternary_expression" => self.parse_ternary(node), |
| 038eebd | 340 | "closure" => Expr::Closure(Box::new(self.parse_closure(node))), |
| 038eebd | 341 | _ => self.parse_primary_expression(node), |
| 038eebd | 342 | } |
| 038eebd | 343 | } |
| 038eebd | 344 | |
| 038eebd | 345 | fn parse_closure(&self, node: Node) -> Closure { |
| 038eebd | 346 | // closure: "|" var_identifier,* "|" body |
| 038eebd | 347 | let params: Vec<String> = self.children_of_kind(node, "var_identifier") |
| 038eebd | 348 | .into_iter() |
| 038eebd | 349 | .map(|n| self.text(n)) |
| 038eebd | 350 | .collect(); |
| 038eebd | 351 | let body = self.children_of_kind(node, "body") |
| 038eebd | 352 | .into_iter() |
| 038eebd | 353 | .next() |
| 038eebd | 354 | .map(|n| self.parse_block(n)) |
| 038eebd | 355 | .unwrap_or(Block { stmts: vec![] }); |
| 038eebd | 356 | Closure { params, body } |
| 038eebd | 357 | } |
| 038eebd | 358 | ``` |
| 038eebd | 359 | |
| 038eebd | 360 | Find `parse_param` (it currently matches `n.kind() == "variadic_type"` vs. the `else` branch treating everything else as a plain `type`): |
| 038eebd | 361 | |
| 038eebd | 362 | ```rust |
| 038eebd | 363 | fn parse_param(&self, node: Node) -> Param { |
| 038eebd | 364 | // param: var_identifier ":" (type | variadic_type) ("=" expression)? |
| 038eebd | 365 | let name = node.named_child(0).map(|n| self.text(n)).unwrap_or_default(); |
| 038eebd | 366 | let ty = node.named_child(1).map(|n| { |
| 038eebd | 367 | if n.kind() == "variadic_type" { |
| 038eebd | 368 | let inner = n.named_child(0) |
| 038eebd | 369 | .map(|t| self.parse_type(t)) |
| 038eebd | 370 | .unwrap_or(Type { name: String::new(), generics: vec![] }); |
| 038eebd | 371 | ParamType::Variadic(inner) |
| 038eebd | 372 | } else { |
| 038eebd | 373 | ParamType::Type(self.parse_type(n)) |
| 038eebd | 374 | } |
| 038eebd | 375 | }).unwrap_or(ParamType::Type(Type { name: String::new(), generics: vec![] })); |
| 038eebd | 376 | let default = node.named_child(2).map(|n| { |
| 038eebd | 377 | let unwrapped = self.unwrap_expr_node(n); |
| 038eebd | 378 | self.parse_expression(unwrapped) |
| 038eebd | 379 | }); |
| 038eebd | 380 | Param { name, ty, default } |
| 038eebd | 381 | } |
| 038eebd | 382 | ``` |
| 038eebd | 383 | |
| 038eebd | 384 | Change the type-dispatch to also handle `fn_value_type`: |
| 038eebd | 385 | |
| 038eebd | 386 | ```rust |
| 038eebd | 387 | fn parse_param(&self, node: Node) -> Param { |
| 038eebd | 388 | // param: var_identifier ":" (type | variadic_type | fn_value_type) ("=" expression)? |
| 038eebd | 389 | let name = node.named_child(0).map(|n| self.text(n)).unwrap_or_default(); |
| 038eebd | 390 | let ty = node.named_child(1).map(|n| match n.kind() { |
| 038eebd | 391 | "variadic_type" => { |
| 038eebd | 392 | let inner = n.named_child(0) |
| 038eebd | 393 | .map(|t| self.parse_type(t)) |
| 038eebd | 394 | .unwrap_or(Type { name: String::new(), generics: vec![] }); |
| 038eebd | 395 | ParamType::Variadic(inner) |
| 038eebd | 396 | } |
| 038eebd | 397 | "fn_value_type" => self.parse_fn_value_type(n), |
| 038eebd | 398 | _ => ParamType::Type(self.parse_type(n)), |
| 038eebd | 399 | }).unwrap_or(ParamType::Type(Type { name: String::new(), generics: vec![] })); |
| 038eebd | 400 | let default = node.named_child(2).map(|n| { |
| 038eebd | 401 | let unwrapped = self.unwrap_expr_node(n); |
| 038eebd | 402 | self.parse_expression(unwrapped) |
| 038eebd | 403 | }); |
| 038eebd | 404 | Param { name, ty, default } |
| 038eebd | 405 | } |
| 038eebd | 406 | |
| 038eebd | 407 | fn parse_fn_value_type(&self, node: Node) -> ParamType { |
| 038eebd | 408 | // fn_value_type: "fn" "(" field("params", type,*) ")" ("->" field("returns", type))? |
| 038eebd | 409 | // The "returns" field (if present) is a distinct field from "params", so the |
| 038eebd | 410 | // two are disambiguated unambiguously by field name, not by counting/position |
| 038eebd | 411 | // among same-kind "type" children — the same idiom `fn`'s own `returns` field |
| 038eebd | 412 | // already uses. |
| 038eebd | 413 | let returns_node = node.child_by_field_name("returns"); |
| 038eebd | 414 | let param_types: Vec<Type> = self.children_of_kind(node, "type") |
| 038eebd | 415 | .into_iter() |
| 038eebd | 416 | .filter(|n| Some(*n) != returns_node) |
| 038eebd | 417 | .map(|n| self.parse_type(n)) |
| 038eebd | 418 | .collect(); |
| 038eebd | 419 | let ret = returns_node.map(|n| Box::new(self.parse_type(n))); |
| 038eebd | 420 | ParamType::Fn(param_types, ret) |
| 038eebd | 421 | } |
| 038eebd | 422 | ``` |
| 038eebd | 423 | |
| 038eebd | 424 | - [ ] **Step 5: Run parser tests** |
| 038eebd | 425 | |
| 038eebd | 426 | Run: `cargo test -p plum-core --test parser_test` |
| 038eebd | 427 | Expected: all 4 tests pass. If `parse_fn_value_type`'s field extraction needed a grammar tweak (per the note above), go back and apply it, re-running Task 1's corpus tests too. |
| 038eebd | 428 | |
| 038eebd | 429 | - [ ] **Step 6: Run the full workspace suite** |
| 038eebd | 430 | |
| 038eebd | 431 | Run: `cargo test --workspace` |
| 038eebd | 432 | Expected: green (new AST variants are additive; nothing existing matches on `Expr`/`ParamType` exhaustively in a way that would fail to compile — verify this by checking for compile errors from non-exhaustive `match` arms in `plum-checker`/`plum-wasm-codegen`, and add a minimal `_ => ...` fallback arm or explicit handling wherever the compiler flags one). |
| 038eebd | 433 | |
| 038eebd | 434 | - [ ] **Step 7: Commit** |
| 038eebd | 435 | |
| 038eebd | 436 | ```bash |
| 038eebd | 437 | git add plum-core/src/ast.rs plum-core/src/parser.rs plum-core/tests/parser_test.rs |
| 038eebd | 438 | git commit -m "feat(plum-core): parse closure literals and fn(...) type annotations" |
| 038eebd | 439 | ``` |
| 038eebd | 440 | |
| 038eebd | 441 | --- |
| 038eebd | 442 | |
| 038eebd | 443 | ### Task 3: Checker — infer a closure's type; confirm closure calls type-check |
| 038eebd | 444 | |
| 038eebd | 445 | **Files:** |
| 038eebd | 446 | - Modify: `plum-checker/src/lib.rs` |
| 038eebd | 447 | - Test: `plum-checker/tests/checker_tests.rs` |
| 038eebd | 448 | |
| 038eebd | 449 | **Interfaces:** |
| 038eebd | 450 | - Consumes: `ast::Expr::Closure`, `ast::ParamType::Fn` (Task 2). |
| 038eebd | 451 | - Produces: `infer_expr` handles `Expr::Closure`, returning `PlumType::TFun`. No other function signatures change — `check_fn`'s existing `ParamType` match (used to bind each param's type into the local env) needs one new arm for `ParamType::Fn` too, converting it to `PlumType::TFun`. |
| 038eebd | 452 | |
| 038eebd | 453 | - [ ] **Step 1: Write failing tests** |
| 038eebd | 454 | |
| 038eebd | 455 | Append to `plum-checker/tests/checker_tests.rs`: |
| 038eebd | 456 | |
| 038eebd | 457 | ```rust |
| 038eebd | 458 | #[test] |
| 038eebd | 459 | fn closure_literal_infers_as_a_function_type() { |
| 038eebd | 460 | let src = "\ |
| 038eebd | 461 | useClosure() -> Bool = |
| 038eebd | 462 | cb = |v| |
| 038eebd | 463 | True |
| 038eebd | 464 | cb(5) |
| 038eebd | 465 | "; |
| 038eebd | 466 | let source = parse(src); |
| 038eebd | 467 | let result = check_source(&source); |
| 038eebd | 468 | assert!(result.is_ok(), "expected Ok, got {:?}", result.err()); |
| 038eebd | 469 | } |
| 038eebd | 470 | |
| 038eebd | 471 | #[test] |
| 038eebd | 472 | fn fn_value_typed_param_can_be_called() { |
| 038eebd | 473 | let src = "\ |
| 038eebd | 474 | each(cb: fn(Int) -> Bool) -> Bool = |
| 038eebd | 475 | cb(5) |
| 038eebd | 476 | "; |
| 038eebd | 477 | let source = parse(src); |
| 038eebd | 478 | let result = check_source(&source); |
| 038eebd | 479 | assert!(result.is_ok(), "expected Ok, got {:?}", result.err()); |
| 038eebd | 480 | } |
| 038eebd | 481 | |
| 038eebd | 482 | #[test] |
| 038eebd | 483 | fn closure_passed_to_fn_value_typed_param_type_checks() { |
| 038eebd | 484 | let src = "\ |
| 038eebd | 485 | each(cb: fn(Int) -> Bool) -> Bool = |
| 038eebd | 486 | cb(5) |
| 038eebd | 487 | |
| 038eebd | 488 | use() -> Bool = |
| 038eebd | 489 | each(|v| |
| 038eebd | 490 | True) |
| 038eebd | 491 | "; |
| 038eebd | 492 | let source = parse(src); |
| 038eebd | 493 | let result = check_source(&source); |
| 038eebd | 494 | assert!(result.is_ok(), "expected Ok, got {:?}", result.err()); |
| 038eebd | 495 | } |
| 038eebd | 496 | ``` |
| 038eebd | 497 | |
| 038eebd | 498 | - [ ] **Step 2: Run to see them fail** |
| 038eebd | 499 | |
| 038eebd | 500 | Run: `cargo test -p plum-checker --test checker_tests closure` |
| 038eebd | 501 | Expected: fails to compile initially (`infer_expr` doesn't exhaustively handle `Expr::Closure` — a non-exhaustive match compile error) or, once that's stubbed minimally, fails at runtime because `ParamType::Fn` isn't converted to a `PlumType` anywhere yet. |
| 038eebd | 502 | |
| 038eebd | 503 | - [ ] **Step 3: Add `infer_expr`'s `Closure` arm** |
| 038eebd | 504 | |
| 038eebd | 505 | In `plum-checker/src/lib.rs`, find `infer_expr`'s match (it has arms for `Expr::Int`, `Expr::Var`, etc., ending around the `Expr::Attribute` arm). Add: |
| 038eebd | 506 | |
| 038eebd | 507 | ```rust |
| 038eebd | 508 | ast::Expr::Closure(cl) => { |
| 038eebd | 509 | let mut closure_env = env.clone(); |
| 038eebd | 510 | let param_types: Vec<PlumType> = cl.params.iter().map(|p| { |
| 038eebd | 511 | let t = PlumType::TVar(format!("_closure_{}", p)); |
| 038eebd | 512 | closure_env.insert(p.clone(), TypeScheme::mono(t.clone())); |
| 038eebd | 513 | t |
| 038eebd | 514 | }).collect(); |
| 038eebd | 515 | let body_ty = match &cl.body.stmts.last() { |
| 038eebd | 516 | Some(ast::Stmt::Expr(e)) => infer_expr(e, &closure_env, ctx)?, |
| 038eebd | 517 | Some(ast::Stmt::Return(Some(e))) => infer_expr(e, &closure_env, ctx)?, |
| 038eebd | 518 | _ => PlumType::TUnit, |
| 038eebd | 519 | }; |
| 038eebd | 520 | Ok(PlumType::TFun(param_types, Box::new(body_ty))) |
| 038eebd | 521 | } |
| 038eebd | 522 | ``` |
| 038eebd | 523 | |
| 038eebd | 524 | - [ ] **Step 4: Convert `ParamType::Fn` to a `PlumType` wherever `plum_type_from_ast`-style conversion happens for params** |
| 038eebd | 525 | |
| 038eebd | 526 | Find every place in `plum-checker/src/lib.rs` that matches on `ast::ParamType::Type(t) => ...` / `ast::ParamType::Variadic(t) => ...` together (e.g. inside `build_global_tables`, `check_fn`) — there are a few such call sites. For each, add a third arm: |
| 038eebd | 527 | |
| 038eebd | 528 | ```rust |
| 038eebd | 529 | ast::ParamType::Fn(param_types, ret) => PlumType::TFun( |
| 038eebd | 530 | param_types.iter().map(plum_type_from_ast).collect(), |
| 038eebd | 531 | Box::new(ret.as_ref().map(|r| plum_type_from_ast(r)).unwrap_or(PlumType::TUnit)), |
| 038eebd | 532 | ), |
| 038eebd | 533 | ``` |
| 038eebd | 534 | |
| 038eebd | 535 | (Match the exact surrounding style at each call site — some are inline closures passed to `.map()`, some are `match &p.ty { ... }` blocks. Search for `ast::ParamType::Variadic` to find every site that needs the parallel `Fn` arm — the compiler will also flag any non-exhaustive match as a hard error, which is the authoritative list.) |
| 038eebd | 536 | |
| 038eebd | 537 | - [ ] **Step 5: Run checker tests** |
| 038eebd | 538 | |
| 038eebd | 539 | Run: `cargo test -p plum-checker --test checker_tests closure` |
| 038eebd | 540 | Expected: all 3 pass. |
| 038eebd | 541 | |
| 038eebd | 542 | - [ ] **Step 6: Run the full checker crate suite** |
| 038eebd | 543 | |
| 038eebd | 544 | Run: `cargo test -p plum-checker` |
| 038eebd | 545 | Expected: green. |
| 038eebd | 546 | |
| 038eebd | 547 | - [ ] **Step 7: Commit** |
| 038eebd | 548 | |
| 038eebd | 549 | ```bash |
| 038eebd | 550 | git add plum-checker/src/lib.rs plum-checker/tests/checker_tests.rs |
| 038eebd | 551 | git commit -m "feat(plum-checker): infer closure literal types; type-check fn(...)-typed params" |
| 038eebd | 552 | ``` |
| 038eebd | 553 | |
| 038eebd | 554 | --- |
| 038eebd | 555 | |
| 038eebd | 556 | ### Task 4: Codegen infrastructure — wasm function table + element section support |
| 038eebd | 557 | |
| 038eebd | 558 | **Files:** |
| 038eebd | 559 | - Modify: `plum-wasm-codegen/src/lib.rs` |
| 038eebd | 560 | - Test: `plum-wasm-codegen/tests/codegen_tests.rs` |
| 038eebd | 561 | |
| 038eebd | 562 | **Interfaces:** |
| 038eebd | 563 | - Consumes: nothing from other tasks — this is pure `WasmModule` infrastructure, independently testable without any closure-compiling logic existing yet. |
| 038eebd | 564 | - Produces: `WasmModule::add_table_element(&mut self, func_idx: u32) -> u32`, appending `func_idx` to a single funcref table and returning its table index. `WasmModule::finish()` now emits a `Table` section (only if any elements were added) between the existing Function and Memory sections, and an `Element` section between the existing Export and Code sections — both are the correct binary positions per the wasm module section order (Type, Import, Function, **Table**, Memory, Global, Export, **Element**, Code, Data). |
| 038eebd | 565 | |
| 038eebd | 566 | - [ ] **Step 1: Write a failing test proving a table + element section round-trips through a real module** |
| 038eebd | 567 | |
| 038eebd | 568 | Append to `plum-wasm-codegen/tests/codegen_tests.rs`: |
| 038eebd | 569 | |
| 038eebd | 570 | ```rust |
| 038eebd | 571 | #[test] |
| 038eebd | 572 | fn wasm_module_with_a_table_element_validates_and_call_indirect_works() { |
| 038eebd | 573 | // Exercises WasmModule's new table/element support directly, independent of any |
| 038eebd | 574 | // closure-compiling logic (which doesn't exist yet) — builds a tiny module by |
| 038eebd | 575 | // hand: one function that returns 42, registered as table element 0, called via |
| 038eebd | 576 | // `call_indirect` from `main` using a runtime-computed (not compile-time-constant) |
| 038eebd | 577 | // table index, to prove the table/element wiring is real, not coincidentally |
| 038eebd | 578 | // skipped by validation. |
| 038eebd | 579 | let mut module = plum_wasm_codegen::WasmModule::new(); |
| 038eebd | 580 | let ret42_type = module.add_type(&[], &[wasm_encoder::ValType::I64]); |
| 038eebd | 581 | let ret42_idx = module.add_function(ret42_type, &{ |
| 038eebd | 582 | let mut body = Vec::new(); |
| 038eebd | 583 | wasm_encoder::Instruction::I64Const(42).encode(&mut body); |
| 038eebd | 584 | wasm_encoder::Instruction::End.encode(&mut body); |
| 038eebd | 585 | body |
| 038eebd | 586 | }); |
| 038eebd | 587 | let table_idx = module.add_table_element(ret42_idx); |
| 038eebd | 588 | assert_eq!(table_idx, 0); |
| 038eebd | 589 | |
| 038eebd | 590 | let main_type = module.add_type(&[], &[wasm_encoder::ValType::I64]); |
| 038eebd | 591 | let main_idx = module.add_function(main_type, &{ |
| 038eebd | 592 | let mut body = Vec::new(); |
| 038eebd | 593 | wasm_encoder::Instruction::I32Const(0).encode(&mut body); // table index operand |
| 038eebd | 594 | wasm_encoder::Instruction::CallIndirect { type_index: ret42_type, table_index: 0 }.encode(&mut body); |
| 038eebd | 595 | wasm_encoder::Instruction::End.encode(&mut body); |
| 038eebd | 596 | body |
| 038eebd | 597 | }); |
| 038eebd | 598 | module.add_export("main", wasm_encoder::ExportKind::Func, main_idx); |
| 038eebd | 599 | |
| 038eebd | 600 | let bytes = module.finish(); |
| 038eebd | 601 | let result = wasmparser::validate(&bytes); |
| 038eebd | 602 | assert!(result.is_ok(), "wasm validation failed: {:?}", result.err()); |
| 038eebd | 603 | |
| 038eebd | 604 | let engine = wasmtime::Engine::default(); |
| 038eebd | 605 | let wasm_module = wasmtime::Module::new(&engine, &bytes).expect("module should be loadable"); |
| 038eebd | 606 | let mut store = wasmtime::Store::new(&engine, ()); |
| 038eebd | 607 | let instance = wasmtime::Instance::new(&mut store, &wasm_module, &[]).expect("module should instantiate"); |
| 038eebd | 608 | let main = instance.get_typed_func::<(), i64>(&mut store, "main").expect("main should have signature () -> i64"); |
| 038eebd | 609 | assert_eq!(main.call(&mut store, ()).expect("main should not trap"), 42); |
| 038eebd | 610 | } |
| 038eebd | 611 | ``` |
| 038eebd | 612 | |
| 038eebd | 613 | - [ ] **Step 2: Run to see it fail** |
| 038eebd | 614 | |
| 038eebd | 615 | Run: `cargo test -p plum-wasm-codegen --test codegen_tests wasm_module_with_a_table` |
| 038eebd | 616 | Expected: fails to compile — `add_table_element` doesn't exist yet. |
| 038eebd | 617 | |
| 038eebd | 618 | - [ ] **Step 3: Add table/element support to `WasmModule`** |
| 038eebd | 619 | |
| 038eebd | 620 | In `plum-wasm-codegen/src/lib.rs`, find the `WasmModule` struct: |
| 038eebd | 621 | |
| 038eebd | 622 | ```rust |
| 038eebd | 623 | pub struct WasmModule { |
| 038eebd | 624 | types: Vec<FuncType>, |
| 038eebd | 625 | imports: Vec<(String, String, u32)>, |
| 038eebd | 626 | functions: Vec<(u32, Vec<u8>)>, |
| 038eebd | 627 | exports: Vec<(String, ExportKind, u32)>, |
| 038eebd | 628 | memories: Vec<MemoryType>, |
| 038eebd | 629 | globals: Vec<(ValType, bool, Vec<u8>)>, |
| 038eebd | 630 | data_segments: Vec<(u32, Vec<u8>)>, |
| 038eebd | 631 | pub func_import_count: u32, |
| 038eebd | 632 | pub func_count: u32, |
| 038eebd | 633 | global_count: u32, |
| 038eebd | 634 | } |
| 038eebd | 635 | ``` |
| 038eebd | 636 | |
| 038eebd | 637 | Add a new field: |
| 038eebd | 638 | |
| 038eebd | 639 | ```rust |
| 038eebd | 640 | pub struct WasmModule { |
| 038eebd | 641 | types: Vec<FuncType>, |
| 038eebd | 642 | imports: Vec<(String, String, u32)>, |
| 038eebd | 643 | functions: Vec<(u32, Vec<u8>)>, |
| 038eebd | 644 | exports: Vec<(String, ExportKind, u32)>, |
| 038eebd | 645 | memories: Vec<MemoryType>, |
| 038eebd | 646 | globals: Vec<(ValType, bool, Vec<u8>)>, |
| 038eebd | 647 | data_segments: Vec<(u32, Vec<u8>)>, |
| 038eebd | 648 | /// Function indices, in table order — the single funcref table used for |
| 038eebd | 649 | /// closure `call_indirect` dispatch. Index into this vec IS the table index. |
| 038eebd | 650 | table_elements: Vec<u32>, |
| 038eebd | 651 | pub func_import_count: u32, |
| 038eebd | 652 | pub func_count: u32, |
| 038eebd | 653 | global_count: u32, |
| 038eebd | 654 | } |
| 038eebd | 655 | ``` |
| 038eebd | 656 | |
| 038eebd | 657 | Update `WasmModule::new()`'s struct literal to add `table_elements: Vec::new(),`. |
| 038eebd | 658 | |
| 038eebd | 659 | Add a new method, near `add_memory`: |
| 038eebd | 660 | |
| 038eebd | 661 | ```rust |
| 038eebd | 662 | /// Registers `func_idx` as the next slot in the single funcref table used for |
| 038eebd | 663 | /// closure `call_indirect` dispatch, returning its table index. |
| 038eebd | 664 | pub fn add_table_element(&mut self, func_idx: u32) -> u32 { |
| 038eebd | 665 | let table_idx = self.table_elements.len() as u32; |
| 038eebd | 666 | self.table_elements.push(func_idx); |
| 038eebd | 667 | table_idx |
| 038eebd | 668 | } |
| 038eebd | 669 | ``` |
| 038eebd | 670 | |
| 038eebd | 671 | In `finish()`, find the Function section block, ending with `module.section(&funcs); }`, and insert a Table section right after it (before the Memory section block): |
| 038eebd | 672 | |
| 038eebd | 673 | ```rust |
| 038eebd | 674 | // Table section |
| 038eebd | 675 | if !self.table_elements.is_empty() { |
| 038eebd | 676 | let mut tables = TableSection::new(); |
| 038eebd | 677 | tables.table(TableType { |
| 038eebd | 678 | element_type: RefType::FUNCREF, |
| 038eebd | 679 | minimum: self.table_elements.len() as u64, |
| 038eebd | 680 | maximum: Some(self.table_elements.len() as u64), |
| 038eebd | 681 | table64: false, |
| 038eebd | 682 | shared: false, |
| 038eebd | 683 | }); |
| 038eebd | 684 | module.section(&tables); |
| 038eebd | 685 | } |
| 038eebd | 686 | ``` |
| 038eebd | 687 | |
| 038eebd | 688 | Find the Export section block, ending with `module.section(&exports); }`, and insert an Element section right after it (before the Code section block): |
| 038eebd | 689 | |
| 038eebd | 690 | ```rust |
| 038eebd | 691 | // Element section |
| 038eebd | 692 | if !self.table_elements.is_empty() { |
| 038eebd | 693 | let mut elements = ElementSection::new(); |
| 038eebd | 694 | let offset = ConstExpr::i32_const(0); |
| 038eebd | 695 | elements.active(Some(0), &offset, Elements::Functions(std::borrow::Cow::Borrowed(&self.table_elements))); |
| 038eebd | 696 | module.section(&elements); |
| 038eebd | 697 | } |
| 038eebd | 698 | ``` |
| 038eebd | 699 | |
| 038eebd | 700 | - [ ] **Step 4: Run the new test** |
| 038eebd | 701 | |
| 038eebd | 702 | Run: `cargo test -p plum-wasm-codegen --test codegen_tests wasm_module_with_a_table` |
| 038eebd | 703 | Expected: passes. |
| 038eebd | 704 | |
| 038eebd | 705 | - [ ] **Step 5: Run the full workspace suite** |
| 038eebd | 706 | |
| 038eebd | 707 | Run: `cargo test --workspace` |
| 038eebd | 708 | Expected: green — confirm no existing test that builds a `WasmModule` and checks its exact byte output (if any) is affected by the new always-present-but-conditionally-emitted table field (it should be a pure no-op when `table_elements` is empty, since both new sections are gated on `!self.table_elements.is_empty()`). |
| 038eebd | 709 | |
| 038eebd | 710 | - [ ] **Step 6: Commit** |
| 038eebd | 711 | |
| 038eebd | 712 | ```bash |
| 038eebd | 713 | git add plum-wasm-codegen/src/lib.rs plum-wasm-codegen/tests/codegen_tests.rs |
| 038eebd | 714 | git commit -m "feat(plum-wasm-codegen): add wasm function table + element section support" |
| 038eebd | 715 | ``` |
| 038eebd | 716 | |
| 038eebd | 717 | --- |
| 038eebd | 718 | |
| 038eebd | 719 | ### Task 5: Codegen — compile closure literals and closure calls |
| 038eebd | 720 | |
| 038eebd | 721 | **Files:** |
| 038eebd | 722 | - Modify: `plum-wasm-codegen/src/lib.rs` |
| 038eebd | 723 | - Test: `plum-wasm-codegen/tests/codegen_tests.rs` |
| 038eebd | 724 | |
| 038eebd | 725 | **This is the largest, most judgment-requiring task in this plan** — genuinely new algorithmic code (free-variable analysis, a program-wide discovery pre-pass, synthetic function generation), not a modification of existing tested logic. Treat the design below as your starting architecture, not verified-correct code to transcribe — validate every piece via the tests as you go, and stop and report BLOCKED/NEEDS_CONTEXT if a specific piece doesn't hold up under testing, per this plan's Global Constraints. |
| 038eebd | 726 | |
| 038eebd | 727 | **Interfaces:** |
| 038eebd | 728 | - Consumes: `ast::Expr::Closure`, `ast::ParamType::Fn` (Task 2); `WasmModule::add_table_element` (Task 4); `PlumType::TFun` (already exists; Task 3 makes the checker produce it for closures). |
| 038eebd | 729 | - Produces: `compile_source` compiles every closure literal in the program to its own wasm function and table entry; `compile_expr` handles `Expr::Closure` (constructing the runtime closure value) and calling a closure-typed binding (`cb(x)` where `cb`'s inferred type is `TFun`, via `call_indirect` instead of a direct `Call`). |
| 038eebd | 730 | |
| 038eebd | 731 | **Design:** |
| 038eebd | 732 | |
| 038eebd | 733 | 1. **Runtime representation.** A closure value is a single `i32` pointer to a heap-allocated pair `{table_index: i32, env_pointer: i32}` (8-byte stride per field, matching every other class-like struct already in this codegen — `table_index` at offset 0, `env_pointer` at offset 8). The captured-environment struct itself is a separate heap allocation: one 8-byte slot per captured variable, in a stable (e.g. alphabetical, or first-appearance) order. |
| 038eebd | 734 | |
| 038eebd | 735 | 2. **Discovery pre-pass.** Before compiling any function body, walk every `ast::Item::Fn`'s body (a new walker, since `Collector` runs per-function *after* registration and doesn't cross function boundaries) to find every `ast::Expr::Closure` node. For each one found as an argument to a call whose corresponding declared param type is `ast::ParamType::Fn(param_types, ret)` (already-monomorphized concrete types, since this pre-pass runs on the output of `monomorphize_source`), record: |
| 038eebd | 736 | - a synthetic mangled name (e.g. `format!("closure${}", n)` with a simple incrementing counter); |
| 038eebd | 737 | - the concrete wasm param/return `ValType`s (from `param_types`/`ret` via the existing `ast_type_to_wasm`); |
| 038eebd | 738 | - its free variables — every `Var(name)` referenced in the closure's body that is NOT one of the closure's own params — resolved against the *enclosing* function's locals (both real params/assigned locals AND, if the closure is itself nested inside another closure, that outer closure's own free variables/params) — with each free variable's `PlumType` (via `infer_local_type` against the enclosing scope's type env, exactly as `Collector`/`compile_expr` already do elsewhere in this file). |
| 038eebd | 739 | |
| 038eebd | 740 | Store the result in a new `HashMap<usize, ClosureInfo>` (keyed by the closure `Expr`'s pointer identity, the same keying convention `classcall_scratch`/`match_scratch_index` already use), where: |
| 038eebd | 741 | ```rust |
| 038eebd | 742 | struct ClosureInfo { |
| 038eebd | 743 | mangled_name: String, |
| 038eebd | 744 | func_idx: u32, |
| 038eebd | 745 | table_idx: u32, |
| 038eebd | 746 | param_vts: Vec<ValType>, |
| 038eebd | 747 | ret_vt: Option<ValType>, |
| 038eebd | 748 | free_vars: Vec<(String, PlumType)>, // stable order |
| 038eebd | 749 | } |
| 038eebd | 750 | ``` |
| 038eebd | 751 | Add this map to `CompileCtx` as `pub closures: HashMap<usize, ClosureInfo>`. |
| 038eebd | 752 | |
| 038eebd | 753 | 3. **Registering each closure as a real function.** For each discovered closure, `module.add_type(...)` for `(env_ptr: I32, ...param_vts) -> ret_vt`, then reserve a function slot via `module.add_function(type_idx, &[])` (placeholder body, patched later — exactly how `compile_source` already pre-registers every ordinary `Fn`'s slot before compiling bodies) to get `func_idx`, then `module.add_table_element(func_idx)` to get `table_idx`. |
| 038eebd | 754 | |
| 038eebd | 755 | 4. **Compiling each closure's own body.** Build a `LocalCtx` for the closure much like `compile_fn_body` already does for an ordinary `Fn`, except: local 0 is the (unused-by-name) env pointer param; each free variable gets its own local slot, loaded from the env pointer at function entry (`LocalGet(env_ptr_local); {I64,F64,I32}Load(offset); LocalSet(free_var_local)`, offsets assigned in the same stable order used when constructing the env struct at the call site); each real closure param gets ordinary param-local treatment, offset by 1 (to account for the implicit env-pointer param at index 0) plus however many free-var locals precede it in your chosen local-numbering scheme. Compile the closure's `Block` body via the existing `compile_block_as_fn_body`/value-position machinery, exactly as an ordinary function's block body already is. |
| 038eebd | 756 | |
| 038eebd | 757 | 5. **Compiling a closure literal's construction site** (new `Expr::Closure` arm in `compile_expr`): reserve two scratch locals (extend the existing `classcall_scratch`-style pool, or add a parallel `closure_scratch: HashMap<usize, u32>` pool reserving 2 consecutive slots per closure literal — mirroring how `Collector`/`LocalCtx` already reserve scratch slots for `ClassCall`). Bump-allocate the env struct (size `8 * free_vars.len()`), store each free variable's *current* value (loaded via the existing `Expr::Var` local-lookup path in the *enclosing* function) into it; bump-allocate the 2-word closure struct, store `table_idx` (a compile-time `i32.const`) at offset 0 and the env struct's pointer at offset 8; leave the closure struct's pointer as the result. |
| 038eebd | 758 | |
| 038eebd | 759 | 6. **Compiling a closure call.** In `compile_expr`'s existing `ast::Expr::FnCall(call) => { ... }` arm, before the existing `ctx.func_ids.get(&call.name)` lookup: check whether `call.name` is a *local* (`ctx.locals.contains_key(&call.name)`) whose inferred type (`infer_local_type` on `ast::Expr::Var(call.name.clone())`) is `PlumType::TFun(..)`. If so, compile as a closure call instead of a direct `Call`: `LocalGet` the closure pointer, `I32Load` its `env_pointer` (offset 8) into a scratch, push that env pointer, then compile+push each real argument, then `LocalGet` the closure pointer again and `I32Load` its `table_index` (offset 0), then `Instruction::CallIndirect { type_index, table_index: 0 }` — `type_index` resolved from the call site's own already-known concrete arg/return types (build/reuse a `module.add_type` call keyed by that signature, or thread the specific `ClosureInfo`/type index through if the call site's closure binding can be traced back to a specific `ClosureInfo` — use your judgment on the cleanest way to get a consistent type index here, since multiple closures of the same signature can share one). |
| 038eebd | 760 | |
| 038eebd | 761 | - [ ] **Step 1: Write failing tests** |
| 038eebd | 762 | |
| 038eebd | 763 | Append to `plum-wasm-codegen/tests/codegen_tests.rs`: |
| 038eebd | 764 | |
| 038eebd | 765 | ```rust |
| 038eebd | 766 | #[test] |
| 038eebd | 767 | fn non_capturing_closure_passed_and_called_runs_correctly() { |
| 038eebd | 768 | let src = "\ |
| 038eebd | 769 | each(cb: fn(Int) -> Int) -> Int = |
| 038eebd | 770 | cb(5) |
| 038eebd | 771 | |
| 038eebd | 772 | main() -> Int = |
| 038eebd | 773 | each(|v| |
| 038eebd | 774 | v) |
| 038eebd | 775 | "; |
| 038eebd | 776 | let source = parse(src); |
| 038eebd | 777 | let bytes = compile_source(&source).expect("compile failed"); |
| 038eebd | 778 | assert_eq!(run_main(&bytes), 5); |
| 038eebd | 779 | } |
| 038eebd | 780 | |
| 038eebd | 781 | #[test] |
| 038eebd | 782 | fn capturing_closure_snapshots_value_at_creation_time_runs_correctly() { |
| 038eebd | 783 | let src = "\ |
| 038eebd | 784 | each(cb: fn(Int) -> Int) -> Int = |
| 038eebd | 785 | cb(0) |
| 038eebd | 786 | |
| 038eebd | 787 | useClosure() -> Int = |
| 038eebd | 788 | x = 10 |
| 038eebd | 789 | cb = |v| |
| 038eebd | 790 | x + v |
| 038eebd | 791 | x = 999 |
| 038eebd | 792 | each(cb) |
| 038eebd | 793 | |
| 038eebd | 794 | main() -> Int = |
| 038eebd | 795 | useClosure() |
| 038eebd | 796 | "; |
| 038eebd | 797 | let source = parse(src); |
| 038eebd | 798 | let bytes = compile_source(&source).expect("compile failed"); |
| 038eebd | 799 | // The closure must see x==10 (its value when the closure was created), not 999 |
| 038eebd | 800 | // (its value when `each(cb)` is actually called) — proving snapshot-by-value |
| 038eebd | 801 | // capture, not a live/shared reference. |
| 038eebd | 802 | assert_eq!(run_main(&bytes), 10); |
| 038eebd | 803 | } |
| 038eebd | 804 | |
| 038eebd | 805 | #[test] |
| 038eebd | 806 | fn closure_passed_through_already_generic_higher_order_function_runs_correctly() { |
| 038eebd | 807 | let src = "\ |
| 038eebd | 808 | identity(value: a) -> a = |
| 038eebd | 809 | value |
| 038eebd | 810 | |
| 038eebd | 811 | each(cb: fn(Int) -> Int) -> Int = |
| 038eebd | 812 | cb(identity(7)) |
| 038eebd | 813 | |
| 038eebd | 814 | main() -> Int = |
| 038eebd | 815 | each(|v| |
| 038eebd | 816 | v * 2) |
| 038eebd | 817 | "; |
| 038eebd | 818 | let source = parse(src); |
| 038eebd | 819 | let bytes = compile_source(&source).expect("compile failed"); |
| 038eebd | 820 | assert_eq!(run_main(&bytes), 14); |
| 038eebd | 821 | } |
| 038eebd | 822 | ``` |
| 038eebd | 823 | |
| 038eebd | 824 | - [ ] **Step 2: Run to see them fail** |
| 038eebd | 825 | |
| 038eebd | 826 | Run: `cargo test -p plum-wasm-codegen --test codegen_tests capturing_closure non_capturing_closure closure_passed_through` |
| 038eebd | 827 | Expected: all fail — none of the compiling logic exists yet. |
| 038eebd | 828 | |
| 038eebd | 829 | - [ ] **Step 3: Implement the design above** |
| 038eebd | 830 | |
| 038eebd | 831 | Follow the 6-part design. Iterate test-by-test — get `non_capturing_closure_passed_and_called_runs_correctly` passing first (no free-variable analysis needed for that one, simplifying the first pass), then tackle capture, then the generics-interop test. |
| 038eebd | 832 | |
| 038eebd | 833 | - [ ] **Step 4: Run all three new tests, then the full codegen suite** |
| 038eebd | 834 | |
| 038eebd | 835 | ```bash |
| 038eebd | 836 | cargo test -p plum-wasm-codegen --test codegen_tests |
| 038eebd | 837 | ``` |
| 038eebd | 838 | |
| 038eebd | 839 | Expected: all pass, including every pre-existing test (confirming the new discovery pre-pass and `FnCall` arm change don't regress ordinary function calls). |
| 038eebd | 840 | |
| 038eebd | 841 | - [ ] **Step 5: Run the full workspace and tree-sitter suites** |
| 038eebd | 842 | |
| 038eebd | 843 | ```bash |
| 038eebd | 844 | cargo test --workspace |
| 038eebd | 845 | cd tooling/tree-sitter-plum && npx --yes tree-sitter-cli test |
| 038eebd | 846 | ``` |
| 038eebd | 847 | |
| 038eebd | 848 | Expected: fully green. |
| 038eebd | 849 | |
| 038eebd | 850 | - [ ] **Step 6: Commit** |
| 038eebd | 851 | |
| 038eebd | 852 | ```bash |
| 038eebd | 853 | git add plum-wasm-codegen/src/lib.rs plum-wasm-codegen/tests/codegen_tests.rs |
| 038eebd | 854 | git commit -m "feat(plum-wasm-codegen): compile closure literals and closure calls via a function table" |
| 038eebd | 855 | ``` |
| 038eebd | 856 | |
| 038eebd | 857 | --- |
| 038eebd | 858 | |
| 038eebd | 859 | ### Task 6: Examples and docs |
| 038eebd | 860 | |
| 038eebd | 861 | **Files:** |
| 038eebd | 862 | - Modify or create: `examples/closures.plum` (new file, following this repo's existing `examples/*.plum` convention — one file per feature area, exercised by both crates' `examples_test.rs`) |
| 038eebd | 863 | - Modify: `plum-checker/tests/examples_test.rs`, `plum-wasm-codegen/tests/examples_test.rs` |
| 038eebd | 864 | - Modify: `README.md` |
| 038eebd | 865 | |
| 038eebd | 866 | **Interfaces:** |
| 038eebd | 867 | - Consumes: everything from Tasks 1-5. |
| 038eebd | 868 | - Produces: nothing further downstream — final integration/documentation task. |
| 038eebd | 869 | |
| 038eebd | 870 | - [ ] **Step 1: Add `examples/closures.plum`** |
| 038eebd | 871 | |
| 038eebd | 872 | ```plum |
| 038eebd | 873 | each(cb: fn(Int) -> Int) -> Int = |
| 038eebd | 874 | cb(5) |
| 038eebd | 875 | |
| 038eebd | 876 | double(v: Int) -> Int = |
| 038eebd | 877 | v * 2 |
| 038eebd | 878 | |
| 038eebd | 879 | useNamedFunctionAsValue() -> Int = |
| 038eebd | 880 | each(double) |
| 038eebd | 881 | |
| 038eebd | 882 | useCapturingClosure() -> Int = |
| 038eebd | 883 | offset = 100 |
| 038eebd | 884 | each(|v| |
| 038eebd | 885 | v + offset) |
| 038eebd | 886 | |
| 038eebd | 887 | main() -> Int = |
| 038eebd | 888 | each(|v| |
| 038eebd | 889 | v * 3) |
| 038eebd | 890 | ``` |
| 038eebd | 891 | |
| 038eebd | 892 | (If `each(double)` — passing a top-level *named function* wherever a `fn(...)`-typed param is expected, not just a closure literal — isn't yet supported by Task 5's design (it may need a small addition: wrapping a plain function reference in the same `{table_index, env_pointer}` shape with an empty/null env), either extend Task 5 minimally to support it, or drop `useNamedFunctionAsValue`/adjust this example to closures only and note the named-function-as-value gap in README's Known Gaps instead. Verify which via a quick test before deciding.) |
| 038eebd | 893 | |
| 038eebd | 894 | - [ ] **Step 2: Extend both crates' `examples_test.rs`** |
| 038eebd | 895 | |
| 038eebd | 896 | In `plum-checker/tests/examples_test.rs`, confirm `examples/closures.plum` is picked up automatically (it likely already iterates every `.plum` file in the directory — check `example_files()`'s implementation; if so, no change needed beyond adding the file). |
| 038eebd | 897 | |
| 038eebd | 898 | In `plum-wasm-codegen/tests/examples_test.rs`, add: |
| 038eebd | 899 | |
| 038eebd | 900 | ```rust |
| 038eebd | 901 | #[test] |
| 038eebd | 902 | fn closures_example_compiles_and_runs_correctly() { |
| 038eebd | 903 | let bytes = assert_compiles("closures.plum"); |
| 038eebd | 904 | let engine = wasmtime::Engine::default(); |
| 038eebd | 905 | let module = wasmtime::Module::new(&engine, &bytes).expect("module should be loadable"); |
| 038eebd | 906 | let mut store = wasmtime::Store::new(&engine, ()); |
| 038eebd | 907 | let instance = wasmtime::Instance::new(&mut store, &module, &[]).expect("module should instantiate"); |
| 038eebd | 908 | let main = instance.get_typed_func::<(), i64>(&mut store, "main").expect("main should have signature () -> i64"); |
| 038eebd | 909 | assert_eq!(main.call(&mut store, ()).expect("main should not trap"), 15); |
| 038eebd | 910 | } |
| 038eebd | 911 | ``` |
| 038eebd | 912 | |
| 038eebd | 913 | (Adjust the expected result if Step 1's `main` body ends up different from what's shown above.) |
| 038eebd | 914 | |
| 038eebd | 915 | - [ ] **Step 3: Run both examples suites** |
| 038eebd | 916 | |
| 038eebd | 917 | ```bash |
| 038eebd | 918 | cargo test -p plum-checker --test examples_test |
| 038eebd | 919 | cargo test -p plum-wasm-codegen --test examples_test |
| 038eebd | 920 | ``` |
| 038eebd | 921 | |
| 038eebd | 922 | Expected: both green. |
| 038eebd | 923 | |
| 038eebd | 924 | - [ ] **Step 4: Update README** |
| 038eebd | 925 | |
| 038eebd | 926 | Add a new "Closures" section (find a sensible place — likely after the existing "Generics" section, before "`self`, field access, and methods"), documenting: the `|params| body` syntax, the `fn(...)` / `fn(...) -> T` type annotation syntax (positional types only), snapshot-by-value capture semantics, and a link to `examples/closures.plum`. Update the "Known gaps" list: remove the `closure` bullet (`` `closure` (`|params| body`) exists in `grammar.js` but isn't wired into any reachable rule yet, so it doesn't actually parse in context. ``) if it's still present verbatim — check the file's current end for this exact sentence — and add any residual gap actually discovered during Task 5 (e.g. named-function-as-closure-value, if that turned out to be unsupported per Task 6 Step 1's note). |
| 038eebd | 927 | |
| 038eebd | 928 | - [ ] **Step 5: Run the full workspace and tree-sitter suites one final time** |
| 038eebd | 929 | |
| 038eebd | 930 | ```bash |
| 038eebd | 931 | cargo test --workspace |
| 038eebd | 932 | cd tooling/tree-sitter-plum && npx --yes tree-sitter-cli test |
| 038eebd | 933 | ``` |
| 038eebd | 934 | |
| 038eebd | 935 | Expected: fully green, zero known failures. |
| 038eebd | 936 | |
| 038eebd | 937 | - [ ] **Step 6: Commit** |
| 038eebd | 938 | |
| 038eebd | 939 | ```bash |
| 038eebd | 940 | git add examples/closures.plum plum-checker/tests/examples_test.rs plum-wasm-codegen/tests/examples_test.rs README.md |
| 038eebd | 941 | git commit -m "docs+test: closures complete; add example and update README" |
| 038eebd | 942 | ``` |