42452f8 | plum | pyrossh

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

42452f8 — Peter John 2026-11-04T15:48:01+05:30

docs: compiler cleanup phase 1 plan; explicit function generics queued after phase 2

Files changed (2) hide show

docs/backlog.md +2 -1
docs/plans/2026-11-04-compiler-cleanup-phase1.md +251 -0

docs/backlog.md CHANGED Viewed

@@ -4,7 +4,8 @@ Open work, in the order the user queued it, plus smaller items found during buil
 ## Planned (in order)
-1. **Compiler cleanup and architecture** — `docs/plans/2026-11-04-compiler-cleanup.review.md` (written in another session; meant to run after language-restrictions merged). Needs an approve.
+1. **Compiler cleanup and architecture** — `docs/plans/2026-11-04-compiler-cleanup.review.md` (written in another session; meant to run after language-restrictions merged). Approved 2026-11-04; running on branch `superlite/compiler-cleanup`, phase 1 first.
+   - **Explicit function generics** (user 2026-11-04: right after cleanup phase 2) — functions declare their generics (`fun map[B](...)`) instead of every single capital letter being a generic; allows longer generic names and one-letter concrete types/tags. Grammar change + about 140 signatures in std/examples. Phase 2's "generics by scope" makes this mostly a grammar + signature rewrite.
 2. **Native inline variants, then decide union enums** — compile inline variants directly as struct subtypes of their enum and delete the hidden-record lowering (`hoistInlineVariants`, the `List__Cons(Cons(..))` wrappers, union forwarders). Then decide whether user-facing union enums (`enum X = A | B` of records) stay as a thin feature or fold into traits; if they go, `JsonParse` in `plum-std/Json.plum` becomes inline variants and the union examples (`unions.plum`, `struct_union_members.plum`) are rewritten. `unboxed_variants.plum` is really a compiler probe: move it to a codegen test. (Decision record §8b.)
 3. **Benchmark** before FBIP (user: FBIP waits for it).
 4. **FBIP: no heap mutation** — persistent std collections updated in place when unique, Koka-style `fip` functions as a checked in-place guarantee, a reference count used only as a uniqueness check if static proof falls short. Worklist = the `mut` fields left today (see below). (Decision record §9.)

docs/plans/2026-11-04-compiler-cleanup-phase1.md ADDED Viewed

@@ -0,0 +1,251 @@
+# Compiler cleanup, phase 1: deletions and test helpers
+**Review plan:** docs/plans/2026-11-04-compiler-cleanup.review.md
+**Goal:** Delete dead code and small duplicates in the compiler; one shared test-support crate; fewer, de-duplicated test binaries. No behaviour change: every example compiles to byte-identical wasm.
+**Architecture:** Rust workspace (plum-core parser+passes, plum-checker, plum-wasm-codegen, plum-cli, plum-lsp). New workspace member `plum-test-support` (dev-dependency only).
+**Tech stack:** Rust 2021, cargo, wasmtime (GC + function references + tail calls), tree-sitter-plum.
+**Detail:** contracts
+**Target-min:** 14
+## Global constraints
+- Language: Rust. Test command per crate: `cargo test -p <crate>` (do NOT trust `run_tests --fast` path filters in this repo; run the crate's tests directly). Full: `cargo test --workspace`.
+- `touch` an edited .rs file before trusting a cargo result (stale-build gotcha in this sandbox).
+- Functions are camelCase (`#![allow(non_snake_case)]` at crate/test top), matching the repo.
+- No behaviour change. No test may be deleted unless this plan names it.
+- `plum-game` and `plum-runtime` are not workspace members: out of scope this phase.
+- Never run `plum format` on real files (it rewrites in place).
+## Main diagram
+```mermaid
+flowchart LR
+  TS["plum-test-support"] --> CT["checker tests"]
+  TS --> GT["codegen tests"]
+  TS --> E2E["e2e binaries"]
+  TS --> CO["core + cli tests"]
+  DC["dead code: core / checker / codegen"] --> V["verify: suite + wasm hashes"]
+  WH["wasm-hashes script"] --> V
+  CT --> V
+  GT --> V
+  E2E --> V
+  CO --> V
+```
+## File structure
+- `scripts/wasm-hashes.sh` — compile every example with `main` at a git rev, print sha256 per file.
+- `plum-test-support/{Cargo.toml,src/lib.rs}` — shared test helpers (parse, load, compile, engine, run, errors).
+- `plum-wasm-codegen/tests/{e2e_records,e2e_types,e2e_tail,e2e_mem}/` — 4 e2e binaries replacing 19 files.
+- `plum-checker/tests/features/` — one binary for the small checker test files.
+## Task graph
+```mermaid
+flowchart LR
+  T1[wasm hash script]
+  T2[core dead code]
+  T3[checker duplicates]
+  T4[codegen dead code]
+  T5[test-support crate]
+  T6[checker tests]
+  T7[codegen unit tests]
+  T8[e2e records binary]
+  T12[e2e types binary]
+  T9[e2e tail binary]
+  T13[e2e memory binary]
+  T10[core and cli tests]
+  T11[verify phase 1]
+  %% taskgraph
+  T5 --> T6
+  T5 --> T7
+  T5 --> T8
+  T5 --> T9
+  T5 --> T12
+  T5 --> T13
+  T5 --> T10
+  T1 --> T11
+  T2 --> T11
+  T3 --> T11
+  T4 --> T11
+  T6 --> T11
+  T7 --> T11
+  T8 --> T11
+  T9 --> T11
+  T12 --> T11
+  T13 --> T11
+  T10 --> T11
+```
+## Package A: compiler deletions
+**Probe entry:** run=cargo test --workspace
+### T1 wasm hash script · Complexity 3 · Est. context 25k
+**Owns:** scripts/wasm-hashes.sh
+**Components:** 1
+**LOC owned:** 60
+**Tests:** slow
+**Kind:** design
+**Model:** sonnet
+**Produces:** scripts/wasm-hashes.sh REV OUT -> writes OUT with lines `<sha256>  <relpath>` sorted by relpath
+**Test code:** `bash scripts/wasm-hashes.sh HEAD /tmp/a.txt && bash scripts/wasm-hashes.sh HEAD /tmp/b.txt && diff /tmp/a.txt /tmp/b.txt` exits 0 and a.txt has ≥ 20 lines.
+**Steps:** `git worktree add --detach $TMP REV`; `cargo build -p plum-cli` there (separate target dir under $TMP); for every `plum-examples/**/*.plum` containing `fun main`, run a fresh `plum compile <file> -o <tmp>.wasm` (one process per file, so global counters start at 0); sha256 each; print `sha  relpath`; files that fail to compile print `FAIL  relpath`; remove the worktree on exit (trap).
+### T2 plum-core dead code · Complexity 2 · Est. context 30k
+**Owns:** plum-core/src/drive.rs, plum-core/src/simplify.rs, plum-core/src/closure_spec.rs, plum-core/src/parser.rs, plum-core/src/uniqueness.rs
+**Components:** 1
+**LOC owned:** 40
+**Tests:** fast
+**Kind:** mechanical
+**Model:** haiku
+**Produces:** none
+**Test code:** `cargo test -p plum-core` passes; `cargo build -p plum-core` prints no dead_code warning.
+**Steps:** delete `struct Scoper` and its impl in drive.rs (≈line 2275; simplify.rs has its own used Scoper — keep that one); delete `pub fn specialise` in closure_spec.rs:22 (keep `specialiseWith`); remove `#![allow(dead_code)]` from drive.rs:11 and simplify.rs:7, then delete every item the warnings name (expected: simplify `Scoper.ctors` field, uniqueness `selfUnit`); inline `parser::forStmts` (parser.rs:903) at its call sites. Grep all crates before deleting anything `pub`.
+### T3 checker duplicates · Complexity 4 · Est. context 45k
+**Owns:** plum-checker/src/lib.rs
+**Components:** 2
+**LOC owned:** 90
+**Tests:** fast
+**Model:** sonnet
+**Produces:** PlumType::primitiveName(&self) -> Option<&'static str> · PlumType::fromPrimitiveName(name: &str) -> Option<PlumType>
+e.g. PlumType::TInt.primitiveName() -> Some("Int")
+e.g. PlumType::fromPrimitiveName("Bool") -> Some(PlumType::TBool)
+e.g. PlumType::fromPrimitiveName("Str") -> None
+**Review Focus:** `trmc_recheck_uses_recheck_keep`: the TRMC block in `typedSource` (≈1294-1309) calls `recheckKeep` and existing trmc/fusion tests still pass; `empty_union_plan_single_check`: `checkUnions` with an empty recorded plan returns the lowered copy without a second `check(source)`, and unions_tests still pass.
+**Test code:** unit tests in a `#[cfg(test)] mod primitive_names` inside lib.rs asserting the three e.g. lines plus round-trip for Int/Float/Bool/Byte/Unit; `cargo test -p plum-checker` passes.
+**Steps:** add the two fns as an inherent `impl PlumType` block in lib.rs (not types.rs); replace the 7 hand-written name↔type matches (≈lines 655, 681, 715, 1387, 1879, 2157, 4823) with them where the mapping is identical (leave a site alone if it differs, e.g. extra names); replace `closureHints::head` (≈1384) with `methodReceiverName`; TRMC block → `recheckKeep`; `checkUnions` empty-plan shortcut; fix the stale `OP_CALLS` doc comment (≈45).
+### T4 codegen dead code · Complexity 1 · Est. context 22k
+**Owns:** plum-wasm-codegen/src/lib.rs
+**Components:** 1
+**LOC owned:** 20
+**Tests:** fast
+**Kind:** mechanical
+**Model:** haiku
+**Produces:** none
+**Test code:** `cargo test -p plum-wasm-codegen --lib` and `cargo build --workspace` pass.
+**Steps:** delete `WasmModule::addDataSegment` (lib.rs:137), the `DataSegmentEntry::Active` variant and its encoding branch (≈317) if nothing else constructs it (grep); remove the stale comments "Task 1" (≈156) and "Before wasm-gc" (≈4487).
+## Package B: tests
+**Probe entry:** run=cargo test --workspace
+### T5 plum-test-support crate · Complexity 4 · Est. context 40k
+**Owns:** plum-test-support/Cargo.toml, plum-test-support/src/lib.rs, Cargo.toml, plum-checker/Cargo.toml, plum-wasm-codegen/Cargo.toml, plum-core/Cargo.toml, plum-cli/Cargo.toml
+**Components:** 1
+**LOC owned:** 140
+**Tests:** fast
+**Model:** sonnet
+**Produces:**
+- plum_test_support::repoRoot() -> PathBuf
+- plum_test_support::parse(src: &str) -> plum_core::ast::Source (panics with the syntax errors if the tree has_error)
+- plum_test_support::parseRaw(src: &str) -> plum_core::ast::Source (no assert)
+- plum_test_support::load(name: &str, src: &str) -> Result<plum_core::ast::Source, String>
+- plum_test_support::compile(name: &str, src: &str) -> Vec<u8> (load + compileSource + validate, panics on error)
+- plum_test_support::engine() -> wasmtime::Engine (gc, function_references, tail_call on)
+- plum_test_support::runInt(name: &str, src: &str) -> i64
+- plum_test_support::runStr(name: &str, src: &str) -> String
+- plum_test_support::checkErrors(name: &str, src: &str) -> Vec<String> (load + typedSource errors; load error returned as one string)
+- plum_test_support::PRE: &str = "import std/Str\nimport std/Int\nimport std/Float\nimport std/Slice\n\n"
+e.g. parse("fun main() -> Int = 1").fns.len() -> 1
+e.g. runInt("t1", &format!("{PRE}fun main() -> Int = 40 + 2")) -> 42
+e.g. checkErrors("t2", &format!("{PRE}fun main() -> Int = True")) -> vec with one message containing "Bool"
+**Test code:** `plum-test-support/src/lib.rs` `#[cfg(test)]` tests asserting each e.g. line, plus `parse("fun main( ->")` panics (`#[should_panic]`).
+**Notes:** copy behaviour from `plum-wasm-codegen/tests/record_init_e2e_test.rs` (load uses a temp dir `plum_ts_<name>_<pid>` + `loadAndMerge(&path, &repoRoot())`; run uses `define_unknown_imports_as_default_values`). `runStr` reads a returned Str the way `plum-wasm-codegen/tests/examples_test.rs` does. Add the crate to workspace members, and add `plum-test-support = { path = "../plum-test-support" }` under `[dev-dependencies]` of plum-checker, plum-wasm-codegen, plum-core and plum-cli (T5 alone edits these Cargo.toml files). If the exact checker error API differs, match what `plum-checker/tests/*` already call.
+### T6 checker tests · Complexity 4 · Est. context 55k
+**Owns:** plum-checker/tests/*.rs, plum-checker/tests/features/*, plum-checker/src/types.rs
+**Components:** 2
+**LOC owned:** 120
+**Tests:** fast
+**Model:** sonnet
+**Consumes:** plum_test_support::parse(src: &str) -> plum_core::ast::Source (T5) · plum_test_support::load(name: &str, src: &str) -> Result<plum_core::ast::Source, String> (T5) · plum_test_support::repoRoot() -> PathBuf (T5)
+**Produces:** none
+**Review Focus:** `sweep_recurses`: `typed_source_has_no_unresolved_infer_vars` (checker_tests.rs:315) walks plum-std and plum-examples recursively (incl. plum-std/game/* and plum-examples/games/**) and still passes — if a newly covered file fails, keep it listed in an explicit skip list with a comment and report it; `parse_asserts`: checker_tests' `parse()` now fails on syntax errors.
+**Test code:** `cargo test -p plum-checker` passes; test count unchanged except the deleted `freshVarsAreUnique` InferState test.
+**Steps:** replace local parse/load/repoRoot helpers; merge io_capability, primitive_generics, variant_pattern, types, pattern_casing, closure_types, optional_unpinned, mutability test files into `tests/features/main.rs` with one `mod` file each; delete `types::InferState` and its test.
+### T7 codegen unit tests · Complexity 3 · Est. context 50k
+**Owns:** plum-wasm-codegen/tests/codegen_tests.rs, plum-wasm-codegen/tests/examples_test.rs, plum-wasm-codegen/tests/fusion_alloc_test.rs
+**Components:** 1
+**LOC owned:** 80
+**Tests:** slow
+**Model:** sonnet
+**Consumes:** plum_test_support::parse(src: &str) -> plum_core::ast::Source (T5) · plum_test_support::engine() -> wasmtime::Engine (T5) · plum_test_support::load(name: &str, src: &str) -> Result<plum_core::ast::Source, String> (T5) · plum_test_support::repoRoot() -> PathBuf (T5)
+**Produces:** none
+**Review Focus:** `parse_asserts`: codegen_tests' `parse()` (line 6) fails on syntax errors; any of its 16 callers that relied on a partial parse is fixed in the test source, not by `parseRaw`, and reported.
+**Test code:** `cargo test -p plum-wasm-codegen --test codegen_tests --test examples_test --test fusion_alloc_test` passes, same test count.
+**Steps:** replace local helpers; examples_test's examples walk recursive if it isn't already.
+### T8 e2e records binary · Complexity 3 · Est. context 35k
+**Notes:** all e2e tasks (T8, T12, T9, T13): move files to `tests/<bin>/<feature>.rs` as `mod`s of `tests/<bin>/main.rs` with the shared helpers, delete the old files; temp names unique per test; `cargo test -p plum-wasm-codegen --test <bin>`.
+**Owns:** plum-wasm-codegen/tests/e2e_records/*, plum-wasm-codegen/tests/{empty_record_embed,enum_init,inline_variants,record_embed,record_init,unboxed_variants}_e2e_test.rs
+**Components:** 1
+**LOC owned:** 60
+**Tests:** slow
+**Model:** sonnet
+**Consumes:** plum_test_support::runInt(name: &str, src: &str) -> i64 (T5) · plum_test_support::PRE: &str (T5)
+**Produces:** none
+**Test code:** `--test e2e_records` passes, same count.
+### T12 e2e types binary · Complexity 3 · Est. context 35k
+**Owns:** plum-wasm-codegen/tests/e2e_types/*, plum-wasm-codegen/tests/{traits,unions,io_entry,closure_types}_e2e_test.rs, plum-wasm-codegen/tests/type_param_static_test.rs
+**Components:** 1
+**LOC owned:** 60
+**Tests:** slow
+**Model:** sonnet
+**Consumes:** plum_test_support::runInt(name: &str, src: &str) -> i64 (T5) · plum_test_support::compile(name: &str, src: &str) -> Vec<u8> (T5) · plum_test_support::PRE: &str (T5)
+**Produces:** none
+**Review Focus:** `closure_dupes`: delete closure_types `g2_never_called_is_error` and `g1_typed_param` only if their source equals checker `local_closure_never_called_error` / `typed_param_used`; else keep and report.
+**Test code:** `--test e2e_types` passes, count minus the deleted ones.
+### T9 e2e tail binary · Complexity 3 · Est. context 35k
+**Owns:** plum-wasm-codegen/tests/e2e_tail/*, plum-wasm-codegen/tests/{closure_tail,error_tail,loop_break,tail_call,tail_calls}_e2e_test.rs
+**Components:** 1
+**LOC owned:** 60
+**Tests:** slow
+**Model:** sonnet
+**Consumes:** plum_test_support::runInt(name: &str, src: &str) -> i64 (T5) · plum_test_support::compile(name: &str, src: &str) -> Vec<u8> (T5) · plum_test_support::PRE: &str (T5)
+**Produces:** none
+**Review Focus:** `tail_dedupe`: tail_call + tail_calls become one `tail` mod; delete `self_tail_call_deep`, `mutual_tail_call_deep`, `closure_tail_call`; keep `self_tail_million`, `mutual_tail_million`, `closure_tail_call_runs` and the wasm-shape tests.
+**Test code:** `--test e2e_tail` passes, count minus 3.
+### T13 e2e memory binary · Complexity 3 · Est. context 30k
+**Owns:** plum-wasm-codegen/tests/e2e_mem/*, plum-wasm-codegen/tests/{reuse,unique_inplace,unique_spread}_e2e_test.rs
+**Components:** 1
+**LOC owned:** 50
+**Tests:** slow
+**Model:** sonnet
+**Consumes:** plum_test_support::runInt(name: &str, src: &str) -> i64 (T5) · plum_test_support::compile(name: &str, src: &str) -> Vec<u8> (T5) · plum_test_support::PRE: &str (T5)
+**Produces:** none
+**Test code:** `--test e2e_mem` passes, same count.
+### T10 core and cli tests · Complexity 3 · Est. context 45k
+**Owns:** plum-core/tests/*.rs, plum-cli/tests/*.rs
+**Components:** 1
+**LOC owned:** 100
+**Tests:** fast
+**Model:** sonnet
+**Consumes:** plum_test_support::parse(src: &str) -> plum_core::ast::Source (T5) · plum_test_support::repoRoot() -> PathBuf (T5)
+**Produces:** none
+**Test code:** `cargo test -p plum-core` and `cargo test -p plum-cli` pass, same counts.
+**Notes:** plum-core dev-depending on plum-test-support is a dev-dependency cycle. If it causes `ast::Source` type mismatches (then remove nothing from Cargo.toml; just don't use it), keep plum-core's local parse helpers but make every one assert `!tree.root_node().has_error()`, and report it. plum-cli tests: use the support crate only where they hand-roll the same helpers.
+## Package C: verify
+**Probe entry:** run=bash scripts/wasm-hashes.sh
+### T11 verify phase 1 · Complexity 2 · Est. context 30k
+**Owns:** docs/plans/2026-11-04-compiler-cleanup-phase1.results.md
+**Components:** 1
+**LOC owned:** 40
+**Tests:** slow
+**Model:** sonnet
+**Consumes:** scripts/wasm-hashes.sh REV OUT -> writes OUT with lines `<sha256>  <relpath>` sorted by relpath (T1)
+**Produces:** none
+**Review Focus:** `wasm_identical`: hashes at the start sha (`.superlite/progress.md`) equal hashes of the current tree (`git stash create` gives a sha; commit nothing).
+**Test code:** `cargo test --workspace` green; hash diff empty.
+**Steps:** results file: hash diff, workspace test totals, Rust LOC, test-binary count, `#[test]` count, each before/after.
+## Review Focus
+- Package A (library/compiler): deep, state — risk: deleting something only used via another crate; the checkUnions shortcut changing which errors surface.
+- Package B (test harness): file, state — risks: temp-dir name collisions between merged tests (name must stay unique per test), test count silently dropping, dev-dep cycle.
+- Package C: file — wasm output must be identical.
+## Self-review
+Coverage: every phase-1 row of the review plan maps to T2–T13. Scope rulings are in `.superlite/progress.md`.