889886f | plum

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

889886f — Peter John 2026-07-19T17:15:37+05:30

docs: add CLI formatter implementation plan

Files changed (1) hide show

docs/superpowers/plans/2026-07-19-cli-formatter.md +625 -0

docs/superpowers/plans/2026-07-19-cli-formatter.md ADDED Viewed

@@ -0,0 +1,625 @@
+# CLI & Formatter Implementation Plan
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+**Goal:** Restructure the project as a Cargo workspace, extract a `plum-core` library, and add a `plum-cli` binary with a `format` subcommand backed by Topiary.
+**Architecture:** Current `src/` code splits into `plum-core` (library: ast, parser, formatter) and `plum-cli` (thin binary). `formatter.rs` wraps `topiary-core`, embedding `format.scm` at compile time via `include_str!`. The CLI uses `clap` for argument parsing and delegates all logic to `plum-core`.
+**Tech Stack:** Rust 2021 edition, Cargo workspaces, topiary-core 0.7.3, topiary-tree-sitter-facade 0.7.3, tree-sitter 0.26 (required by topiary), tree-sitter-plum 0.1.0, clap 4, anyhow 1.
+## Global Constraints
+- topiary-core version: `"0.7.3"` — exact, do not upgrade
+- topiary-tree-sitter-facade version: `"0.7.3"` — exact, matches topiary-core
+- tree-sitter version: `"0.26"` — topiary-core 0.7.3 resolves tree-sitter 0.26.11; use `"0.26"` (not `"0.24.5"`) in plum-core
+- tree-sitter-plum version: `"0.1.0"` — local crate at `tooling/tree-sitter-plum`; reference via `path`
+- clap version: `"4"` with `features = ["derive"]`
+- anyhow version: `"1"`
+- format.scm lives at `tooling/tree-sitter-plum/queries/plum/format.scm`
+- `plum format <file>` edits in place silently on success; `--check` exits 1 if file would change; `--stdin` reads stdin, writes stdout
+- `tolerate_parsing_errors = false` — refuse to format files with syntax errors
+---
+## File Map
+| Path | Action | Responsibility |
+|---|---|---|
+| `Cargo.toml` | Replace | Workspace root — lists members, no `[package]` |
+| `plum-core/Cargo.toml` | Create | Library dependencies (tree-sitter 0.26, topiary, etc.) |
+| `plum-core/src/lib.rs` | Create | `pub mod` declarations + re-exports |
+| `plum-core/src/ast.rs` | Move from `src/ast.rs` | Typed AST types (unchanged) |
+| `plum-core/src/parser.rs` | Move from `src/parser.rs` | CST → AST walking (unchanged) |
+| `plum-core/src/formatter.rs` | Create | Topiary wrapper — `format_source` |
+| `plum-core/tests/formatter_test.rs` | Create | Integration tests for `format_source` |
+| `plum-cli/Cargo.toml` | Create | Binary dependencies (plum-core, clap, anyhow) |
+| `plum-cli/src/main.rs` | Create | clap CLI, file I/O, calls `plum_core::format_source` |
+| `tooling/tree-sitter-plum/queries/plum/format.scm` | Create | Topiary formatting rules |
+---
+### Task 1: Restructure as Cargo Workspace
+**Files:**
+- Replace: `Cargo.toml`
+- Create: `plum-core/Cargo.toml`
+- Create: `plum-core/src/lib.rs`
+- Move: `src/ast.rs` → `plum-core/src/ast.rs`
+- Move: `src/parser.rs` → `plum-core/src/parser.rs`
+**Interfaces:**
+- Produces: `plum-core` crate compilable with `cargo build -p plum-core`
+- [ ] **Step 1: Replace root Cargo.toml with workspace definition**
+```toml
+# Cargo.toml
+[workspace]
+members = ["plum-core", "plum-cli"]
+resolver = "2"
+```
+- [ ] **Step 2: Create `plum-core/Cargo.toml`**
+```toml
+[package]
+name = "plum-core"
+version = "0.1.0"
+edition = "2021"
+[dependencies]
+tree-sitter = "0.26"
+tree-sitter-plum = { path = "../tooling/tree-sitter-plum" }
+topiary-core = "0.7.3"
+topiary-tree-sitter-facade = "0.7.3"
+```
+- [ ] **Step 3: Move source files into plum-core**
+```bash
+mkdir -p plum-core/src
+cp src/ast.rs plum-core/src/ast.rs
+cp src/parser.rs plum-core/src/parser.rs
+```
+- [ ] **Step 4: Create `plum-core/src/lib.rs`**
+```rust
+pub mod ast;
+pub mod parser;
+pub mod formatter;
+pub use formatter::{format_source, FormatterError};
+pub use parser::AstParser;
+```
+- [ ] **Step 5: Verify plum-core compiles (formatter module is absent — add stub)**
+Add an empty `plum-core/src/formatter.rs` so lib.rs compiles:
+```rust
+// plum-core/src/formatter.rs
+#[derive(Debug)]
+pub enum FormatterError {
+    TopiaryCoreError(String),
+}
+impl std::fmt::Display for FormatterError {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        match self {
+            FormatterError::TopiaryCoreError(msg) => write!(f, "Topiary error: {msg}"),
+        }
+    }
+}
+impl std::error::Error for FormatterError {}
+pub fn format_source(_source: &str) -> Result<String, FormatterError> {
+    unimplemented!()
+}
+```
+Run: `cargo build -p plum-core`
+Expected: compiles (formatter fn is `unimplemented!()` — no test calls it yet)
+- [ ] **Step 6: Commit**
+```bash
+git add Cargo.toml plum-core/
+git commit -m "refactor: restructure as Cargo workspace, add plum-core skeleton"
+```
+---
+### Task 2: Write `format.scm` Topiary Formatting Rules
+**Files:**
+- Create: `tooling/tree-sitter-plum/queries/plum/format.scm`
+**Interfaces:**
+- Produces: formatting query embedded at compile time via `include_str!("../../../../tooling/tree-sitter-plum/queries/plum/format.scm")` from `plum-core/src/formatter.rs`
+Topiary annotation reference:
+- `@append_space` — insert a space after the matched node
+- `@prepend_space` — insert a space before the matched node
+- `@prepend_hardline` — start a new line before the matched node
+- `@append_hardline` — start a new line after the matched node
+- `@append_indent_start` — increase indent after this node
+- `@append_indent_end` — decrease indent after this node
+- `@append_empty_softline` — newline if multiline context, nothing if single-line
+- `@delete` — remove this node from output (used to strip indent/dedent tokens)
+- `@allow_blank_line_before` — permit blank lines before this node
+- [ ] **Step 1: Create `tooling/tree-sitter-plum/queries/plum/format.scm`**
+```scheme
+; ============================================================
+; Top-level items — blank line between each
+; ============================================================
+(source
+  (fn) @allow_blank_line_before)
+(source
+  (class) @allow_blank_line_before)
+(source
+  (enum) @allow_blank_line_before)
+(source
+  (trait) @allow_blank_line_before)
+(source
+  (const) @allow_blank_line_before)
+; ============================================================
+; Binary operators — space before and after
+; ============================================================
+[
+  "+"
+  "-"
+  "*"
+  "/"
+  "%"
+  "^"
+  "|"
+  "&"
+  "<<"
+  ">>"
+  ".."
+] @prepend_space @append_space
+[
+  "=="
+  "!="
+  "<"
+  "<="
+  ">"
+  ">="
+  "<>"
+  "&&"
+  "||"
+] @prepend_space @append_space
+; ============================================================
+; Assignment and arrows
+; ============================================================
+"=" @prepend_space @append_space
+"->" @prepend_space @append_space
+"=>" @prepend_space @append_space
+; ============================================================
+; Separators — no space before, one space after
+; ============================================================
+"," @append_space
+":" @append_space
+; ============================================================
+; Enum variant separators — new line before each |
+; ============================================================
+(enum_variant
+  "|" @prepend_hardline)
+; ============================================================
+; Function body — each statement on its own line
+; ============================================================
+(body
+  (_) @prepend_hardline)
+; ============================================================
+; Indent / dedent tokens — strip them (Topiary regenerates indentation)
+; ============================================================
+(indent) @delete
+(dedent) @delete
+(newline) @delete
+```
+- [ ] **Step 2: Commit**
+```bash
+git add tooling/tree-sitter-plum/queries/plum/format.scm
+git commit -m "feat: add Topiary format.scm formatting rules for Plum"
+```
+---
+### Task 3: Implement `formatter.rs`
+**Files:**
+- Modify: `plum-core/src/formatter.rs`
+**Interfaces:**
+- Consumes: `topiary-core`, `topiary-tree-sitter-facade`, `tree-sitter-plum::LANGUAGE`
+- Produces: `pub fn format_source(source: &str) -> Result<String, FormatterError>`
+The Topiary 0.7.3 API:
+```rust
+// topiary_core::formatter
+pub fn formatter(
+    input: &mut impl std::io::Read,
+    output: &mut impl std::io::Write,
+    query: &topiary_core::TopiaryQuery,
+    grammar: topiary_tree_sitter_facade::Language,
+    operation: topiary_core::Operation,
+) -> Result<(), topiary_core::FormatterError>
+```
+`topiary_core::Operation::Format { skip_idempotence, tolerate_parsing_errors }` is the format variant.
+`topiary_core::TopiaryQuery::new(grammar, query_text)` constructs the query.
+`tree_sitter_plum::LANGUAGE` is `LanguageFn`; convert to `tree_sitter::Language` via `.into()`, then to `topiary_tree_sitter_facade::Language` via `.into()`.
+- [ ] **Step 1: Replace the stub in `plum-core/src/formatter.rs`**
+```rust
+use std::io::Cursor;
+use topiary_core::{formatter, Operation, TopiaryQuery};
+use topiary_tree_sitter_facade::Language;
+const FORMAT_QUERY: &str =
+    include_str!("../../../../tooling/tree-sitter-plum/queries/plum/format.scm");
+#[derive(Debug)]
+pub enum FormatterError {
+    TopiaryCoreError(topiary_core::FormatterError),
+}
+impl std::fmt::Display for FormatterError {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        match self {
+            FormatterError::TopiaryCoreError(e) => write!(f, "Topiary error: {e}"),
+        }
+    }
+}
+impl std::error::Error for FormatterError {
+    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+        match self {
+            FormatterError::TopiaryCoreError(e) => Some(e),
+        }
+    }
+}
+impl From<topiary_core::FormatterError> for FormatterError {
+    fn from(e: topiary_core::FormatterError) -> Self {
+        FormatterError::TopiaryCoreError(e)
+    }
+}
+pub fn format_source(source: &str) -> Result<String, FormatterError> {
+    let ts_language: tree_sitter::Language = tree_sitter_plum::LANGUAGE.into();
+    let grammar: Language = ts_language.into();
+    let query = TopiaryQuery::new(&grammar, FORMAT_QUERY)?;
+    let mut input = Cursor::new(source.as_bytes());
+    let mut output = Vec::new();
+    formatter(
+        &mut input,
+        &mut output,
+        &query,
+        grammar,
+        Operation::Format {
+            skip_idempotence: false,
+            tolerate_parsing_errors: false,
+        },
+    )?;
+    Ok(String::from_utf8(output).expect("Topiary output is valid UTF-8"))
+}
+```
+- [ ] **Step 2: Verify it compiles**
+Run: `cargo build -p plum-core`
+Expected: compiles without errors
+If the `TopiaryQuery::new` signature or `formatter` signature differs, check `cargo doc --open -p topiary-core` and adjust accordingly. The 0.7.3 API may use a builder; the key types are `TopiaryQuery`, `Language` (facade), and `Operation`.
+- [ ] **Step 3: Commit**
+```bash
+git add plum-core/src/formatter.rs
+git commit -m "feat: implement format_source wrapping topiary-core"
+```
+---
+### Task 4: Integration Tests for `format_source`
+**Files:**
+- Create: `plum-core/tests/formatter_test.rs`
+**Interfaces:**
+- Consumes: `plum_core::format_source`
+- [ ] **Step 1: Create `plum-core/tests/formatter_test.rs`**
+```rust
+use plum_core::format_source;
+#[test]
+fn formats_simple_function() {
+    let input = "main() =\n  x = 1\n";
+    let result = format_source(input).expect("format_source should succeed");
+    // formatted output has the same content structure (exact spacing may vary)
+    assert!(result.contains("main()"));
+    assert!(result.contains("x ="));
+}
+#[test]
+fn formats_binary_operator_spacing() {
+    let input = "add<Int>(a: Int, b: Int) -> Int =\n  a+b\n";
+    let result = format_source(input).expect("format_source should succeed");
+    // binary operator gets spaces around it
+    assert!(result.contains("a + b"));
+}
+#[test]
+fn rejects_syntax_error() {
+    let input = "fn @@invalid@@\n";
+    let result = format_source(input);
+    assert!(result.is_err());
+}
+#[test]
+fn idempotent_on_already_formatted() {
+    let input = "main() =\n  x = 1 + 2\n";
+    let first = format_source(input).expect("first pass");
+    let second = format_source(&first).expect("second pass");
+    assert_eq!(first, second, "formatting should be idempotent");
+}
+```
+- [ ] **Step 2: Run the tests**
+Run: `cargo test -p plum-core`
+Expected: all 4 tests pass (the idempotence test verifies Topiary's double-format check)
+If `rejects_syntax_error` fails because Topiary tolerates the error anyway, change that test to verify it returns an `Err` with a descriptive message.
+- [ ] **Step 3: Commit**
+```bash
+git add plum-core/tests/formatter_test.rs
+git commit -m "test: add integration tests for format_source"
+```
+---
+### Task 5: Create `plum-cli` Binary
+**Files:**
+- Create: `plum-cli/Cargo.toml`
+- Create: `plum-cli/src/main.rs`
+**Interfaces:**
+- Consumes: `plum_core::format_source`, `plum_core::FormatterError`
+- Produces: `plum format <file>`, `plum format --check <file>`, `plum format --stdin`
+- [ ] **Step 1: Create `plum-cli/Cargo.toml`**
+```toml
+[package]
+name = "plum-cli"
+version = "0.1.0"
+edition = "2021"
+[[bin]]
+name = "plum"
+path = "src/main.rs"
+[dependencies]
+plum-core = { path = "../plum-core" }
+clap = { version = "4", features = ["derive"] }
+anyhow = "1"
+```
+- [ ] **Step 2: Create `plum-cli/src/main.rs`**
+```rust
+use std::fs;
+use std::io::{self, Read};
+use std::process;
+use anyhow::{Context, Result};
+use clap::{Parser, Subcommand};
+use plum_core::format_source;
+#[derive(Parser)]
+#[command(name = "plum", about = "The Plum language toolchain")]
+struct Cli {
+    #[command(subcommand)]
+    command: Command,
+}
+#[derive(Subcommand)]
+enum Command {
+    /// Format a Plum source file
+    Format {
+        /// File to format (omit to use --stdin)
+        file: Option<std::path::PathBuf>,
+        /// Check if file is formatted; exit 1 if it would change
+        #[arg(long)]
+        check: bool,
+        /// Read from stdin and write formatted source to stdout
+        #[arg(long)]
+        stdin: bool,
+    },
+}
+fn main() {
+    if let Err(e) = run() {
+        eprintln!("error: {e:#}");
+        process::exit(1);
+    }
+}
+fn run() -> Result<()> {
+    let cli = Cli::parse();
+    match cli.command {
+        Command::Format { file, check, stdin } => cmd_format(file, check, stdin),
+    }
+}
+fn cmd_format(
+    file: Option<std::path::PathBuf>,
+    check: bool,
+    use_stdin: bool,
+) -> Result<()> {
+    if use_stdin {
+        let mut source = String::new();
+        io::stdin()
+            .read_to_string(&mut source)
+            .context("failed to read stdin")?;
+        let formatted = format_source(&source).map_err(|e| anyhow::anyhow!("{e}"))?;
+        print!("{formatted}");
+        return Ok(());
+    }
+    let path = file.ok_or_else(|| anyhow::anyhow!("provide a file path or --stdin"))?;
+    let source =
+        fs::read_to_string(&path).with_context(|| format!("failed to read {}", path.display()))?;
+    let formatted = format_source(&source).map_err(|e| anyhow::anyhow!("{e}"))?;
+    if check {
+        if source != formatted {
+            eprintln!("{}: would reformat", path.display());
+            process::exit(1);
+        }
+        return Ok(());
+    }
+    if source != formatted {
+        fs::write(&path, formatted.as_bytes())
+            .with_context(|| format!("failed to write {}", path.display()))?;
+    }
+    Ok(())
+}
+```
+- [ ] **Step 3: Verify build**
+Run: `cargo build -p plum-cli`
+Expected: produces `target/debug/plum` binary
+- [ ] **Step 4: Smoke-test the binary**
+```bash
+echo 'main() =\n  x=1+2\n' > /tmp/smoke.plum
+cargo run -p plum-cli -- format /tmp/smoke.plum
+cat /tmp/smoke.plum
+```
+Expected: `x = 1 + 2` (spaces around `=` and `+`)
+```bash
+cargo run -p plum-cli -- format --check /tmp/smoke.plum
+echo "exit: $?"
+```
+Expected: exits 0 (file is already formatted after the previous run)
+```bash
+echo 'main()=\n  x=1\n' | cargo run -p plum-cli -- format --stdin
+```
+Expected: formatted source printed to stdout
+- [ ] **Step 5: Commit**
+```bash
+git add plum-cli/
+git commit -m "feat: add plum-cli binary with format subcommand"
+```
+---
+### Task 6: Clean Up Old Root Crate
+**Files:**
+- Delete: `src/` directory (the old root `src/ast.rs`, `src/parser.rs`, `src/main.rs`)
+**Interfaces:**
+- Consumes: nothing (cleanup only)
+- Produces: clean workspace with no dangling `src/`
+- [ ] **Step 1: Verify `src/` is no longer referenced**
+Run: `cargo build` from workspace root
+Expected: builds both `plum-core` and `plum-cli` with no errors
+- [ ] **Step 2: Remove old `src/` directory**
+```bash
+rm -rf src/
+```
+- [ ] **Step 3: Run full workspace check**
+Run: `cargo test`
+Expected: all tests pass; `src/` is gone
+- [ ] **Step 4: Commit**
+```bash
+git add -A
+git commit -m "chore: remove old root src/ after workspace migration"
+```
+---
+## Self-Review
+**Spec coverage check:**
+| Spec requirement | Covered by |
+|---|---|
+| Workspace restructure | Task 1 |
+| `plum-core` with ast, parser, formatter | Tasks 1, 3 |
+| `format.scm` Topiary rules | Task 2 |
+| `format_source` wrapping topiary-core | Task 3 |
+| Formatter integration tests | Task 4 |
+| `plum format <file>` in-place | Task 5 |
+| `plum format --check <file>` | Task 5 |
+| `plum format --stdin` | Task 5 |
+| `tolerate_parsing_errors = false` | Task 3 (`Operation::Format` field) |
+| Errors to stderr, exit 1 | Task 5 (`run()` + `process::exit(1)`) |
+| tree-sitter upgrade to 0.26 | Task 1 (`plum-core/Cargo.toml`) |
+| Old `src/` removed | Task 6 |
+**tree-sitter version note:** `topiary-core = "0.7.3"` resolves `tree-sitter = "0.26.11"`. `plum-core/Cargo.toml` must declare `tree-sitter = "0.26"` (not `"0.24.5"`) so Cargo uses a single copy. The `tree-sitter-plum` grammar crate uses `tree-sitter-language = "0.1"` at runtime (no direct tree-sitter dependency), so no patch is needed.
+**`include_str!` path:** `format.scm` is embedded via a path relative to `plum-core/src/formatter.rs`, which is `../../../../tooling/tree-sitter-plum/queries/plum/format.scm`. Verify this resolves correctly after the workspace move. If it doesn't, an alternative is to add a build script that copies the file into `OUT_DIR`.