plum

# CLI & Formatter Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Restructure the project as a Cargo workspace, extract a `plum-core` library, and add a `plum-cli` binary with a `format` subcommand backed by Topiary.

**Architecture:** Current `src/` code splits into `plum-core` (library: ast, parser, formatter) and `plum-cli` (thin binary). `formatter.rs` wraps `topiary-core`, embedding `format.scm` at compile time via `include_str!`. The CLI uses `clap` for argument parsing and delegates all logic to `plum-core`.

**Tech Stack:** Rust 2021 edition, Cargo workspaces, topiary-core 0.7.3, topiary-tree-sitter-facade 0.7.3, tree-sitter 0.26 (required by topiary), tree-sitter-plum 0.1.0, clap 4, anyhow 1.

## Global Constraints

- topiary-core version: `"0.7.3"` — exact, do not upgrade

- topiary-tree-sitter-facade version: `"0.7.3"` — exact, matches topiary-core

- tree-sitter version: `"0.26"` — topiary-core 0.7.3 resolves tree-sitter 0.26.11; use `"0.26"` (not `"0.24.5"`) in plum-core

- tree-sitter-plum version: `"0.1.0"` — local crate at `tooling/tree-sitter-plum`; reference via `path`

- clap version: `"4"` with `features = ["derive"]`

- anyhow version: `"1"`

- format.scm lives at `tooling/tree-sitter-plum/queries/plum/format.scm`

- `plum format <file>` edits in place silently on success; `--check` exits 1 if file would change; `--stdin` reads stdin, writes stdout

- `tolerate_parsing_errors = false` — refuse to format files with syntax errors

---

## File Map

| Path | Action | Responsibility |

|---|---|---|

| `Cargo.toml` | Replace | Workspace root — lists members, no `[package]` |

| `plum-core/Cargo.toml` | Create | Library dependencies (tree-sitter 0.26, topiary, etc.) |

| `plum-core/src/lib.rs` | Create | `pub mod` declarations + re-exports |

| `plum-core/src/ast.rs` | Move from `src/ast.rs` | Typed AST types (unchanged) |

| `plum-core/src/parser.rs` | Move from `src/parser.rs` | CST → AST walking (unchanged) |

| `plum-core/src/formatter.rs` | Create | Topiary wrapper — `format_source` |

| `plum-core/tests/formatter_test.rs` | Create | Integration tests for `format_source` |

| `plum-cli/Cargo.toml` | Create | Binary dependencies (plum-core, clap, anyhow) |

| `plum-cli/src/main.rs` | Create | clap CLI, file I/O, calls `plum_core::format_source` |

| `tooling/tree-sitter-plum/queries/plum/format.scm` | Create | Topiary formatting rules |

---

### Task 1: Restructure as Cargo Workspace

**Files:**

- Replace: `Cargo.toml`

- Create: `plum-core/Cargo.toml`

- Create: `plum-core/src/lib.rs`

- Move: `src/ast.rs` → `plum-core/src/ast.rs`

- Move: `src/parser.rs` → `plum-core/src/parser.rs`

**Interfaces:**

- Produces: `plum-core` crate compilable with `cargo build -p plum-core`

- [ ] **Step 1: Replace root Cargo.toml with workspace definition**

```toml

# Cargo.toml

[workspace]

members = ["plum-core", "plum-cli"]

resolver = "2"

```

- [ ] **Step 2: Create `plum-core/Cargo.toml`**

```toml

[package]

name = "plum-core"

version = "0.1.0"

edition = "2021"

[dependencies]

tree-sitter = "0.26"

tree-sitter-plum = { path = "../tooling/tree-sitter-plum" }

topiary-core = "0.7.3"

topiary-tree-sitter-facade = "0.7.3"

```

- [ ] **Step 3: Move source files into plum-core**

```bash

mkdir -p plum-core/src

cp src/ast.rs plum-core/src/ast.rs

cp src/parser.rs plum-core/src/parser.rs

```

- [ ] **Step 4: Create `plum-core/src/lib.rs`**

```rust

pub mod ast;

pub mod parser;

pub mod formatter;

pub use formatter::{format_source, FormatterError};

pub use parser::AstParser;

```

- [ ] **Step 5: Verify plum-core compiles (formatter module is absent — add stub)**

Add an empty `plum-core/src/formatter.rs` so lib.rs compiles:

```rust

// plum-core/src/formatter.rs

#[derive(Debug)]

pub enum FormatterError {

    TopiaryCoreError(String),

}

impl std::fmt::Display for FormatterError {

    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {

        match self {

            FormatterError::TopiaryCoreError(msg) => write!(f, "Topiary error: {msg}"),

        }

    }

}

impl std::error::Error for FormatterError {}

pub fn format_source(_source: &str) -> Result<String, FormatterError> {

    unimplemented!()

}

```

Run: `cargo build -p plum-core`

Expected: compiles (formatter fn is `unimplemented!()` — no test calls it yet)

- [ ] **Step 6: Commit**

```bash

git add Cargo.toml plum-core/

git commit -m "refactor: restructure as Cargo workspace, add plum-core skeleton"

```

---

### Task 2: Write `format.scm` Topiary Formatting Rules

**Files:**

- Create: `tooling/tree-sitter-plum/queries/plum/format.scm`

**Interfaces:**

- Produces: formatting query embedded at compile time via `include_str!("../../../../tooling/tree-sitter-plum/queries/plum/format.scm")` from `plum-core/src/formatter.rs`

Topiary annotation reference:

- `@append_space` — insert a space after the matched node

- `@prepend_space` — insert a space before the matched node

- `@prepend_hardline` — start a new line before the matched node

- `@append_hardline` — start a new line after the matched node

- `@append_indent_start` — increase indent after this node

- `@append_indent_end` — decrease indent after this node

- `@append_empty_softline` — newline if multiline context, nothing if single-line

- `@delete` — remove this node from output (used to strip indent/dedent tokens)

- `@allow_blank_line_before` — permit blank lines before this node

- [ ] **Step 1: Create `tooling/tree-sitter-plum/queries/plum/format.scm`**

```scheme

; ============================================================

; Top-level items — blank line between each

; ============================================================

(source

  (fn) @allow_blank_line_before)

(source

  (class) @allow_blank_line_before)

(source

  (enum) @allow_blank_line_before)

(source

  (trait) @allow_blank_line_before)

(source

  (const) @allow_blank_line_before)

; ============================================================

; Binary operators — space before and after

; ============================================================

[

  "+"

  "-"

  "*"

  "/"

  "%"

  "^"

  "|"

  "&"

  "<<"

  ">>"

  ".."

] @prepend_space @append_space

[

  "=="

  "!="

  "<"

  "<="

  ">"

  ">="

  "<>"

  "&&"

  "||"

] @prepend_space @append_space

; ============================================================

; Assignment and arrows

; ============================================================

"=" @prepend_space @append_space

"->" @prepend_space @append_space

"=>" @prepend_space @append_space

; ============================================================

; Separators — no space before, one space after

; ============================================================

"," @append_space

":" @append_space

; ============================================================

; Enum variant separators — new line before each |

; ============================================================

(enum_variant

  "|" @prepend_hardline)

; ============================================================

; Function body — each statement on its own line

; ============================================================

(body

  (_) @prepend_hardline)

; ============================================================

; Indent / dedent tokens — strip them (Topiary regenerates indentation)

; ============================================================

(indent) @delete

(dedent) @delete

(newline) @delete

```

- [ ] **Step 2: Commit**

```bash

git add tooling/tree-sitter-plum/queries/plum/format.scm

git commit -m "feat: add Topiary format.scm formatting rules for Plum"

```

---

### Task 3: Implement `formatter.rs`

**Files:**

- Modify: `plum-core/src/formatter.rs`

**Interfaces:**

- Consumes: `topiary-core`, `topiary-tree-sitter-facade`, `tree-sitter-plum::LANGUAGE`

- Produces: `pub fn format_source(source: &str) -> Result<String, FormatterError>`

The Topiary 0.7.3 API:

```rust

// topiary_core::formatter

pub fn formatter(

    input: &mut impl std::io::Read,

    output: &mut impl std::io::Write,

    query: &topiary_core::TopiaryQuery,

    grammar: topiary_tree_sitter_facade::Language,

    operation: topiary_core::Operation,

) -> Result<(), topiary_core::FormatterError>

```

`topiary_core::Operation::Format { skip_idempotence, tolerate_parsing_errors }` is the format variant.

`topiary_core::TopiaryQuery::new(grammar, query_text)` constructs the query.

`tree_sitter_plum::LANGUAGE` is `LanguageFn`; convert to `tree_sitter::Language` via `.into()`, then to `topiary_tree_sitter_facade::Language` via `.into()`.

- [ ] **Step 1: Replace the stub in `plum-core/src/formatter.rs`**

```rust

use std::io::Cursor;

use topiary_core::{formatter, Operation, TopiaryQuery};

use topiary_tree_sitter_facade::Language;

const FORMAT_QUERY: &str =

    include_str!("../../../../tooling/tree-sitter-plum/queries/plum/format.scm");

#[derive(Debug)]

pub enum FormatterError {

    TopiaryCoreError(topiary_core::FormatterError),

}

impl std::fmt::Display for FormatterError {

    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {

        match self {

            FormatterError::TopiaryCoreError(e) => write!(f, "Topiary error: {e}"),

        }

    }

}

impl std::error::Error for FormatterError {

    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {

        match self {

            FormatterError::TopiaryCoreError(e) => Some(e),

        }

    }

}

impl From<topiary_core::FormatterError> for FormatterError {

    fn from(e: topiary_core::FormatterError) -> Self {

        FormatterError::TopiaryCoreError(e)

    }

}

pub fn format_source(source: &str) -> Result<String, FormatterError> {

    let ts_language: tree_sitter::Language = tree_sitter_plum::LANGUAGE.into();

    let grammar: Language = ts_language.into();

    let query = TopiaryQuery::new(&grammar, FORMAT_QUERY)?;

    let mut input = Cursor::new(source.as_bytes());

    let mut output = Vec::new();

    formatter(

        &mut input,

        &mut output,

        &query,

        grammar,

        Operation::Format {

            skip_idempotence: false,

            tolerate_parsing_errors: false,

        },

    )?;

    Ok(String::from_utf8(output).expect("Topiary output is valid UTF-8"))

}

```

- [ ] **Step 2: Verify it compiles**

Run: `cargo build -p plum-core`

Expected: compiles without errors

If the `TopiaryQuery::new` signature or `formatter` signature differs, check `cargo doc --open -p topiary-core` and adjust accordingly. The 0.7.3 API may use a builder; the key types are `TopiaryQuery`, `Language` (facade), and `Operation`.

- [ ] **Step 3: Commit**

```bash

git add plum-core/src/formatter.rs

git commit -m "feat: implement format_source wrapping topiary-core"

```

---

### Task 4: Integration Tests for `format_source`

**Files:**

- Create: `plum-core/tests/formatter_test.rs`

**Interfaces:**

- Consumes: `plum_core::format_source`

- [ ] **Step 1: Create `plum-core/tests/formatter_test.rs`**

```rust

use plum_core::format_source;

#[test]

fn formats_simple_function() {

    let input = "main() =\n  x = 1\n";

    let result = format_source(input).expect("format_source should succeed");

    // formatted output has the same content structure (exact spacing may vary)

    assert!(result.contains("main()"));

    assert!(result.contains("x ="));

}

#[test]

fn formats_binary_operator_spacing() {

    let input = "add<Int>(a: Int, b: Int) -> Int =\n  a+b\n";

    let result = format_source(input).expect("format_source should succeed");

    // binary operator gets spaces around it

    assert!(result.contains("a + b"));

}

#[test]

fn rejects_syntax_error() {

    let input = "fn @@invalid@@\n";

    let result = format_source(input);

    assert!(result.is_err());

}

#[test]

fn idempotent_on_already_formatted() {

    let input = "main() =\n  x = 1 + 2\n";

    let first = format_source(input).expect("first pass");

    let second = format_source(&first).expect("second pass");

    assert_eq!(first, second, "formatting should be idempotent");

}

```

- [ ] **Step 2: Run the tests**

Run: `cargo test -p plum-core`

Expected: all 4 tests pass (the idempotence test verifies Topiary's double-format check)

If `rejects_syntax_error` fails because Topiary tolerates the error anyway, change that test to verify it returns an `Err` with a descriptive message.

- [ ] **Step 3: Commit**

```bash

git add plum-core/tests/formatter_test.rs

git commit -m "test: add integration tests for format_source"

```

---

### Task 5: Create `plum-cli` Binary

**Files:**

- Create: `plum-cli/Cargo.toml`

- Create: `plum-cli/src/main.rs`

**Interfaces:**

- Consumes: `plum_core::format_source`, `plum_core::FormatterError`

- Produces: `plum format <file>`, `plum format --check <file>`, `plum format --stdin`

- [ ] **Step 1: Create `plum-cli/Cargo.toml`**

```toml

[package]

name = "plum-cli"

version = "0.1.0"

edition = "2021"

[[bin]]

name = "plum"

path = "src/main.rs"

[dependencies]

plum-core = { path = "../plum-core" }

clap = { version = "4", features = ["derive"] }

anyhow = "1"

```

- [ ] **Step 2: Create `plum-cli/src/main.rs`**

```rust

use std::fs;

use std::io::{self, Read};

use std::process;

use anyhow::{Context, Result};

use clap::{Parser, Subcommand};

use plum_core::format_source;

#[derive(Parser)]

#[command(name = "plum", about = "The Plum language toolchain")]

struct Cli {

    #[command(subcommand)]

    command: Command,

}

#[derive(Subcommand)]

enum Command {

    /// Format a Plum source file

    Format {

        /// File to format (omit to use --stdin)

        file: Option<std::path::PathBuf>,

        /// Check if file is formatted; exit 1 if it would change

        #[arg(long)]

        check: bool,

        /// Read from stdin and write formatted source to stdout

        #[arg(long)]

        stdin: bool,

    },

}

fn main() {

    if let Err(e) = run() {

        eprintln!("error: {e:#}");

        process::exit(1);

    }

}

fn run() -> Result<()> {

    let cli = Cli::parse();

    match cli.command {

        Command::Format { file, check, stdin } => cmd_format(file, check, stdin),

    }

}

fn cmd_format(

    file: Option<std::path::PathBuf>,

    check: bool,

    use_stdin: bool,

) -> Result<()> {

    if use_stdin {

        let mut source = String::new();

        io::stdin()

            .read_to_string(&mut source)

            .context("failed to read stdin")?;

        let formatted = format_source(&source).map_err(|e| anyhow::anyhow!("{e}"))?;

        print!("{formatted}");

        return Ok(());

    }

    let path = file.ok_or_else(|| anyhow::anyhow!("provide a file path or --stdin"))?;

    let source =

        fs::read_to_string(&path).with_context(|| format!("failed to read {}", path.display()))?;

    let formatted = format_source(&source).map_err(|e| anyhow::anyhow!("{e}"))?;

    if check {

        if source != formatted {

            eprintln!("{}: would reformat", path.display());

            process::exit(1);

        }

        return Ok(());

    }

    if source != formatted {

        fs::write(&path, formatted.as_bytes())

            .with_context(|| format!("failed to write {}", path.display()))?;

    }

    Ok(())

}

```

- [ ] **Step 3: Verify build**

Run: `cargo build -p plum-cli`

Expected: produces `target/debug/plum` binary

- [ ] **Step 4: Smoke-test the binary**

```bash

echo 'main() =\n  x=1+2\n' > /tmp/smoke.plum

cargo run -p plum-cli -- format /tmp/smoke.plum

cat /tmp/smoke.plum

```

Expected: `x = 1 + 2` (spaces around `=` and `+`)

```bash

cargo run -p plum-cli -- format --check /tmp/smoke.plum

echo "exit: $?"

```

Expected: exits 0 (file is already formatted after the previous run)

```bash

echo 'main()=\n  x=1\n' | cargo run -p plum-cli -- format --stdin

```

Expected: formatted source printed to stdout

- [ ] **Step 5: Commit**

```bash

git add plum-cli/

git commit -m "feat: add plum-cli binary with format subcommand"

```

---

### Task 6: Clean Up Old Root Crate

**Files:**

- Delete: `src/` directory (the old root `src/ast.rs`, `src/parser.rs`, `src/main.rs`)

**Interfaces:**

- Consumes: nothing (cleanup only)

- Produces: clean workspace with no dangling `src/`

- [ ] **Step 1: Verify `src/` is no longer referenced**

Run: `cargo build` from workspace root

Expected: builds both `plum-core` and `plum-cli` with no errors

- [ ] **Step 2: Remove old `src/` directory**

```bash

rm -rf src/

```

- [ ] **Step 3: Run full workspace check**

Run: `cargo test`

Expected: all tests pass; `src/` is gone

- [ ] **Step 4: Commit**

```bash

git add -A

git commit -m "chore: remove old root src/ after workspace migration"

```

---

## Self-Review

**Spec coverage check:**

| Spec requirement | Covered by |

|---|---|

| Workspace restructure | Task 1 |

| `plum-core` with ast, parser, formatter | Tasks 1, 3 |

| `format.scm` Topiary rules | Task 2 |

| `format_source` wrapping topiary-core | Task 3 |

| Formatter integration tests | Task 4 |

| `plum format <file>` in-place | Task 5 |

| `plum format --check <file>` | Task 5 |

| `plum format --stdin` | Task 5 |

| `tolerate_parsing_errors = false` | Task 3 (`Operation::Format` field) |

| Errors to stderr, exit 1 | Task 5 (`run()` + `process::exit(1)`) |

| tree-sitter upgrade to 0.26 | Task 1 (`plum-core/Cargo.toml`) |

| Old `src/` removed | Task 6 |

**tree-sitter version note:** `topiary-core = "0.7.3"` resolves `tree-sitter = "0.26.11"`. `plum-core/Cargo.toml` must declare `tree-sitter = "0.26"` (not `"0.24.5"`) so Cargo uses a single copy. The `tree-sitter-plum` grammar crate uses `tree-sitter-language = "0.1"` at runtime (no direct tree-sitter dependency), so no patch is needed.

**`include_str!` path:** `format.scm` is embedded via a path relative to `plum-core/src/formatter.rs`, which is `../../../../tooling/tree-sitter-plum/queries/plum/format.scm`. Verify this resolves correctly after the workspace move. If it doesn't, an alternative is to add a build script that copies the file into `OUT_DIR`.