plum

# CLI & Formatter Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Restructure the project as a Cargo workspace, extract a `plum-core` library, and add a `plum-cli` binary with a `format` subcommand backed by Topiary.

**Architecture:** Current `src/` code splits into `plum-core` (library: ast, parser, formatter) and `plum-cli` (thin binary). `formatter.rs` wraps `topiary-core`, embedding `format.scm` at compile time via `include_str!`. The CLI uses `clap` for argument parsing and delegates all logic to `plum-core`.

**Tech Stack:** Rust 2021 edition, Cargo workspaces, topiary-core 0.7.3, topiary-tree-sitter-facade 0.7.3, tree-sitter 0.26 (required by topiary), tree-sitter-plum 0.1.0, clap 4, anyhow 1.

## Global Constraints

- topiary-core version: `"0.7.3"` — exact, do not upgrade
- topiary-tree-sitter-facade version: `"0.7.3"` — exact, matches topiary-core
- tree-sitter version: `"0.26"` — topiary-core 0.7.3 resolves tree-sitter 0.26.11; use `"0.26"` (not `"0.24.5"`) in plum-core
- tree-sitter-plum version: `"0.1.0"` — local crate at `tooling/tree-sitter-plum`; reference via `path`
- clap version: `"4"` with `features = ["derive"]`
- anyhow version: `"1"`
- format.scm lives at `tooling/tree-sitter-plum/queries/plum/format.scm`
- `plum format <file>` edits in place silently on success; `--check` exits 1 if file would change; `--stdin` reads stdin, writes stdout
- `tolerate_parsing_errors = false` — refuse to format files with syntax errors

---

## File Map

| Path | Action | Responsibility |
|---|---|---|
| `Cargo.toml` | Replace | Workspace root — lists members, no `[package]` |
| `plum-core/Cargo.toml` | Create | Library dependencies (tree-sitter 0.26, topiary, etc.) |
| `plum-core/src/lib.rs` | Create | `pub mod` declarations + re-exports |
| `plum-core/src/ast.rs` | Move from `src/ast.rs` | Typed AST types (unchanged) |
| `plum-core/src/parser.rs` | Move from `src/parser.rs` | CST → AST walking (unchanged) |
| `plum-core/src/formatter.rs` | Create | Topiary wrapper — `format_source` |
| `plum-core/tests/formatter_test.rs` | Create | Integration tests for `format_source` |
| `plum-cli/Cargo.toml` | Create | Binary dependencies (plum-core, clap, anyhow) |
| `plum-cli/src/main.rs` | Create | clap CLI, file I/O, calls `plum_core::format_source` |
| `tooling/tree-sitter-plum/queries/plum/format.scm` | Create | Topiary formatting rules |

---

### Task 1: Restructure as Cargo Workspace

**Files:**
- Replace: `Cargo.toml`
- Create: `plum-core/Cargo.toml`
- Create: `plum-core/src/lib.rs`
- Move: `src/ast.rs` → `plum-core/src/ast.rs`
- Move: `src/parser.rs` → `plum-core/src/parser.rs`

**Interfaces:**
- Produces: `plum-core` crate compilable with `cargo build -p plum-core`

- [ ] **Step 1: Replace root Cargo.toml with workspace definition**

```toml
# Cargo.toml
[workspace]
members = ["plum-core", "plum-cli"]
resolver = "2"
```

- [ ] **Step 2: Create `plum-core/Cargo.toml`**

```toml
[package]
name = "plum-core"
version = "0.1.0"
edition = "2021"

[dependencies]
tree-sitter = "0.26"
tree-sitter-plum = { path = "../tooling/tree-sitter-plum" }
topiary-core = "0.7.3"
topiary-tree-sitter-facade = "0.7.3"
```

- [ ] **Step 3: Move source files into plum-core**

```bash
mkdir -p plum-core/src
cp src/ast.rs plum-core/src/ast.rs
cp src/parser.rs plum-core/src/parser.rs
```

- [ ] **Step 4: Create `plum-core/src/lib.rs`**

```rust
pub mod ast;
pub mod parser;
pub mod formatter;

pub use formatter::{format_source, FormatterError};
pub use parser::AstParser;
```

- [ ] **Step 5: Verify plum-core compiles (formatter module is absent — add stub)**

Add an empty `plum-core/src/formatter.rs` so lib.rs compiles:

```rust
// plum-core/src/formatter.rs
#[derive(Debug)]
pub enum FormatterError {
    TopiaryCoreError(String),
}

impl std::fmt::Display for FormatterError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            FormatterError::TopiaryCoreError(msg) => write!(f, "Topiary error: {msg}"),
        }
    }
}

impl std::error::Error for FormatterError {}

pub fn format_source(_source: &str) -> Result<String, FormatterError> {
    unimplemented!()
}
```

Run: `cargo build -p plum-core`
Expected: compiles (formatter fn is `unimplemented!()` — no test calls it yet)

- [ ] **Step 6: Commit**

```bash
git add Cargo.toml plum-core/
git commit -m "refactor: restructure as Cargo workspace, add plum-core skeleton"
```

---

### Task 2: Write `format.scm` Topiary Formatting Rules

**Files:**
- Create: `tooling/tree-sitter-plum/queries/plum/format.scm`

**Interfaces:**
- Produces: formatting query embedded at compile time via `include_str!("../../../../tooling/tree-sitter-plum/queries/plum/format.scm")` from `plum-core/src/formatter.rs`

Topiary annotation reference:
- `@append_space` — insert a space after the matched node
- `@prepend_space` — insert a space before the matched node
- `@prepend_hardline` — start a new line before the matched node
- `@append_hardline` — start a new line after the matched node
- `@append_indent_start` — increase indent after this node
- `@append_indent_end` — decrease indent after this node
- `@append_empty_softline` — newline if multiline context, nothing if single-line
- `@delete` — remove this node from output (used to strip indent/dedent tokens)
- `@allow_blank_line_before` — permit blank lines before this node

- [ ] **Step 1: Create `tooling/tree-sitter-plum/queries/plum/format.scm`**

```scheme
; ============================================================
; Top-level items — blank line between each
; ============================================================

(source
  (fn) @allow_blank_line_before)

(source
  (class) @allow_blank_line_before)

(source
  (enum) @allow_blank_line_before)

(source
  (trait) @allow_blank_line_before)

(source
  (const) @allow_blank_line_before)

; ============================================================
; Binary operators — space before and after
; ============================================================

[
  "+"
  "-"
  "*"
  "/"
  "%"
  "^"
  "|"
  "&"
  "<<"
  ">>"
  ".."
] @prepend_space @append_space

[
  "=="
  "!="
  "<"
  "<="
  ">"
  ">="
  "<>"
  "&&"
  "||"
] @prepend_space @append_space

; ============================================================
; Assignment and arrows
; ============================================================

"=" @prepend_space @append_space

"->" @prepend_space @append_space

"=>" @prepend_space @append_space

; ============================================================
; Separators — no space before, one space after
; ============================================================

"," @append_space

":" @append_space

; ============================================================
; Enum variant separators — new line before each |
; ============================================================

(enum_variant
  "|" @prepend_hardline)

; ============================================================
; Function body — each statement on its own line
; ============================================================

(body
  (_) @prepend_hardline)

; ============================================================
; Indent / dedent tokens — strip them (Topiary regenerates indentation)
; ============================================================

(indent) @delete

(dedent) @delete

(newline) @delete
```

- [ ] **Step 2: Commit**

```bash
git add tooling/tree-sitter-plum/queries/plum/format.scm
git commit -m "feat: add Topiary format.scm formatting rules for Plum"
```

---

### Task 3: Implement `formatter.rs`

**Files:**
- Modify: `plum-core/src/formatter.rs`

**Interfaces:**
- Consumes: `topiary-core`, `topiary-tree-sitter-facade`, `tree-sitter-plum::LANGUAGE`
- Produces: `pub fn format_source(source: &str) -> Result<String, FormatterError>`

The Topiary 0.7.3 API:
```rust
// topiary_core::formatter
pub fn formatter(
    input: &mut impl std::io::Read,
    output: &mut impl std::io::Write,
    query: &topiary_core::TopiaryQuery,
    grammar: topiary_tree_sitter_facade::Language,
    operation: topiary_core::Operation,
) -> Result<(), topiary_core::FormatterError>
```

`topiary_core::Operation::Format { skip_idempotence, tolerate_parsing_errors }` is the format variant.

`topiary_core::TopiaryQuery::new(grammar, query_text)` constructs the query.

`tree_sitter_plum::LANGUAGE` is `LanguageFn`; convert to `tree_sitter::Language` via `.into()`, then to `topiary_tree_sitter_facade::Language` via `.into()`.

- [ ] **Step 1: Replace the stub in `plum-core/src/formatter.rs`**

```rust
use std::io::Cursor;

use topiary_core::{formatter, Operation, TopiaryQuery};
use topiary_tree_sitter_facade::Language;

const FORMAT_QUERY: &str =
    include_str!("../../../../tooling/tree-sitter-plum/queries/plum/format.scm");

#[derive(Debug)]
pub enum FormatterError {
    TopiaryCoreError(topiary_core::FormatterError),
}

impl std::fmt::Display for FormatterError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            FormatterError::TopiaryCoreError(e) => write!(f, "Topiary error: {e}"),
        }
    }
}

impl std::error::Error for FormatterError {
    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
        match self {
            FormatterError::TopiaryCoreError(e) => Some(e),
        }
    }
}

impl From<topiary_core::FormatterError> for FormatterError {
    fn from(e: topiary_core::FormatterError) -> Self {
        FormatterError::TopiaryCoreError(e)
    }
}

pub fn format_source(source: &str) -> Result<String, FormatterError> {
    let ts_language: tree_sitter::Language = tree_sitter_plum::LANGUAGE.into();
    let grammar: Language = ts_language.into();
    let query = TopiaryQuery::new(&grammar, FORMAT_QUERY)?;
    let mut input = Cursor::new(source.as_bytes());
    let mut output = Vec::new();
    formatter(
        &mut input,
        &mut output,
        &query,
        grammar,
        Operation::Format {
            skip_idempotence: false,
            tolerate_parsing_errors: false,
        },
    )?;
    Ok(String::from_utf8(output).expect("Topiary output is valid UTF-8"))
}
```

- [ ] **Step 2: Verify it compiles**

Run: `cargo build -p plum-core`
Expected: compiles without errors

If the `TopiaryQuery::new` signature or `formatter` signature differs, check `cargo doc --open -p topiary-core` and adjust accordingly. The 0.7.3 API may use a builder; the key types are `TopiaryQuery`, `Language` (facade), and `Operation`.

- [ ] **Step 3: Commit**

```bash
git add plum-core/src/formatter.rs
git commit -m "feat: implement format_source wrapping topiary-core"
```

---

### Task 4: Integration Tests for `format_source`

**Files:**
- Create: `plum-core/tests/formatter_test.rs`

**Interfaces:**
- Consumes: `plum_core::format_source`

- [ ] **Step 1: Create `plum-core/tests/formatter_test.rs`**

```rust
use plum_core::format_source;

#[test]
fn formats_simple_function() {
    let input = "main() =\n  x = 1\n";
    let result = format_source(input).expect("format_source should succeed");
    // formatted output has the same content structure (exact spacing may vary)
    assert!(result.contains("main()"));
    assert!(result.contains("x ="));
}

#[test]
fn formats_binary_operator_spacing() {
    let input = "add<Int>(a: Int, b: Int) -> Int =\n  a+b\n";
    let result = format_source(input).expect("format_source should succeed");
    // binary operator gets spaces around it
    assert!(result.contains("a + b"));
}

#[test]
fn rejects_syntax_error() {
    let input = "fn @@invalid@@\n";
    let result = format_source(input);
    assert!(result.is_err());
}

#[test]
fn idempotent_on_already_formatted() {
    let input = "main() =\n  x = 1 + 2\n";
    let first = format_source(input).expect("first pass");
    let second = format_source(&first).expect("second pass");
    assert_eq!(first, second, "formatting should be idempotent");
}
```

- [ ] **Step 2: Run the tests**

Run: `cargo test -p plum-core`
Expected: all 4 tests pass (the idempotence test verifies Topiary's double-format check)

If `rejects_syntax_error` fails because Topiary tolerates the error anyway, change that test to verify it returns an `Err` with a descriptive message.

- [ ] **Step 3: Commit**

```bash
git add plum-core/tests/formatter_test.rs
git commit -m "test: add integration tests for format_source"
```

---

### Task 5: Create `plum-cli` Binary

**Files:**
- Create: `plum-cli/Cargo.toml`
- Create: `plum-cli/src/main.rs`

**Interfaces:**
- Consumes: `plum_core::format_source`, `plum_core::FormatterError`
- Produces: `plum format <file>`, `plum format --check <file>`, `plum format --stdin`

- [ ] **Step 1: Create `plum-cli/Cargo.toml`**

```toml
[package]
name = "plum-cli"
version = "0.1.0"
edition = "2021"

[[bin]]
name = "plum"
path = "src/main.rs"

[dependencies]
plum-core = { path = "../plum-core" }
clap = { version = "4", features = ["derive"] }
anyhow = "1"
```

- [ ] **Step 2: Create `plum-cli/src/main.rs`**

```rust
use std::fs;
use std::io::{self, Read};
use std::process;

use anyhow::{Context, Result};
use clap::{Parser, Subcommand};

use plum_core::format_source;

#[derive(Parser)]
#[command(name = "plum", about = "The Plum language toolchain")]
struct Cli {
    #[command(subcommand)]
    command: Command,
}

#[derive(Subcommand)]
enum Command {
    /// Format a Plum source file
    Format {
        /// File to format (omit to use --stdin)
        file: Option<std::path::PathBuf>,
        /// Check if file is formatted; exit 1 if it would change
        #[arg(long)]
        check: bool,
        /// Read from stdin and write formatted source to stdout
        #[arg(long)]
        stdin: bool,
    },
}

fn main() {
    if let Err(e) = run() {
        eprintln!("error: {e:#}");
        process::exit(1);
    }
}

fn run() -> Result<()> {
    let cli = Cli::parse();
    match cli.command {
        Command::Format { file, check, stdin } => cmd_format(file, check, stdin),
    }
}

fn cmd_format(
    file: Option<std::path::PathBuf>,
    check: bool,
    use_stdin: bool,
) -> Result<()> {
    if use_stdin {
        let mut source = String::new();
        io::stdin()
            .read_to_string(&mut source)
            .context("failed to read stdin")?;
        let formatted = format_source(&source).map_err(|e| anyhow::anyhow!("{e}"))?;
        print!("{formatted}");
        return Ok(());
    }

    let path = file.ok_or_else(|| anyhow::anyhow!("provide a file path or --stdin"))?;
    let source =
        fs::read_to_string(&path).with_context(|| format!("failed to read {}", path.display()))?;
    let formatted = format_source(&source).map_err(|e| anyhow::anyhow!("{e}"))?;

    if check {
        if source != formatted {
            eprintln!("{}: would reformat", path.display());
            process::exit(1);
        }
        return Ok(());
    }

    if source != formatted {
        fs::write(&path, formatted.as_bytes())
            .with_context(|| format!("failed to write {}", path.display()))?;
    }
    Ok(())
}
```

- [ ] **Step 3: Verify build**

Run: `cargo build -p plum-cli`
Expected: produces `target/debug/plum` binary

- [ ] **Step 4: Smoke-test the binary**

```bash
echo 'main() =\n  x=1+2\n' > /tmp/smoke.plum
cargo run -p plum-cli -- format /tmp/smoke.plum
cat /tmp/smoke.plum
```

Expected: `x = 1 + 2` (spaces around `=` and `+`)

```bash
cargo run -p plum-cli -- format --check /tmp/smoke.plum
echo "exit: $?"
```

Expected: exits 0 (file is already formatted after the previous run)

```bash
echo 'main()=\n  x=1\n' | cargo run -p plum-cli -- format --stdin
```

Expected: formatted source printed to stdout

- [ ] **Step 5: Commit**

```bash
git add plum-cli/
git commit -m "feat: add plum-cli binary with format subcommand"
```

---

### Task 6: Clean Up Old Root Crate

**Files:**
- Delete: `src/` directory (the old root `src/ast.rs`, `src/parser.rs`, `src/main.rs`)

**Interfaces:**
- Consumes: nothing (cleanup only)
- Produces: clean workspace with no dangling `src/`

- [ ] **Step 1: Verify `src/` is no longer referenced**

Run: `cargo build` from workspace root
Expected: builds both `plum-core` and `plum-cli` with no errors

- [ ] **Step 2: Remove old `src/` directory**

```bash
rm -rf src/
```

- [ ] **Step 3: Run full workspace check**

Run: `cargo test`
Expected: all tests pass; `src/` is gone

- [ ] **Step 4: Commit**

```bash
git add -A
git commit -m "chore: remove old root src/ after workspace migration"
```

---

## Self-Review

**Spec coverage check:**

| Spec requirement | Covered by |
|---|---|
| Workspace restructure | Task 1 |
| `plum-core` with ast, parser, formatter | Tasks 1, 3 |
| `format.scm` Topiary rules | Task 2 |
| `format_source` wrapping topiary-core | Task 3 |
| Formatter integration tests | Task 4 |
| `plum format <file>` in-place | Task 5 |
| `plum format --check <file>` | Task 5 |
| `plum format --stdin` | Task 5 |
| `tolerate_parsing_errors = false` | Task 3 (`Operation::Format` field) |
| Errors to stderr, exit 1 | Task 5 (`run()` + `process::exit(1)`) |
| tree-sitter upgrade to 0.26 | Task 1 (`plum-core/Cargo.toml`) |
| Old `src/` removed | Task 6 |

**tree-sitter version note:** `topiary-core = "0.7.3"` resolves `tree-sitter = "0.26.11"`. `plum-core/Cargo.toml` must declare `tree-sitter = "0.26"` (not `"0.24.5"`) so Cargo uses a single copy. The `tree-sitter-plum` grammar crate uses `tree-sitter-language = "0.1"` at runtime (no direct tree-sitter dependency), so no patch is needed.

**`include_str!` path:** `format.scm` is embedded via a path relative to `plum-core/src/formatter.rs`, which is `../../../../tooling/tree-sitter-plum/queries/plum/format.scm`. Verify this resolves correctly after the workspace move. If it doesn't, an alternative is to add a build script that copies the file into `OUT_DIR`.