plum

#treesitter#compiler#wasm

git clone https://git.pyrossh.dev/plum

A statically typed, imperative programming language inspired by rust, python

e7e7156 — Peter John 2026-11-02T11:25:56+05:30 
docs: simplify slices design to List views + zero-copy Str
Files changed (1) hide show
  1. docs/plans/2026-11-02-go-slices.review.md +36 -51
docs/plans/2026-11-02-go-slices.review.md CHANGED
@@ -1,73 +1,58 @@
1
- # Slices and List (review plan)
1
+ # List views and zero-copy Str (review plan)
2
2
 
3
- **Status:** approved 2026-11-02. Decisions: Option B, a sub-slice append copies, and `get` returns an error union.
3
+ **Status:** approved 2026-11-02. Option B, simplified: `List.slice` returns a `List`, and there is no separate slice type.
4
4
 
5
5
  ## Goal
6
- Two sequence types with one job each:
7
- - **`List[T]`**: the growable, shared, mutable object. It's a `record`, and `push` changes it in place for everyone holding it. Its API is unchanged.
6
+ - **`List[T]`** stays the one growable, shared, mutable sequence type, with its API unchanged.
8
- - **`[]T`** (spelled `Slice[T]` in std): a small value view (backing array, start, length). Slicing is free and copies nothing. Writes through `set` are visible to every view of the same array.
9
-
10
- `Buffer` and `ByteSlice` go away: `List[Byte]` is the growable byte buffer and `[]Byte` is the byte view. `Str` holds a `[]Byte`, so substrings stop copying.
7
+ - **`xs.slice(i, j)`** returns a `List[T]` *view* over the same backing array: one small record, no element copy.
8
+ - **`Str`** gets the same layout, so `sub`, `slice`, `trim` and `split` stop copying bytes.
9
+ - **`Buffer`** becomes `List[Byte]`, and **`ByteSlice`** is removed.
11
10
 
12
11
  ```plum
13
- ps := List[Particle]()
14
- ps.push(Particle(x: 0.0, y: 0.0)) # shared object, as today
15
- spawn(ps) # the callee's push is visible here
16
-
17
- view := ps.slice(1, 3) # []Particle: no copy, no allocation
12
+ ys := xs.slice(1, 3) # view: no copy
18
- view.set(0, p) # writes into ps's storage
13
+ ys.set(0, 20) # visible in xs (shared storage)
14
+ ys.push(9) # a view has cap == size, so this reallocates; xs is untouched
19
- for q := range view # fast loop over a view
15
+ zs := xs.sublist(1, 3) # copying snapshot, as today
20
- q.x
21
16
 
22
- w := "hello world".sub(6, 11) # "world": shares the bytes, O(1)
17
+ w := "hello world".sub(6, 11) # "world": shares the bytes
23
- b := List[Byte]() # was Buffer()
18
+ b := List[Byte]() # was Buffer()
24
19
  b.writeStr("hi ")
25
- t := b.toStr() # copies, so later writes to b don't change t
20
+ t := b.toStr() # copies, so later writes to b don't change t
26
21
  ```
27
22
 
28
23
  ## Layout
29
24
  ```mermaid
30
25
  flowchart LR
31
- L["List[Int] record (data, size)"] --> A["Array[Int]: 1 2 3 4 _ _"]
26
+ X["xs: List {data, start 0, size 5, cap 8}"] --> A["Array[Int]: 1 2 3 4 5 _ _ _"]
32
- V["view = ps.slice(1, 3): header (data, start=1, len=2), lives in locals"] --> A
27
+ Y["ys = xs.slice(1, 3): List {data, start 1, size 2, cap 2}"] --> A
33
28
  ```
34
- - `List[T]` stays `record List[T](data: Array[T], size: Int)`. Elements are unboxed: numbers in native arrays, structs as struct-of-arrays, records as refs.
29
+ - `record List[T](data: Array[T], start: Int, size: Int, cap: Int)`. Element `i` is `data[start + i]`.
30
+ - `push` writes in place while `size < cap`. Otherwise it grows: a new array of double the capacity, a copy of the live range, and `start` reset to 0.
31
+ - `slice(i, j)` returns `List(data, start + i, j - i, cap: j - i)`, so a view never writes past its own end.
35
- - `struct Slice[T](data: Array[T], start: Int, len: Int)`, with `[]T` as its type syntax. Passing and returning it costs three values, not an allocation.
32
+ - `record Str(data: Array[Byte], start: Int, len: Int)`. A substring is a new `Str` record sharing `data`.
36
- - `Array[Byte]` becomes packed `array<i8>` (it's `array<i32>` today), so `[]Byte`, `List[Byte]` and `Str` all share one compact byte layout.
33
+ - `Array[Byte]` becomes packed `array<i8>` (it's `array<i32>` today), so `List[Byte]` and `Str` share one compact byte layout.
37
34
 
38
35
  ## Rulings
39
- - **Appending to a sub-slice copies.** `[]T` has `append`, which returns the new header, and it is a compile error to drop the result. `slice(i, j)` sets capacity equal to length, so the first append on a view reallocates and never overwrites its parent's elements. Reading and `set` stay shared.
40
- - **`get` returns an error union:** `get(i) -> error{OutOfBounds}!T` on both `List` and `[]T`, so `xs.get(i)!`, `catch` and `try` keep working.
41
- - **No `xs[i]` / `xs[a:b]` syntax:** brackets mean generics. Element access is `get`/`set`, and slicing is `slice(i, j)` on a `List`, a `[]T` or a `Str`.
42
- - **`Str` becomes `struct Str(bytes: []Byte)`.**
43
- - `sub`, `slice`, `trim*` and `split` parts share bytes with the original.
44
- - `Str` has no setter. Every way from a mutable `List[Byte]` or `[]Byte` to a `Str` copies (`toStr`, `Str.fromBytes`), as Go's `string(b)` does.
45
- - Trade-off: a small substring keeps its big source alive. `s.clone()` detaches it.
46
- - **Equality:** `==` on `[]T` is a compile error (Go). Use `equals`. `Str ==` keeps comparing contents.
47
- - **Removals:** `Buffer` and `ByteSlice` are removed with no aliases. `List` keeps its whole API and gains `slice(i, j) -> []T`, while `sublist` stays as the copying form.
48
-
49
- ## Language work this needs
50
- 1. **Structs may hold references** (an `Array[T]`, a record, another struct with refs). This lifts today's "Int/Float/Bool/Byte/struct only" rule. A ref leaf is one more wasm local; in struct-of-arrays storage it is an array of refs.
51
- 2. **Generic structs** (`struct Slice[T]`), one copy per instantiation like every generic. `T`'s leaves decide the backing layout.
52
- 3. **Struct fields stored inline in records.** Today a struct field in a record is boxed (an extra allocation per store). Once `Str` is a struct, `record User(name: Str)` would pay that box on every construction. Inline means the record's wasm struct holds the struct's leaves directly.
53
- 4. **`[]T` type syntax** for any `T`. Today's fixed `[]Byte` builtin goes away.
36
+ - **`get` returns an error union:** `get(i) -> error{OutOfBounds}!T`, unchanged.
37
+ - **No `xs[i]` / `xs[a:b]` syntax:** brackets mean generics.
38
+ - **Shared-storage trap (same as Go):** `removeAt`, `sort`, `reverse` and `shuffle` on a list rearrange the elements its views see. `push` does not, because it writes past a view's end. Use `sublist` for a snapshot. Documented in the README.
39
+ - **`Str` is immutable,** so sharing is safe. Every path from mutable bytes (`List[Byte]`) to a `Str` copies.
40
+ - Trade-off: a small substring keeps its source's bytes alive. `s.clone()` detaches it.
41
+ - **Removals:** `Buffer` and `ByteSlice` are removed with no aliases.
54
42
 
55
43
  ## Work, in order
56
- 1. Structs with reference fields: checker rule, codegen leaves, ref arrays in struct-of-arrays, tests.
57
- 2. Generic structs: parser, checker and per-instantiation flattening.
44
+ 1. Pack `Array[Byte]` as `array<i8>`, with codegen and tests.
58
- 3. Struct fields stored inline in records, covering construction, field read and write, spread update, `==`, and host unwrap.
45
+ 2. Give `List` its `start`/`cap` fields and `slice(i, j)`. Copy-on-grow for views. Update every `List` method to index from `start`.
59
- 4. Packed `Array[Byte]`; `Slice[T]` plus the `[]T` syntax (grammar regenerated with `--disable-optimizations`); the must-use check on `append`; `List.slice`.
60
- 5. `Str` on `[]Byte` with zero-copy `sub`/`slice`/`trim`/`split`; update the host-boundary unwraps (cli, runtime, test harnesses).
46
+ 3. Make `Str` a `(data, start, len)` record, with zero-copy `sub`/`slice`/`trim`/`split`. Update the host-boundary unwraps (cli, runtime, test harnesses) and the codegen string-literal builder.
61
- 6. Replace `Buffer` with `List[Byte]` and `ByteSlice` with `[]Byte` across std (`Str`, `Json`, `Base64`, `Os`, `Http`, `Uuid`, ...), then the examples and README.
47
+ 4. Replace `Buffer` with `List[Byte]` and remove `ByteSlice` across std, then the examples and README.
62
- 7. Benchmarks before and after (particle loop, Json parse, string split and trim), then a final review.
48
+ 5. Benchmarks before and after (particle loop, Json parse, string split and trim), then a final review.
63
49
 
64
50
  ## Later performance work (no API change)
65
- - **Escape analysis:** a `List` that never leaves its function keeps `data` and `size` in locals, with no record allocation.
51
+ - **Escape analysis:** a `List` or view that never leaves its function keeps its fields in locals, with no allocation.
66
- - **Loops:** `for x := range list` reads `data` and `size` once before the loop.
52
+ - **Loops:** `for x := range list` reads `data`/`start`/`size` once before the loop.
67
53
  - **Inlining:** `get`, `set` and `push` are inlined.
68
54
 
69
55
  ## Risks
70
- - **Step 1** touches struct flattening everywhere (params, returns, `T?`, error unions, varargs, closures). **Step 3** touches every record access path. These two are the bulk of the work.
71
- - **`Str` as a struct** changes its host-boundary shape (cli, runtime and three test harnesses unwrap it by hand today).
56
+ - **The `Str` layout** is built by hand in codegen (string literals) and unwrapped by hand at the host boundary (cli, runtime, three test harnesses). Every place has to change together.
72
- - **Packed bytes:** `Array[Byte]` going from i32 to i8 changes the get and set codegen for byte arrays and needs its own tests.
57
+ - **Packed bytes:** `Array[Byte]` going from i32 to i8 changes byte-array get/set (`array.get_u`, wrapping on set).
73
- - **Closures capturing a `[]T`** capture the header by value, so an `append` inside doesn't reach the outer variable. A `List` captured by a closure keeps today's shared behavior.
58
+ - **`List` methods** that index `data` directly (`sort`, `reverse`, `removeAt`, `chunk`, ...) all have to add `start`. Missing one is a silent wrong-element bug, so each needs a test on a view.