Ship of Tools — Phase 1 plan (historical)

Status: partly built

This is the original phase-1 plan, kept as a record. The table below says which of its parts are built.

The plan names files, plugin packages and a TUI that were not built, and it was written before capsule rows replaced tmux sessions. Read it for the reasoning behind the milestones, not as a description of the code. What is built is described in the rest of this manual; the unbuilt parts are described, marked unbuilt, in Modes, The Concept Layer, Color Coding, The Dispatch ABI and Mode plugins. (These links resolve in the built manual, where this file is staged as design/roadmap.md; on GitHub they do not.)

Plan part	Status	What is true now
M1 native window: winit, wgpu, ratatui cells painted with cosmic-text	Built	WgpuBackend in rust/frontend/src/ui/render/cells.rs; the window is not a terminal program
M1 previews: PNG, markdown, MathJax SVG	Built	rust/frontend/src/ui/preview/image/, rust/frontend/src/ui/preview/markdown/, rust/backend/src/sidecars/mathjax.rs
M1 remote backend in a named tmux session, reconnect by revision	Partly built	Reconnect replays by last_seen_revision (rust/backend/src/server/hello.rs); the daemon starts no tmux sessions, rows are capsule rows
M2 Files mode with file watching	Built	rust/backend/src/files/watcher.rs sends preview.changed
M3 Modules mode, read-only	Built	project.scan in julia/kernel/src/project_scan.jl, tree in rust/frontend/src/ui/nav/modules.rs; no modules-mode plugin package exists
M4 persistent REPL with line, block and file dispatch	Partly built	REPL in rust/backend/src/sidecars/repl/ and julia/repl/; per-line and per-block dispatch is not built
M5 .concept/ annotations with AST-hash provenance	Partly built	concept.read, concept.write and concept.list exist; staleness compares a file hash, per-entity ast_hash is unused, one-key refresh is not built
M6 orchestrator chat with tools	Not built	no orchestrator code in rust/backend/src/; requirements.md marks it NOT BUILT
M7 HDF5Preview as an external plugin	Partly built	examples/plugins/HDF5Preview/ loads through LAZY_PLUGIN_FOR_EXT; nothing reads [sot].extensions
Types, Math and Outputs modes; the Agents mode	Not built	Modes

A dependency-ordered plan. The first milestone is a thin end-to-end skeleton; each subsequent milestone adds exactly one capability.

Note (2026-05-07): Plan revised after the C-pivot. The original plan assumed the frontend was a TUI rendered into the user's terminal. That premise was wrong for the user's deployment (Windows → SSH → tmux → Linux remote). The revised plan keeps the kernel, core IR, line protocol, and backend daemon intact but rebuilds the frontend as a native local window that owns rendering end-to-end.

Scope

The plan's scope:

• Files mode (filesystem nav with previews — markdown, PNG, syntax-highlighted .jl)

• Modules mode, read-only (structural via JuliaSyntax.jl)

• Persistent Julia REPL with code dispatch (line, block, file)

• Orchestrator LLM chat with file + REPL tools

• .concept/ annotation read+write with AST-hash provenance

• One demo external plugin (HDF5 preview), validating the ABI from outside core

Out: Types/Math/Outputs modes, multi-agent, multi-laptop semantics beyond first-wins, background staleness sweep, embedded editor, MP4 playback, automatic plot capture.

Risk register

Risk	De-risked in
Native rendering of markdown + PNG + MathJax-SVG end-to-end	M1 spike (the go/no-go gate)
ratatui custom backend (cell stream into wgpu) — prior art partial	M1 spike: try ratatui-wgpu first; roll our own if it can't carry our requirements
SSH Unix-socket forwarding on Windows OpenSSH	M1 spike: confirm StreamLocalBindUnlink + LocalForward path; TCP per-session port is the fallback
Reconnect restoring state without losing in-flight work	M1 spike: kill the frontend, restart, see kernel state and tree position resume
Cross-platform Julia subprocess supervision	M2 — verify no orphaned julia after Ctrl-C / taskkill / task-manager kill
AST hash stability across JuliaSyntax versions	M3 property test + pinned version + pinned hash format
JuliaSyntax parsing perf on large files	M3 cache by (path, mtime, size)
Anthropic streaming + tool-use loop	M6 — build against recorded fixture before live API

Milestones

M0 — Decisions and scaffolding (DONE — pivoted)

Status: core/, julia/kernel/, rust/protocol/, rust/backend/ scaffolds are committed and stay. rust/frontend/ as committed targeted crossterm; pruned in the M0 closeout (the ratatui-image / crossterm deps and rust/scratch/img-probe/ are removed).

M1 — Spike: native window + chrome + previews + remote socket + reconnect (1 week)

The go/no-go gate for option C. If this lands, C is defensible. If preview-layer requirements start pulling in HTML/iframes/WebGL within the spike, fall back to option B (Tauri webview wrapping a web frontend) — kernel/backend/protocol stay intact across the swap.

Deliverable: a single sot binary that:

1. Opens a native local window (winit + wgpu).

2. Runs a ratatui chrome layer (the mode tree, status bar, focus) via a custom Backend impl that paints cells with cosmic-text into the wgpu surface.

3. Renders a preview pane in three modes:

   • PNG via image → wgpu texture

   • Markdown via comrak → cosmic-text laid out lines

   • Inline math via MathJax-served SVG → resvg → wgpu texture

4. Connects to a backend instance running in a named tmux session on a remote host via SSH-forwarded per-session Unix socket; reconnects after frontend kill restoring tree position and last preview.

Files (sketch):

• rust/frontend/src/main.rs — winit event loop, wgpu surface init, connect-and-spawn flow.

• rust/frontend/src/ui/render/cells.rs — ratatui custom Backend impl.

• rust/frontend/src/ui/preview/mod.rs — preview-layer dispatch on MIME.

• rust/frontend/src/ui/preview/{image/png,markdown/mod,image/svg}.rs — three concrete renderers.

• rust/frontend/src/net/transport/mod.rs — SSH-spawn + Unix-socket connect + reconnect loop.

• rust/backend/src/main.rs — Unix-socket listener, session-id handshake, kernel supervisor.

• rust/backend/src/session.rs — session state (tree cursor, preview cache, revision counter for reconnect).

• rust/backend/src/sidecars/mathjax.rs — Node sidecar wrapper (math snippet → SVG).

• julia/kernel/src/ShipToolsKernel.jl — minimal NDJSON dispatch returning fixed payloads for the spike.

• rust/protocol/src/lib.rs — Request/Response/Event enums for the spike's ops only (hello, preview.get, tree.root).

Acceptance:

• Render PNG side-by-side with a real CairoMakie figure, eyeball quality.

• Render a markdown file with embedded LaTeX ($\int_0^\infty e^{-x^2}\,dx$); math appears typeset, not as a code box.

• Kill the frontend with the backend running; relaunch; tree cursor and preview restore from last_seen_revision.

• Run on Linux (local → local-loopback) and Windows (Windows local → Linux remote) end to end.

If any of those fail in a way the architecture can't fix, switch to option B before continuing.

M2 — Files mode populated, file watching (3–4 days)

Deliverable: real filesystem navigation. .md rendered, .png rendered, .jl shown as syntax-highlighted text. External edits update the tree within 500 ms.

Files:

• julia/plugins/files-mode/src/FilesMode.jl — tree_root, tree_children, preview_for.

• julia/plugins/markdown/src/MarkdownPreview.jl — preview(::Type{MarkdownDoc}, path) → PreviewPayload(mime="text/markdown", data=bytes).

• julia/plugins/png/src/PngPreview.jl — read bytes → image/png payload.

• julia/plugins/julia-source/src/JuliaSourcePreview.jl — read + syntect-friendly metadata → text/plain with extras.

• rust/backend/src/files/watcher.rs — notify watcher → file.changed events.

Acceptance: open the Ship of Tools repo, navigate to requirements.md, see it rendered. Drop a PNG, navigate, see it. Edit a file externally; tree refreshes.

M3 — Modules mode (read-only) + AST hash (3–4 days)

Deliverable: hotkey switches root tree from Files to Modules. Columns: modules → functions → methods. Preview shows method source.

Files:

• julia/plugins/julia-source/src/JuliaSourceParser.jl — parse_entities walks JuliaSyntax.parseall.

• julia/plugins/julia-source/src/Index.jl — project index keyed by (path, mtime, size); incremental rebuild on file.changed.

• julia/plugins/modules-mode/src/ModulesMode.jl — mode-tree dispatch.

• core/src/ASTHash.jl — hash function.

• core/test/test_ast_hash.jl — property test.

• rust/frontend/src/ui/nav/tree_store.rs — per-mode cursor preservation; modes switch via f / m / s / h in nav focus.

Acceptance: open the Ship of Tools repo, switch to Modules mode, navigate to a method, see source. Property test passes.

M4 — Persistent REPL with code dispatch (3–4 days)

Deliverable: toggleable REPL pane (~). <Enter> evaluates the line at cursor; <S-Enter> evaluates the surrounding top-level block; <C-Enter> includes the whole file. CairoMakie figures render inline through the preview-layer (NOT through any terminal protocol).

Shipped (diverged from the original plan): the REPL toggles with Ctrl+J; whole-file run landed as Files-mode r (fresh REPL) / R (current session); at the prompt Enter submits and Shift+Enter inserts a newline. Per-line / per-block dispatch is still planned.

Files:

• repl/Project.toml, repl/src/ShipToolsRepl.jl.

• repl/src/DisplayShim.jl — MIMEDisplay emits framed JSON.

• rust/backend/src/sidecars/repl/ — REPL supervisor; repl.eval streams repl.frame events tagged with eval_id.

• rust/frontend/src/repl_pane.rs — consumes repl.frame, dispatches to preview-layer for inline images.

• rust/frontend/src/dispatch.rs — block-span computation via kernel round-trip.

Acceptance: x = 1+1 → 2. Plot returns a wgpu texture inline. Killing REPL via UI doesn't kill kernel; restart works.

M5 — .concept/ annotations with AST-hash provenance (2–3 days)

Deliverable: Modules-mode preview shows source on top, matching .concept/ annotation below. Stale badge when AST hash mismatches. <S-e> opens annotation in $EDITOR. <r> refreshes provenance. (As built: the annotation is edited inline with e / Ctrl+S rather than via $EDITOR, and the one-key <r> provenance refresh was not implemented — see the concept layer guide.)

Files:

• julia/plugins/concept/src/ConceptStore.jl — read_annotation, write_annotation, is_stale. YAML frontmatter via YAML.jl.

• julia/plugins/concept/src/Provenance.jl — stamps synced_against and synced_at on write.

• julia/plugins/modules-mode/src/ModulesMode.jl — extend preview_for to compose source + annotation + badge.

• rust/frontend/src/preview/composite.rs — multi-pane preview composition (source on top, annotation below, badge overlay).

• rust/backend/src/editor.rs — shell out to $EDITOR, await exit, signal kernel re-read.

Acceptance: create a method, write annotation, save. Modify the body; reopen → annotation shows stale badge. <r> clears it. (As built, the badge is cleared by re-editing and re-saving the annotation, not by a dedicated <r> keypress.)

M6 — Orchestrator chat with tool dispatch (4–5 days)

Deliverable: chat pane (<Space>c). User types; orchestrator streams. Tools: read_file, write_file, read_annotation, write_annotation, repl_eval, list_modules. Demo: "summarize module X" → tool calls visible as collapsed breadcrumbs → optional annotation written.

Files:

• rust/backend/src/orchestrator/mod.rs — Anthropic client, SSE streaming, tool-use loop.

• rust/backend/src/orchestrator/tools.rs — tool spec construction.

• rust/backend/src/orchestrator/cache.rs — four-block cache scheme.

• rust/frontend/src/chat_pane.rs — input area, transcript, streaming render via cosmic-text, collapsed tool-call breadcrumbs.

• core/src/ToolSpec.jl — tool_spec and tool_call dispatch surfaces defined but unwired. Phase 2 seam.

Acceptance: ask the orchestrator to "create an annotation for function X." File appears with proper frontmatter; reopening Modules mode shows the new annotation.

M7 — Demo external plugin: HDF5Preview.jl (1–2 days)

Deliverable: separate package at examples/plugins/HDF5Preview/ that, when listed in a project's [sot].extensions, makes .h5 files previewable.

Files:

• examples/plugins/HDF5Preview/Project.toml — depends on ConceptExplorerCore, HDF5.

• examples/plugins/HDF5Preview/src/HDF5Preview.jl — struct H5File <: FileType end, dispatches preview.

• examples/plugins/README.md.

Acceptance: drop an .h5, add HDF5Preview to [sot].extensions, navigate, see structured preview. Zero Rust changes were required. That's the test of the ABI. (As built: the kernel loads HDF5Preview through LAZY_PLUGIN_FOR_EXT, and the daemon's is_bounded_output_plugin names .h5, .hdf5 and .hdf, which is a Rust change.)