virtual-controller/CLAUDE_MEMORY.md

# Carried-over session memory

This file is a portable dump of Claude's per-machine auto-memory for this
project (normally stored outside the repo, keyed to the absolute path of the
working directory — so it doesn't survive a clone to a new machine on its
own). This copy is checked into the repo so it travels with `git clone`/`git
pull` instead of staying stuck on one machine.

Auto-generated 2026-07-22. May drift from live memory over time — treat as a
snapshot, not a live sync.

---

## How Paul wants to collaborate (feedback)

### Confirm before changes
Always talk through the diagnosis/plan and get explicit confirmation before
making code changes — do not implement on the first pass, even for what
looks like a clear-cut bug.

- Stated explicitly (2026-07-06): "from here on out, always talk to me
  before we make changes." A standing rule, not one-off.
- Stating a plan and executing it *in the same turn* does not count as
  confirming — waiting for an actual reply is required, especially for
  open/free-choice sub-decisions (a button label, a variable name, which of
  several equally-valid options to pick) that come up mid-task.
- If Paul says something like "just do it" for a specific request, that
  authorizes that one instance, not a standing reversal of the rule.

### Bullet-point replies
Explain diagnoses/what's-happening in short, clear bullet points — not
paragraph prose.

- Said explicitly during iOS FocusToggleWidget color debugging: "I'm going
  to need you to start simplifying your discussions... respond to me in
  clear bullet points about what's actually happening."
- Recurrence: bullets with bold headers still drifted into multi-sentence
  paragraphs underneath. The failure mode isn't lack of bullets, it's
  bullets containing paragraph-length reasoning. One short sentence per
  bullet, no exceptions — split into more, shorter bullets instead of
  letting one run long.
- Second recurrence: when Paul states the correct technical answer himself,
  respond with a short confirmation ("yes, correct" + at most one
  clarifying detail) — don't re-explain it back to him in different words.
  Treat his own correct restatement as a signal the concept has landed.

### ReaLearn is reference-only
`plugin-reaper-relearn` (vendored helgoboss/helgobox ReaLearn) is for
reading/insight only — never edit, patch, or extend it directly.

- Paul: "the realearn stuff is its own project and we dont mod it at all...
  if we need to read from it to gain some insight ok other than that no."
- All new proprietary REAPER integration work happens in
  `extension-reaper-macos` instead.
- Useful reference patterns inside ReaLearn: `main/src/domain/targets/*.rs`
  target-plugin pattern, `mouse_target.rs` OS-level mouse emulation via
  `enigo`, `reaper-high`/`reaper-medium` (reaper-rs) usage — studied, not
  depended on.

---

## Ongoing project context

### extension-reaper-macos / envelope automation pipeline
Paul is building `extension-reaper-macos` as his own proprietary, owned
compiled REAPER extension — not scoped to one feature.

- Language: C/C++ against the real Cockos SDK directly (Rust/reaper-rs was
  floated and explicitly dropped: "REAPER's actual native SDK... no wrapper
  layer, no crate dependency graph").
- Layout: `vendor/reaper-sdk/sdk/` (unmodified headers from
  justinfrankel/reaper-sdk), `vendor/reaper-sdk/WDL/` (full justinfrankel/WDL
  checkout, header types only — nothing compiled/linked from it),
  `src/main.cpp` (entrypoint, owns `REAPERAPI_IMPLEMENT`), one
  subfolder per module (`hello/`, `tracking/`, `socket/`).
- Confirmed working 2026-07-15: bare-minimum skeleton built, installed,
  loads in REAPER, prints console confirmation.
- Since then: extended to read/write automation items (`D_BASELINE` nudge,
  `tracking.cpp`), and a UDP socket listener (`socket.cpp`) wired to the
  desktop app so fader moves nudge the selected automation item's baseline
  live.

**Origin feature:** click an automation item on a REAPER envelope lane to
select it, then move a fader in the desktop app (eventually the tablet
remote) to raise/lower ALL envelope points in that item together — maps to
REAPER's own `D_BASELINE` property (same as ctrl-drag natively), not
per-point editing.

**Design decisions:**
- Mouse-emulation (replaying a physical Y-axis drag) was explicitly
  rejected in favor of calling REAPER's API directly.
- This extension owns its own socket listener rather than going through
  REAPER's OSC-control-surface config or ReaScript/Action-List-learn.
- Open question, unresolved: does this extension eventually *replace* the
  existing OSC in/out to REAPER (track/preset name, transport, position via
  `osc_sender.py`/`osc_receiver.py`), or live alongside it for things OSC
  can't reach?
- Also undecided: relative (nudge) vs absolute fader mapping for baseline
  value.

**Cross-platform build (this is the live thread — Windows port in
progress):**
- macOS: `.dylib`, `clang++ -dynamiclib -arch arm64` via `build.sh`. Needs a
  real Mac — can't be reliably/legally cross-compiled from another host.
  Output now goes to `build/macos/`, copied to
  `~/Library/Application Support/REAPER/UserPlugins/`.
- Windows: needs a `.dll`. `main.cpp`/`tracking.cpp` are portable as-is (pure
  REAPER SDK C++, and `REAPER_PLUGIN_DLL_EXPORT` is already `_WIN32`-aware in
  the vendored SDK header). `socket.cpp` is the one file that needs a real
  port — it uses raw POSIX sockets
  (`sys/socket.h`/`netinet/in.h`/`unistd.h`/`fcntl.h`,
  `recvfrom`/`fcntl`/`close`) which don't exist on Windows; needs a Winsock2
  path (`WSAStartup`/`WSACleanup`, `closesocket`, `ioctlsocket` instead of
  `fcntl`, link `ws2_32`). Also need a Windows-side build script (MSVC
  `cl.exe /LD` or MinGW `g++ -shared`) producing a `.dll`, copied to
  `%APPDATA%\REAPER\UserPlugins\`.
- Dev environment: Paul is setting up a Windows ARM64 VM (Parallels, Apple
  Silicon host) rather than full x86 emulation (tried QEMU/UTM full x86
  emulation first — extremely slow, no hardware acceleration for a foreign
  instruction set). ARM64 Windows runs natively/fast on Apple Silicon;
  x64 compiler + x64 REAPER run inside it via Windows' own built-in
  x64-on-ARM app emulation, which is fast enough for real dev work. REAPER
  also ships a native ARM64 Windows build if x64 ends up unnecessary.
- A self-hosted Gitea instance (separate box) was floated for CI: Linux
  `.so` natively there, Windows `.dll` via `x86_64-w64-mingw32-g++`
  cross-compile from that same Linux box — viable while the extension has
  no REAPER C++ virtual-class interfaces crossing the plugin boundary
  (re-check if things like `PCM_source`/`ProjectStateContext` get used,
  where the SDK's MSVC-ABI requirement actually bites).

### Hardware CC binding backlog
Backlog for the hw_cc/hw_channel binding system in `main.py`,
`faderwidget.py`, `focusfaderwidget.py`, `focustogglewidget.py`:

1. **Assignment collision detection** — no validation today for two widgets
   (faders/toggles/transports/preset_browsers/focus_faders/focus_toggles)
   being bound to the same hw_cc + hw_channel. Not yet designed — open
   question is where the check should live (Learn-time vs. preset-wide
   audit) and what UI signals it.

Raised 2026-07-06 after building JL Cooper touch-sense support (CH15/CH16
same-CC convention) and Focus widget Hardware/Feedback checkboxes — those
made binding conflicts more likely without adding detection.

### Planned features backlog (2026-06-29, ordered)
1. Momentary on Toggle Widget — Mom/Tog mode on ToggleWidget, mirrors
   TransportWidget
2. Add Rows — add new fader/toggle rows to layout at runtime
3. Duplicate (Create New Gen ID) — duplicate a widget/row with a fresh
   generated ID
4. Big Title — large display title element
5. Enable/Disable Preset Browser Row
6. Re-order Rows — drag or button-based row reordering

---

## iOS remote client (companion project, same session lineage)

These entries came out of sessions rooted in this project directory but
concern `remote-client-ios` / `virtual-controller-ios-app`. Kept here since
that's where they were captured.

### Tablet direction (confirmed)
The WebSocket + vanilla-JS web surface is the **permanent** direction for
performance control — not TouchOSC, not a native app. POC validated the
workflow: one-time arrange per preset, auto-switches with preset, instant JS
load, works on any device browser.

Stack: PyQt6 app owns all routing/presets/MIDI/OSC; `QWebSocketServer` on
:8765; HTTP server on :8080 serving `/client`; vanilla JS client, no
frameworks; layout saved per preset UUID in `presets.json` under
`"tablet_layout"`. Native app (Android/iOS) explicitly off the table unless
revisited.

### iOS FaderWidget architecture (settled, do not revisit)
Rebuilt from scratch using raw `touchesBegan`/`touchesMoved` overrides
instead of UISlider or UIScrollView — UISlider needs precise thumb
targeting, UIScrollView's pan gesture recognizer fought a drag gesture
recognizer (`touchesCancelled` froze the fader after one increment). Raw
touch tracking fires at 120hz with zero latency and works anywhere on the
widget surface.

Key decisions:
- Value mapping: `value = (1 - pt.y / activeHeight) * 127`, normalized
  against `bounds.height`.
- Dead zones: 90pt top/bottom (`padHeight = 90`).
- REL mode (default): drag relative to touch start, no value jump. ABS
  mode: touch position maps directly to value.
- Gradient fill: `CAGradientLayer`, transparent black (bottom) to color at
  45% alpha (top), grows upward with value.
- Haptics: `UIImpactFeedbackGenerator(.medium)` on hitting 0 or 127
  (edge-detected via `lastHapticValue`).
- Dedup via `lastSentValue`.
- Gesture fix: `addDragGesture` sets `cancelsTouchesInView = false` so pan
  doesn't steal touches; `arrangeMode` flag guards touch handlers during
  drag; pinch-to-resize removed for faders.

### iOS remote backlog
**Next up:** review pass on the `dest_id` system — Paul flagged things to
go over, "kinda works, things to go over."

Shipped: `remote_checkbox`/`dest_spin` per widget (own `grp_remote` frame),
Bonjour discovery (desktop advertises `_vcremote._tcp.local.` via
`zeroconf`, iOS browses via `NetServiceBrowser`), Feedback widget (Channel
Focus-scoped, not the old global one), `ToggleWidget` off-state now uses
`color.darkened(by: 0.6)` instead of a hardcoded gray, pinch-to-resize +
grid-snap in arrange mode.

Remaining:
1. **Bonjour discovery bug** — desktop-side confirmed correct (registers,
   resolves, correct IP, Local Network permission on, same Wi-Fi), but never
   resolves from a *borrowed* test iPad — possibly MDM/managed-device or
   router client isolation. **Retest on Paul's own iPad before assuming the
   code is broken.**
2. Review pass on dest_id system (see "Next up" above).
3. All-edge snap — `snapFrame` written but unused; call sites still use
   `snapValue` on origin only.
4. Preset save bug — first/last presets not persisting.
5. Model all iOS JSON transactions (outer envelope still raw
   `JSONSerialization`).
6. Model all Python outgoing data — dataclasses in `ws_server.py`
   broadcasts.
7. Clean up `layout` arg — `broadcast_preset` takes it but doesn't send it.
8. Initial sync/ack handshake on connect.
9. Disallow presets without UUID (enforce server-side before broadcast).
10. Desktop activity ring — flash on desktop widget when it receives a
    value from iPad.
11. **No live color-sync broadcast** (deprioritized) — color only travels
    inside a full `"preset"` snapshot, never standalone like
    label/value/feedback. Regressed at some point per Paul; not worth
    chasing until other things are done.