virtual-controller/CLAUDE_MEMORY.md

Carried-over session memory

This file is a portable dump of Claude's per-machine auto-memory for this project (normally stored outside the repo, keyed to the absolute path of the working directory — so it doesn't survive a clone to a new machine on its own). This copy is checked into the repo so it travels with git clone/git pull instead of staying stuck on one machine.

Auto-generated 2026-07-22. May drift from live memory over time — treat as a snapshot, not a live sync.

---

How Paul wants to collaborate (feedback)

Confirm before changes

Always talk through the diagnosis/plan and get explicit confirmation before making code changes — do not implement on the first pass, even for what looks like a clear-cut bug.

• Stated explicitly (2026-07-06): "from here on out, always talk to me before we make changes." A standing rule, not one-off.
• Stating a plan and executing it in the same turn does not count as confirming — waiting for an actual reply is required, especially for open/free-choice sub-decisions (a button label, a variable name, which of several equally-valid options to pick) that come up mid-task.
• If Paul says something like "just do it" for a specific request, that authorizes that one instance, not a standing reversal of the rule.

Bullet-point replies

Explain diagnoses/what's-happening in short, clear bullet points — not paragraph prose.

• Said explicitly during iOS FocusToggleWidget color debugging: "I'm going to need you to start simplifying your discussions... respond to me in clear bullet points about what's actually happening."
• Recurrence: bullets with bold headers still drifted into multi-sentence paragraphs underneath. The failure mode isn't lack of bullets, it's bullets containing paragraph-length reasoning. One short sentence per bullet, no exceptions — split into more, shorter bullets instead of letting one run long.
• Second recurrence: when Paul states the correct technical answer himself, respond with a short confirmation ("yes, correct" + at most one clarifying detail) — don't re-explain it back to him in different words. Treat his own correct restatement as a signal the concept has landed.

ReaLearn is reference-only

plugin-reaper-relearn (vendored helgoboss/helgobox ReaLearn) is for reading/insight only — never edit, patch, or extend it directly.

• Paul: "the realearn stuff is its own project and we dont mod it at all... if we need to read from it to gain some insight ok other than that no."
• All new proprietary REAPER integration work happens in extension-reaper-macos instead.
• Useful reference patterns inside ReaLearn: main/src/domain/targets/*.rs target-plugin pattern, mouse_target.rs OS-level mouse emulation via enigo, reaper-high/reaper-medium (reaper-rs) usage — studied, not depended on.

---

Ongoing project context

extension-reaper-macos / envelope automation pipeline

Paul is building extension-reaper-macos as his own proprietary, owned compiled REAPER extension — not scoped to one feature.

• Language: C/C++ against the real Cockos SDK directly (Rust/reaper-rs was floated and explicitly dropped: "REAPER's actual native SDK... no wrapper layer, no crate dependency graph").
• Layout: vendor/reaper-sdk/sdk/ (unmodified headers from justinfrankel/reaper-sdk), vendor/reaper-sdk/WDL/ (full justinfrankel/WDL checkout, header types only — nothing compiled/linked from it), src/main.cpp (entrypoint, owns REAPERAPI_IMPLEMENT), one subfolder per module (hello/, tracking/, socket/).
• Confirmed working 2026-07-15: bare-minimum skeleton built, installed, loads in REAPER, prints console confirmation.
• Since then: extended to read/write automation items (D_BASELINE nudge, tracking.cpp), and a UDP socket listener (socket.cpp) wired to the desktop app so fader moves nudge the selected automation item's baseline live.

Origin feature: click an automation item on a REAPER envelope lane to select it, then move a fader in the desktop app (eventually the tablet remote) to raise/lower ALL envelope points in that item together — maps to REAPER's own D_BASELINE property (same as ctrl-drag natively), not per-point editing.

Design decisions:

• Mouse-emulation (replaying a physical Y-axis drag) was explicitly rejected in favor of calling REAPER's API directly.
• This extension owns its own socket listener rather than going through REAPER's OSC-control-surface config or ReaScript/Action-List-learn.
• Open question, unresolved: does this extension eventually replace the existing OSC in/out to REAPER (track/preset name, transport, position via osc_sender.py/osc_receiver.py), or live alongside it for things OSC can't reach?
• Also undecided: relative (nudge) vs absolute fader mapping for baseline value.

Cross-platform build (this is the live thread — Windows port in progress):

• macOS: .dylib, clang++ -dynamiclib -arch arm64 via build.sh. Needs a real Mac — can't be reliably/legally cross-compiled from another host. Output now goes to build/macos/, copied to ~/Library/Application Support/REAPER/UserPlugins/.
• Windows: needs a .dll. main.cpp/tracking.cpp are portable as-is (pure REAPER SDK C++, and REAPER_PLUGIN_DLL_EXPORT is already _WIN32-aware in the vendored SDK header). socket.cpp is the one file that needs a real port — it uses raw POSIX sockets (sys/socket.h/netinet/in.h/unistd.h/fcntl.h, recvfrom/fcntl/close) which don't exist on Windows; needs a Winsock2 path (WSAStartup/WSACleanup, closesocket, ioctlsocket instead of fcntl, link ws2_32). Also need a Windows-side build script (MSVC cl.exe /LD or MinGW g++ -shared) producing a .dll, copied to %APPDATA%\REAPER\UserPlugins\.
• Dev environment: Paul is setting up a Windows ARM64 VM (Parallels, Apple Silicon host) rather than full x86 emulation (tried QEMU/UTM full x86 emulation first — extremely slow, no hardware acceleration for a foreign instruction set). ARM64 Windows runs natively/fast on Apple Silicon; x64 compiler + x64 REAPER run inside it via Windows' own built-in x64-on-ARM app emulation, which is fast enough for real dev work. REAPER also ships a native ARM64 Windows build if x64 ends up unnecessary.
• A self-hosted Gitea instance (separate box) was floated for CI: Linux .so natively there, Windows .dll via x86_64-w64-mingw32-g++ cross-compile from that same Linux box — viable while the extension has no REAPER C++ virtual-class interfaces crossing the plugin boundary (re-check if things like PCM_source/ProjectStateContext get used, where the SDK's MSVC-ABI requirement actually bites).

Hardware CC binding backlog

Backlog for the hw_cc/hw_channel binding system in main.py, faderwidget.py, focusfaderwidget.py, focustogglewidget.py:

1. Assignment collision detection — no validation today for two widgets (faders/toggles/transports/preset_browsers/focus_faders/focus_toggles) being bound to the same hw_cc + hw_channel. Not yet designed — open question is where the check should live (Learn-time vs. preset-wide audit) and what UI signals it.

Raised 2026-07-06 after building JL Cooper touch-sense support (CH15/CH16 same-CC convention) and Focus widget Hardware/Feedback checkboxes — those made binding conflicts more likely without adding detection.

Planned features backlog (2026-06-29, ordered)

1. Momentary on Toggle Widget — Mom/Tog mode on ToggleWidget, mirrors TransportWidget
2. Add Rows — add new fader/toggle rows to layout at runtime
3. Duplicate (Create New Gen ID) — duplicate a widget/row with a fresh generated ID
4. Big Title — large display title element
5. Enable/Disable Preset Browser Row
6. Re-order Rows — drag or button-based row reordering

---

iOS remote client (companion project, same session lineage)

These entries came out of sessions rooted in this project directory but concern remote-client-ios / virtual-controller-ios-app. Kept here since that's where they were captured.

Tablet direction (confirmed)

The WebSocket + vanilla-JS web surface is the permanent direction for performance control — not TouchOSC, not a native app. POC validated the workflow: one-time arrange per preset, auto-switches with preset, instant JS load, works on any device browser.

Stack: PyQt6 app owns all routing/presets/MIDI/OSC; QWebSocketServer on :8765; HTTP server on :8080 serving /client; vanilla JS client, no frameworks; layout saved per preset UUID in presets.json under "tablet_layout". Native app (Android/iOS) explicitly off the table unless revisited.

iOS FaderWidget architecture (settled, do not revisit)

Rebuilt from scratch using raw touchesBegan/touchesMoved overrides instead of UISlider or UIScrollView — UISlider needs precise thumb targeting, UIScrollView's pan gesture recognizer fought a drag gesture recognizer (touchesCancelled froze the fader after one increment). Raw touch tracking fires at 120hz with zero latency and works anywhere on the widget surface.

Key decisions:

• Value mapping: value = (1 - pt.y / activeHeight) * 127, normalized against bounds.height.
• Dead zones: 90pt top/bottom (padHeight = 90).
• REL mode (default): drag relative to touch start, no value jump. ABS mode: touch position maps directly to value.
• Gradient fill: CAGradientLayer, transparent black (bottom) to color at 45% alpha (top), grows upward with value.
• Haptics: UIImpactFeedbackGenerator(.medium) on hitting 0 or 127 (edge-detected via lastHapticValue).
• Dedup via lastSentValue.
• Gesture fix: addDragGesture sets cancelsTouchesInView = false so pan doesn't steal touches; arrangeMode flag guards touch handlers during drag; pinch-to-resize removed for faders.

iOS remote backlog

Next up: review pass on the dest_id system — Paul flagged things to go over, "kinda works, things to go over."

Shipped: remote_checkbox/dest_spin per widget (own grp_remote frame), Bonjour discovery (desktop advertises _vcremote._tcp.local. via zeroconf, iOS browses via NetServiceBrowser), Feedback widget (Channel Focus-scoped, not the old global one), ToggleWidget off-state now uses color.darkened(by: 0.6) instead of a hardcoded gray, pinch-to-resize + grid-snap in arrange mode.

Remaining:

1. Bonjour discovery bug — desktop-side confirmed correct (registers, resolves, correct IP, Local Network permission on, same Wi-Fi), but never resolves from a borrowed test iPad — possibly MDM/managed-device or router client isolation. Retest on Paul's own iPad before assuming the code is broken.
2. Review pass on dest_id system (see "Next up" above).
3. All-edge snap — snapFrame written but unused; call sites still use snapValue on origin only.
4. Preset save bug — first/last presets not persisting.
5. Model all iOS JSON transactions (outer envelope still raw JSONSerialization).
6. Model all Python outgoing data — dataclasses in ws_server.py broadcasts.
7. Clean up layout arg — broadcast_preset takes it but doesn't send it.
8. Initial sync/ack handshake on connect.
9. Disallow presets without UUID (enforce server-side before broadcast).
10. Desktop activity ring — flash on desktop widget when it receives a value from iPad.
11. No live color-sync broadcast (deprioritized) — color only travels inside a full "preset" snapshot, never standalone like label/value/feedback. Regressed at some point per Paul; not worth chasing until other things are done.