From f7ac34141a5199a804ac4ac8f2d89f091a09682d Mon Sep 17 00:00:00 2001 From: Paul Lipscomb Date: Wed, 22 Jul 2026 18:55:42 -0400 Subject: [PATCH] Check in Claude session memory as CLAUDE_MEMORY.md, imported from CLAUDE.md Auto-memory is normally stored outside the repo, keyed to the working directory's absolute path, so it doesn't follow a clone to a new machine. This snapshot travels with the repo instead, so Claude has the same project/feedback context when opened from a different machine (e.g. the Windows VM being set up for the extension-reaper-macos port). Co-Authored-By: Claude Sonnet 5 --- CLAUDE.md | 2 + CLAUDE_MEMORY.md | 231 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 233 insertions(+) create mode 100644 CLAUDE_MEMORY.md diff --git a/CLAUDE.md b/CLAUDE.md index 1e55186..717a19c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,5 +1,7 @@ # Virtual Controller — Project Context +@CLAUDE_MEMORY.md + ## What this app is A PyQt6 desktop application that acts as a MIDI/OSC routing hub and control surface. It sits between hardware MIDI controllers (AKAI, JL Cooper motorized fader bank, iPad via Network MIDI) and a DAW (REAPER). It translates incoming CC numbers to different output CC numbers and channels, applies momentary/toggle logic, and manages named presets per instrument/VSTi. diff --git a/CLAUDE_MEMORY.md b/CLAUDE_MEMORY.md new file mode 100644 index 0000000..a298c49 --- /dev/null +++ b/CLAUDE_MEMORY.md @@ -0,0 +1,231 @@ +# Carried-over session memory + +This file is a portable dump of Claude's per-machine auto-memory for this +project (normally stored outside the repo, keyed to the absolute path of the +working directory — so it doesn't survive a clone to a new machine on its +own). This copy is checked into the repo so it travels with `git clone`/`git +pull` instead of staying stuck on one machine. + +Auto-generated 2026-07-22. May drift from live memory over time — treat as a +snapshot, not a live sync. + +--- + +## How Paul wants to collaborate (feedback) + +### Confirm before changes +Always talk through the diagnosis/plan and get explicit confirmation before +making code changes — do not implement on the first pass, even for what +looks like a clear-cut bug. + +- Stated explicitly (2026-07-06): "from here on out, always talk to me + before we make changes." A standing rule, not one-off. +- Stating a plan and executing it *in the same turn* does not count as + confirming — waiting for an actual reply is required, especially for + open/free-choice sub-decisions (a button label, a variable name, which of + several equally-valid options to pick) that come up mid-task. +- If Paul says something like "just do it" for a specific request, that + authorizes that one instance, not a standing reversal of the rule. + +### Bullet-point replies +Explain diagnoses/what's-happening in short, clear bullet points — not +paragraph prose. + +- Said explicitly during iOS FocusToggleWidget color debugging: "I'm going + to need you to start simplifying your discussions... respond to me in + clear bullet points about what's actually happening." +- Recurrence: bullets with bold headers still drifted into multi-sentence + paragraphs underneath. The failure mode isn't lack of bullets, it's + bullets containing paragraph-length reasoning. One short sentence per + bullet, no exceptions — split into more, shorter bullets instead of + letting one run long. +- Second recurrence: when Paul states the correct technical answer himself, + respond with a short confirmation ("yes, correct" + at most one + clarifying detail) — don't re-explain it back to him in different words. + Treat his own correct restatement as a signal the concept has landed. + +### ReaLearn is reference-only +`plugin-reaper-relearn` (vendored helgoboss/helgobox ReaLearn) is for +reading/insight only — never edit, patch, or extend it directly. + +- Paul: "the realearn stuff is its own project and we dont mod it at all... + if we need to read from it to gain some insight ok other than that no." +- All new proprietary REAPER integration work happens in + `extension-reaper-macos` instead. +- Useful reference patterns inside ReaLearn: `main/src/domain/targets/*.rs` + target-plugin pattern, `mouse_target.rs` OS-level mouse emulation via + `enigo`, `reaper-high`/`reaper-medium` (reaper-rs) usage — studied, not + depended on. + +--- + +## Ongoing project context + +### extension-reaper-macos / envelope automation pipeline +Paul is building `extension-reaper-macos` as his own proprietary, owned +compiled REAPER extension — not scoped to one feature. + +- Language: C/C++ against the real Cockos SDK directly (Rust/reaper-rs was + floated and explicitly dropped: "REAPER's actual native SDK... no wrapper + layer, no crate dependency graph"). +- Layout: `vendor/reaper-sdk/sdk/` (unmodified headers from + justinfrankel/reaper-sdk), `vendor/reaper-sdk/WDL/` (full justinfrankel/WDL + checkout, header types only — nothing compiled/linked from it), + `src/main.cpp` (entrypoint, owns `REAPERAPI_IMPLEMENT`), one + subfolder per module (`hello/`, `tracking/`, `socket/`). +- Confirmed working 2026-07-15: bare-minimum skeleton built, installed, + loads in REAPER, prints console confirmation. +- Since then: extended to read/write automation items (`D_BASELINE` nudge, + `tracking.cpp`), and a UDP socket listener (`socket.cpp`) wired to the + desktop app so fader moves nudge the selected automation item's baseline + live. + +**Origin feature:** click an automation item on a REAPER envelope lane to +select it, then move a fader in the desktop app (eventually the tablet +remote) to raise/lower ALL envelope points in that item together — maps to +REAPER's own `D_BASELINE` property (same as ctrl-drag natively), not +per-point editing. + +**Design decisions:** +- Mouse-emulation (replaying a physical Y-axis drag) was explicitly + rejected in favor of calling REAPER's API directly. +- This extension owns its own socket listener rather than going through + REAPER's OSC-control-surface config or ReaScript/Action-List-learn. +- Open question, unresolved: does this extension eventually *replace* the + existing OSC in/out to REAPER (track/preset name, transport, position via + `osc_sender.py`/`osc_receiver.py`), or live alongside it for things OSC + can't reach? +- Also undecided: relative (nudge) vs absolute fader mapping for baseline + value. + +**Cross-platform build (this is the live thread — Windows port in +progress):** +- macOS: `.dylib`, `clang++ -dynamiclib -arch arm64` via `build.sh`. Needs a + real Mac — can't be reliably/legally cross-compiled from another host. + Output now goes to `build/macos/`, copied to + `~/Library/Application Support/REAPER/UserPlugins/`. +- Windows: needs a `.dll`. `main.cpp`/`tracking.cpp` are portable as-is (pure + REAPER SDK C++, and `REAPER_PLUGIN_DLL_EXPORT` is already `_WIN32`-aware in + the vendored SDK header). `socket.cpp` is the one file that needs a real + port — it uses raw POSIX sockets + (`sys/socket.h`/`netinet/in.h`/`unistd.h`/`fcntl.h`, + `recvfrom`/`fcntl`/`close`) which don't exist on Windows; needs a Winsock2 + path (`WSAStartup`/`WSACleanup`, `closesocket`, `ioctlsocket` instead of + `fcntl`, link `ws2_32`). Also need a Windows-side build script (MSVC + `cl.exe /LD` or MinGW `g++ -shared`) producing a `.dll`, copied to + `%APPDATA%\REAPER\UserPlugins\`. +- Dev environment: Paul is setting up a Windows ARM64 VM (Parallels, Apple + Silicon host) rather than full x86 emulation (tried QEMU/UTM full x86 + emulation first — extremely slow, no hardware acceleration for a foreign + instruction set). ARM64 Windows runs natively/fast on Apple Silicon; + x64 compiler + x64 REAPER run inside it via Windows' own built-in + x64-on-ARM app emulation, which is fast enough for real dev work. REAPER + also ships a native ARM64 Windows build if x64 ends up unnecessary. +- A self-hosted Gitea instance (separate box) was floated for CI: Linux + `.so` natively there, Windows `.dll` via `x86_64-w64-mingw32-g++` + cross-compile from that same Linux box — viable while the extension has + no REAPER C++ virtual-class interfaces crossing the plugin boundary + (re-check if things like `PCM_source`/`ProjectStateContext` get used, + where the SDK's MSVC-ABI requirement actually bites). + +### Hardware CC binding backlog +Backlog for the hw_cc/hw_channel binding system in `main.py`, +`faderwidget.py`, `focusfaderwidget.py`, `focustogglewidget.py`: + +1. **Assignment collision detection** — no validation today for two widgets + (faders/toggles/transports/preset_browsers/focus_faders/focus_toggles) + being bound to the same hw_cc + hw_channel. Not yet designed — open + question is where the check should live (Learn-time vs. preset-wide + audit) and what UI signals it. + +Raised 2026-07-06 after building JL Cooper touch-sense support (CH15/CH16 +same-CC convention) and Focus widget Hardware/Feedback checkboxes — those +made binding conflicts more likely without adding detection. + +### Planned features backlog (2026-06-29, ordered) +1. Momentary on Toggle Widget — Mom/Tog mode on ToggleWidget, mirrors + TransportWidget +2. Add Rows — add new fader/toggle rows to layout at runtime +3. Duplicate (Create New Gen ID) — duplicate a widget/row with a fresh + generated ID +4. Big Title — large display title element +5. Enable/Disable Preset Browser Row +6. Re-order Rows — drag or button-based row reordering + +--- + +## iOS remote client (companion project, same session lineage) + +These entries came out of sessions rooted in this project directory but +concern `remote-client-ios` / `virtual-controller-ios-app`. Kept here since +that's where they were captured. + +### Tablet direction (confirmed) +The WebSocket + vanilla-JS web surface is the **permanent** direction for +performance control — not TouchOSC, not a native app. POC validated the +workflow: one-time arrange per preset, auto-switches with preset, instant JS +load, works on any device browser. + +Stack: PyQt6 app owns all routing/presets/MIDI/OSC; `QWebSocketServer` on +:8765; HTTP server on :8080 serving `/client`; vanilla JS client, no +frameworks; layout saved per preset UUID in `presets.json` under +`"tablet_layout"`. Native app (Android/iOS) explicitly off the table unless +revisited. + +### iOS FaderWidget architecture (settled, do not revisit) +Rebuilt from scratch using raw `touchesBegan`/`touchesMoved` overrides +instead of UISlider or UIScrollView — UISlider needs precise thumb +targeting, UIScrollView's pan gesture recognizer fought a drag gesture +recognizer (`touchesCancelled` froze the fader after one increment). Raw +touch tracking fires at 120hz with zero latency and works anywhere on the +widget surface. + +Key decisions: +- Value mapping: `value = (1 - pt.y / activeHeight) * 127`, normalized + against `bounds.height`. +- Dead zones: 90pt top/bottom (`padHeight = 90`). +- REL mode (default): drag relative to touch start, no value jump. ABS + mode: touch position maps directly to value. +- Gradient fill: `CAGradientLayer`, transparent black (bottom) to color at + 45% alpha (top), grows upward with value. +- Haptics: `UIImpactFeedbackGenerator(.medium)` on hitting 0 or 127 + (edge-detected via `lastHapticValue`). +- Dedup via `lastSentValue`. +- Gesture fix: `addDragGesture` sets `cancelsTouchesInView = false` so pan + doesn't steal touches; `arrangeMode` flag guards touch handlers during + drag; pinch-to-resize removed for faders. + +### iOS remote backlog +**Next up:** review pass on the `dest_id` system — Paul flagged things to +go over, "kinda works, things to go over." + +Shipped: `remote_checkbox`/`dest_spin` per widget (own `grp_remote` frame), +Bonjour discovery (desktop advertises `_vcremote._tcp.local.` via +`zeroconf`, iOS browses via `NetServiceBrowser`), Feedback widget (Channel +Focus-scoped, not the old global one), `ToggleWidget` off-state now uses +`color.darkened(by: 0.6)` instead of a hardcoded gray, pinch-to-resize + +grid-snap in arrange mode. + +Remaining: +1. **Bonjour discovery bug** — desktop-side confirmed correct (registers, + resolves, correct IP, Local Network permission on, same Wi-Fi), but never + resolves from a *borrowed* test iPad — possibly MDM/managed-device or + router client isolation. **Retest on Paul's own iPad before assuming the + code is broken.** +2. Review pass on dest_id system (see "Next up" above). +3. All-edge snap — `snapFrame` written but unused; call sites still use + `snapValue` on origin only. +4. Preset save bug — first/last presets not persisting. +5. Model all iOS JSON transactions (outer envelope still raw + `JSONSerialization`). +6. Model all Python outgoing data — dataclasses in `ws_server.py` + broadcasts. +7. Clean up `layout` arg — `broadcast_preset` takes it but doesn't send it. +8. Initial sync/ack handshake on connect. +9. Disallow presets without UUID (enforce server-side before broadcast). +10. Desktop activity ring — flash on desktop widget when it receives a + value from iPad. +11. **No live color-sync broadcast** (deprioritized) — color only travels + inside a full `"preset"` snapshot, never standalone like + label/value/feedback. Regressed at some point per Paul; not worth + chasing until other things are done.