Check in Claude session memory as CLAUDE_MEMORY.md, imported from CLAUDE.md
Auto-memory is normally stored outside the repo, keyed to the working directory's absolute path, so it doesn't follow a clone to a new machine. This snapshot travels with the repo instead, so Claude has the same project/feedback context when opened from a different machine (e.g. the Windows VM being set up for the extension-reaper-macos port). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Paul Lipscomb
@@ -1,5 +1,7 @@
|
||||
# Virtual Controller — Project Context
|
||||
|
||||
@CLAUDE_MEMORY.md
|
||||
|
||||
## What this app is
|
||||
|
||||
A PyQt6 desktop application that acts as a MIDI/OSC routing hub and control surface. It sits between hardware MIDI controllers (AKAI, JL Cooper motorized fader bank, iPad via Network MIDI) and a DAW (REAPER). It translates incoming CC numbers to different output CC numbers and channels, applies momentary/toggle logic, and manages named presets per instrument/VSTi.
|
||||
|
||||
@@ -0,0 +1,231 @@
|
||||
# Carried-over session memory
|
||||
|
||||
This file is a portable dump of Claude's per-machine auto-memory for this
|
||||
project (normally stored outside the repo, keyed to the absolute path of the
|
||||
working directory — so it doesn't survive a clone to a new machine on its
|
||||
own). This copy is checked into the repo so it travels with `git clone`/`git
|
||||
pull` instead of staying stuck on one machine.
|
||||
|
||||
Auto-generated 2026-07-22. May drift from live memory over time — treat as a
|
||||
snapshot, not a live sync.
|
||||
|
||||
---
|
||||
|
||||
## How Paul wants to collaborate (feedback)
|
||||
|
||||
### Confirm before changes
|
||||
Always talk through the diagnosis/plan and get explicit confirmation before
|
||||
making code changes — do not implement on the first pass, even for what
|
||||
looks like a clear-cut bug.
|
||||
|
||||
- Stated explicitly (2026-07-06): "from here on out, always talk to me
|
||||
before we make changes." A standing rule, not one-off.
|
||||
- Stating a plan and executing it *in the same turn* does not count as
|
||||
confirming — waiting for an actual reply is required, especially for
|
||||
open/free-choice sub-decisions (a button label, a variable name, which of
|
||||
several equally-valid options to pick) that come up mid-task.
|
||||
- If Paul says something like "just do it" for a specific request, that
|
||||
authorizes that one instance, not a standing reversal of the rule.
|
||||
|
||||
### Bullet-point replies
|
||||
Explain diagnoses/what's-happening in short, clear bullet points — not
|
||||
paragraph prose.
|
||||
|
||||
- Said explicitly during iOS FocusToggleWidget color debugging: "I'm going
|
||||
to need you to start simplifying your discussions... respond to me in
|
||||
clear bullet points about what's actually happening."
|
||||
- Recurrence: bullets with bold headers still drifted into multi-sentence
|
||||
paragraphs underneath. The failure mode isn't lack of bullets, it's
|
||||
bullets containing paragraph-length reasoning. One short sentence per
|
||||
bullet, no exceptions — split into more, shorter bullets instead of
|
||||
letting one run long.
|
||||
- Second recurrence: when Paul states the correct technical answer himself,
|
||||
respond with a short confirmation ("yes, correct" + at most one
|
||||
clarifying detail) — don't re-explain it back to him in different words.
|
||||
Treat his own correct restatement as a signal the concept has landed.
|
||||
|
||||
### ReaLearn is reference-only
|
||||
`plugin-reaper-relearn` (vendored helgoboss/helgobox ReaLearn) is for
|
||||
reading/insight only — never edit, patch, or extend it directly.
|
||||
|
||||
- Paul: "the realearn stuff is its own project and we dont mod it at all...
|
||||
if we need to read from it to gain some insight ok other than that no."
|
||||
- All new proprietary REAPER integration work happens in
|
||||
`extension-reaper-macos` instead.
|
||||
- Useful reference patterns inside ReaLearn: `main/src/domain/targets/*.rs`
|
||||
target-plugin pattern, `mouse_target.rs` OS-level mouse emulation via
|
||||
`enigo`, `reaper-high`/`reaper-medium` (reaper-rs) usage — studied, not
|
||||
depended on.
|
||||
|
||||
---
|
||||
|
||||
## Ongoing project context
|
||||
|
||||
### extension-reaper-macos / envelope automation pipeline
|
||||
Paul is building `extension-reaper-macos` as his own proprietary, owned
|
||||
compiled REAPER extension — not scoped to one feature.
|
||||
|
||||
- Language: C/C++ against the real Cockos SDK directly (Rust/reaper-rs was
|
||||
floated and explicitly dropped: "REAPER's actual native SDK... no wrapper
|
||||
layer, no crate dependency graph").
|
||||
- Layout: `vendor/reaper-sdk/sdk/` (unmodified headers from
|
||||
justinfrankel/reaper-sdk), `vendor/reaper-sdk/WDL/` (full justinfrankel/WDL
|
||||
checkout, header types only — nothing compiled/linked from it),
|
||||
`src/main.cpp` (entrypoint, owns `REAPERAPI_IMPLEMENT`), one
|
||||
subfolder per module (`hello/`, `tracking/`, `socket/`).
|
||||
- Confirmed working 2026-07-15: bare-minimum skeleton built, installed,
|
||||
loads in REAPER, prints console confirmation.
|
||||
- Since then: extended to read/write automation items (`D_BASELINE` nudge,
|
||||
`tracking.cpp`), and a UDP socket listener (`socket.cpp`) wired to the
|
||||
desktop app so fader moves nudge the selected automation item's baseline
|
||||
live.
|
||||
|
||||
**Origin feature:** click an automation item on a REAPER envelope lane to
|
||||
select it, then move a fader in the desktop app (eventually the tablet
|
||||
remote) to raise/lower ALL envelope points in that item together — maps to
|
||||
REAPER's own `D_BASELINE` property (same as ctrl-drag natively), not
|
||||
per-point editing.
|
||||
|
||||
**Design decisions:**
|
||||
- Mouse-emulation (replaying a physical Y-axis drag) was explicitly
|
||||
rejected in favor of calling REAPER's API directly.
|
||||
- This extension owns its own socket listener rather than going through
|
||||
REAPER's OSC-control-surface config or ReaScript/Action-List-learn.
|
||||
- Open question, unresolved: does this extension eventually *replace* the
|
||||
existing OSC in/out to REAPER (track/preset name, transport, position via
|
||||
`osc_sender.py`/`osc_receiver.py`), or live alongside it for things OSC
|
||||
can't reach?
|
||||
- Also undecided: relative (nudge) vs absolute fader mapping for baseline
|
||||
value.
|
||||
|
||||
**Cross-platform build (this is the live thread — Windows port in
|
||||
progress):**
|
||||
- macOS: `.dylib`, `clang++ -dynamiclib -arch arm64` via `build.sh`. Needs a
|
||||
real Mac — can't be reliably/legally cross-compiled from another host.
|
||||
Output now goes to `build/macos/`, copied to
|
||||
`~/Library/Application Support/REAPER/UserPlugins/`.
|
||||
- Windows: needs a `.dll`. `main.cpp`/`tracking.cpp` are portable as-is (pure
|
||||
REAPER SDK C++, and `REAPER_PLUGIN_DLL_EXPORT` is already `_WIN32`-aware in
|
||||
the vendored SDK header). `socket.cpp` is the one file that needs a real
|
||||
port — it uses raw POSIX sockets
|
||||
(`sys/socket.h`/`netinet/in.h`/`unistd.h`/`fcntl.h`,
|
||||
`recvfrom`/`fcntl`/`close`) which don't exist on Windows; needs a Winsock2
|
||||
path (`WSAStartup`/`WSACleanup`, `closesocket`, `ioctlsocket` instead of
|
||||
`fcntl`, link `ws2_32`). Also need a Windows-side build script (MSVC
|
||||
`cl.exe /LD` or MinGW `g++ -shared`) producing a `.dll`, copied to
|
||||
`%APPDATA%\REAPER\UserPlugins\`.
|
||||
- Dev environment: Paul is setting up a Windows ARM64 VM (Parallels, Apple
|
||||
Silicon host) rather than full x86 emulation (tried QEMU/UTM full x86
|
||||
emulation first — extremely slow, no hardware acceleration for a foreign
|
||||
instruction set). ARM64 Windows runs natively/fast on Apple Silicon;
|
||||
x64 compiler + x64 REAPER run inside it via Windows' own built-in
|
||||
x64-on-ARM app emulation, which is fast enough for real dev work. REAPER
|
||||
also ships a native ARM64 Windows build if x64 ends up unnecessary.
|
||||
- A self-hosted Gitea instance (separate box) was floated for CI: Linux
|
||||
`.so` natively there, Windows `.dll` via `x86_64-w64-mingw32-g++`
|
||||
cross-compile from that same Linux box — viable while the extension has
|
||||
no REAPER C++ virtual-class interfaces crossing the plugin boundary
|
||||
(re-check if things like `PCM_source`/`ProjectStateContext` get used,
|
||||
where the SDK's MSVC-ABI requirement actually bites).
|
||||
|
||||
### Hardware CC binding backlog
|
||||
Backlog for the hw_cc/hw_channel binding system in `main.py`,
|
||||
`faderwidget.py`, `focusfaderwidget.py`, `focustogglewidget.py`:
|
||||
|
||||
1. **Assignment collision detection** — no validation today for two widgets
|
||||
(faders/toggles/transports/preset_browsers/focus_faders/focus_toggles)
|
||||
being bound to the same hw_cc + hw_channel. Not yet designed — open
|
||||
question is where the check should live (Learn-time vs. preset-wide
|
||||
audit) and what UI signals it.
|
||||
|
||||
Raised 2026-07-06 after building JL Cooper touch-sense support (CH15/CH16
|
||||
same-CC convention) and Focus widget Hardware/Feedback checkboxes — those
|
||||
made binding conflicts more likely without adding detection.
|
||||
|
||||
### Planned features backlog (2026-06-29, ordered)
|
||||
1. Momentary on Toggle Widget — Mom/Tog mode on ToggleWidget, mirrors
|
||||
TransportWidget
|
||||
2. Add Rows — add new fader/toggle rows to layout at runtime
|
||||
3. Duplicate (Create New Gen ID) — duplicate a widget/row with a fresh
|
||||
generated ID
|
||||
4. Big Title — large display title element
|
||||
5. Enable/Disable Preset Browser Row
|
||||
6. Re-order Rows — drag or button-based row reordering
|
||||
|
||||
---
|
||||
|
||||
## iOS remote client (companion project, same session lineage)
|
||||
|
||||
These entries came out of sessions rooted in this project directory but
|
||||
concern `remote-client-ios` / `virtual-controller-ios-app`. Kept here since
|
||||
that's where they were captured.
|
||||
|
||||
### Tablet direction (confirmed)
|
||||
The WebSocket + vanilla-JS web surface is the **permanent** direction for
|
||||
performance control — not TouchOSC, not a native app. POC validated the
|
||||
workflow: one-time arrange per preset, auto-switches with preset, instant JS
|
||||
load, works on any device browser.
|
||||
|
||||
Stack: PyQt6 app owns all routing/presets/MIDI/OSC; `QWebSocketServer` on
|
||||
:8765; HTTP server on :8080 serving `/client`; vanilla JS client, no
|
||||
frameworks; layout saved per preset UUID in `presets.json` under
|
||||
`"tablet_layout"`. Native app (Android/iOS) explicitly off the table unless
|
||||
revisited.
|
||||
|
||||
### iOS FaderWidget architecture (settled, do not revisit)
|
||||
Rebuilt from scratch using raw `touchesBegan`/`touchesMoved` overrides
|
||||
instead of UISlider or UIScrollView — UISlider needs precise thumb
|
||||
targeting, UIScrollView's pan gesture recognizer fought a drag gesture
|
||||
recognizer (`touchesCancelled` froze the fader after one increment). Raw
|
||||
touch tracking fires at 120hz with zero latency and works anywhere on the
|
||||
widget surface.
|
||||
|
||||
Key decisions:
|
||||
- Value mapping: `value = (1 - pt.y / activeHeight) * 127`, normalized
|
||||
against `bounds.height`.
|
||||
- Dead zones: 90pt top/bottom (`padHeight = 90`).
|
||||
- REL mode (default): drag relative to touch start, no value jump. ABS
|
||||
mode: touch position maps directly to value.
|
||||
- Gradient fill: `CAGradientLayer`, transparent black (bottom) to color at
|
||||
45% alpha (top), grows upward with value.
|
||||
- Haptics: `UIImpactFeedbackGenerator(.medium)` on hitting 0 or 127
|
||||
(edge-detected via `lastHapticValue`).
|
||||
- Dedup via `lastSentValue`.
|
||||
- Gesture fix: `addDragGesture` sets `cancelsTouchesInView = false` so pan
|
||||
doesn't steal touches; `arrangeMode` flag guards touch handlers during
|
||||
drag; pinch-to-resize removed for faders.
|
||||
|
||||
### iOS remote backlog
|
||||
**Next up:** review pass on the `dest_id` system — Paul flagged things to
|
||||
go over, "kinda works, things to go over."
|
||||
|
||||
Shipped: `remote_checkbox`/`dest_spin` per widget (own `grp_remote` frame),
|
||||
Bonjour discovery (desktop advertises `_vcremote._tcp.local.` via
|
||||
`zeroconf`, iOS browses via `NetServiceBrowser`), Feedback widget (Channel
|
||||
Focus-scoped, not the old global one), `ToggleWidget` off-state now uses
|
||||
`color.darkened(by: 0.6)` instead of a hardcoded gray, pinch-to-resize +
|
||||
grid-snap in arrange mode.
|
||||
|
||||
Remaining:
|
||||
1. **Bonjour discovery bug** — desktop-side confirmed correct (registers,
|
||||
resolves, correct IP, Local Network permission on, same Wi-Fi), but never
|
||||
resolves from a *borrowed* test iPad — possibly MDM/managed-device or
|
||||
router client isolation. **Retest on Paul's own iPad before assuming the
|
||||
code is broken.**
|
||||
2. Review pass on dest_id system (see "Next up" above).
|
||||
3. All-edge snap — `snapFrame` written but unused; call sites still use
|
||||
`snapValue` on origin only.
|
||||
4. Preset save bug — first/last presets not persisting.
|
||||
5. Model all iOS JSON transactions (outer envelope still raw
|
||||
`JSONSerialization`).
|
||||
6. Model all Python outgoing data — dataclasses in `ws_server.py`
|
||||
broadcasts.
|
||||
7. Clean up `layout` arg — `broadcast_preset` takes it but doesn't send it.
|
||||
8. Initial sync/ack handshake on connect.
|
||||
9. Disallow presets without UUID (enforce server-side before broadcast).
|
||||
10. Desktop activity ring — flash on desktop widget when it receives a
|
||||
value from iPad.
|
||||
11. **No live color-sync broadcast** (deprioritized) — color only travels
|
||||
inside a full `"preset"` snapshot, never standalone like
|
||||
label/value/feedback. Regressed at some point per Paul; not worth
|
||||
chasing until other things are done.
|
||||
Reference in New Issue
Block a user