feat(audio): per-app input device selection via the direct Core Audio path#642
Conversation
github-actions
Bot
added
needs PR template
|
The PR Policy check is blocking this PR because required template information is missing. Please update the PR description with:
Visual files detected:
Screenshots or video are required for UI, UX, settings, onboarding, overlay, menu bar, or visual behavior changes. If this PR has no visual changes, check the no-visual-change box in the template. If this remains incomplete for 48 hours after opening, the PR may be closed. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a9ddf78e92
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| guard self.experimentalDirectAudioCaptureEnabled else { return true } | ||
| return (self.defaults.object(forKey: Keys.syncAudioDevicesWithSystem) as? Bool) ?? true |
There was a problem hiding this comment.
When independent mode is enabled and the selected microphone is unplugged, the existing device-refresh paths replace the saved preference with the system default: ContentView.swift:401-404 writes the fallback UID directly, and SettingsView.swift:1156-1169 changes the picker to the default, whose handler at line 1149 persists it. The original UID is therefore lost, so the advertised auto-switch on reconnect cannot occur; fallback should leave preferredInputDeviceUID intact.
Useful? React with 👍 / 👎.
| guard SettingsStore.shared.syncAudioDevicesWithSystem == false, | ||
| SettingsStore.shared.experimentalDirectAudioCaptureEnabled == false else { |
There was a problem hiding this comment.
With Direct Audio Capture enabled and syncing disabled, choosing an output device only saves preferredOutputDeviceUID because the picker skips changing the system default, while this guard also skips binding that preferred output. The picker consequently displays a selection that has no effect and audio continues through the macOS default; either output should remain synchronized independently of the input-only mode, or its picker should follow and clearly expose the actual default.
Useful? React with 👍 / 👎.
| Toggle("", isOn: Binding( | ||
| get: { SettingsStore.shared.syncAudioDevicesWithSystem }, | ||
| set: { SettingsStore.shared.syncAudioDevicesWithSystem = $0 } |
There was a problem hiding this comment.
If a user selects a dedicated microphone while syncing is off and then turns this toggle back on, the setter only changes the persisted Boolean. resolvedInputDeviceForCapture() immediately switches to the macOS default, but selectedInputUID remains on the former preferred device until an unrelated hardware notification, so Settings shows one microphone while the next recording uses another; the toggle-on transition should update the picker state to the current system defaults.
Useful? React with 👍 / 👎.
Greptile SummaryThis PR re-introduces per-app input device selection by routing exclusively through
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
a9ddf78 to
490868f
Compare
github-actions
Bot
removed
needs PR template
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 490868f4ba
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| SettingsStore.shared.preferredOutputDeviceUID = newUID | ||
| // Only change system default if sync is enabled | ||
| if SettingsStore.shared.syncAudioDevicesWithSystem { | ||
| _ = AudioDevice.setDefaultOutputDevice(uid: newUID) | ||
| } | ||
| // Independent mode is input-only (direct capture has no output | ||
| // path), so selecting an output always updates the macOS system | ||
| // default — otherwise the picker would have no effect. | ||
| _ = AudioDevice.setDefaultOutputDevice(uid: newUID) |
There was a problem hiding this comment.
When syncing is disabled, output remains system-controlled—this handler always changes the macOS default—but ContentView's independent-mode refresh at lines 408-415 and startup initialization at line 4342 continue restoring preferredOutputDeviceUID. Consequently, after the user changes output in Control Center while independent input mode is active, FluidVoice keeps displaying the previous output even though audio uses the new system default. The output selection should follow external default-output changes regardless of this input-only mode.
Useful? React with 👍 / 👎.
| guard self.experimentalDirectAudioCaptureEnabled else { return true } | ||
| return (self.defaults.object(forKey: Keys.syncAudioDevicesWithSystem) as? Bool) ?? true |
There was a problem hiding this comment.
Once this getter permits independent mode, selecting a microphone from the menu bar persists the dedicated UID without changing the system default (MenuBarManager.selectMicrophone), but currentPreferredInputUID in that same file still unconditionally returns the default UID. Reopening the menu therefore checkmarks the system microphone rather than the microphone FluidVoice will capture, making the primary menu-bar UI report the wrong active device whenever syncing is off.
Useful? React with 👍 / 👎.
| return self.inputDevices.first { $0.name == self.cachedDefaultInputName }?.uid | ||
| ?? self.selectedInputUID |
There was a problem hiding this comment.
When two attached input devices have the same display name—for example, two microphones of the same model—this lookup always returns the first matching UID, which need not be the actual system default. In synchronized mode the picker can therefore checkmark one device while capture uses the other; cache or otherwise retain the default device UID instead of deriving it from the non-unique name.
Useful? React with 👍 / 👎.
…dio capture Per-app input selection was removed because binding a non-default device through AVAudioEngine's inputNode fails on macOS with kAudioUnitErr_InvalidPropertyValue (-10851), notably for aggregate/Bluetooth devices (see the syncAudioDevicesWithSystem comment and the note at the bindPreferred* call site). But the direct Core Audio capture path added since then (DirectCoreAudioInput -> CoreAudioCaptureSupport.c, using AudioDeviceCreateIOProcID/AudioDeviceStart on a specific device) binds an arbitrary input device per-app WITHOUT touching the system default and WITHOUT the AVAudioEngine limitation. resolvedInputDeviceForCapture() already returns the preferred device when sync is disabled — the feature was only gated off by syncAudioDevicesWithSystem being hardcoded to true. This change: - Restores syncAudioDevicesWithSystem as a real, persisted setting (default true for backward compatibility), gated on Direct Audio Capture being enabled so independent selection only ever rides the safe IOProc path. - Keeps the AVAudioEngine compatibility path pinned to the system default input/output whenever Direct Audio Capture is on, so the -10851 bind is never attempted (only in the legacy direct-capture-off configuration). - Fixes the Settings input-device picker to reflect the saved preferred device in independent mode (it previously always showed the system default at startup/reload, while the output picker already restored its preference) - capture was already correct; this fixes the display. - Adds a "Sync devices with macOS" toggle to Audio Devices settings (shown when Direct Audio Capture is on). With sync disabled, choosing an input device stores it as the preference and captures from it directly; when unplugged, capture falls back to the system default and auto-switches back on reconnect (existing behavior). Compiles clean via xcodebuild (Debug, macOS). Validated end-to-end on hardware: USB handset, generic Bluetooth (Shokz, HFP), and AirPods (AAC-ELD) each captured per-app with the system default left on the built-in mic and zero -10851 / fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
490868f to
ba52b9d
Compare
|
Thanks for the thorough reviews. Addressed in Fixed
Already resolved by the picker refactor
Deferred (out of scope here)
Local: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba52b9ddeb
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if let defaultUID = AudioDevice.getDefaultInputDevice()?.uid, | ||
| self.inputDevices.contains(where: { $0.uid == defaultUID }) { |
There was a problem hiding this comment.
Whenever SwiftUI evaluates this picker, the binding getter now calls AudioDevice.getDefaultInputDevice(), which enumerates Core Audio devices via listAllDevices(). This same view explicitly caches device information because Core Audio calls during view-body evaluation can race with AttributeGraph and cause EXC_BAD_ACCESS (see the warning at SettingsView.swift:52-54); derive the selection from a cached default UID populated outside rendering instead.
Useful? React with 👍 / 👎.
| set { | ||
| // No-op: sync mode is always enabled | ||
| // Kept for backward compatibility but value is ignored | ||
| _ = newValue | ||
| objectWillChange.send() | ||
| self.defaults.set(newValue, forKey: Keys.syncAudioDevicesWithSystem) |
There was a problem hiding this comment.
When a user backs up an independent-input configuration and restores it into a fresh profile, this newly persisted value is neither represented in SettingsBackupPayload nor saved/restored by makeBackupPayload() and restore(from:), although the preferred device UIDs are. The restored profile therefore defaults syncing back to true, so it follows and modifies the macOS default instead of preserving the backed-up per-app input behavior; add an optional backup field for backward-compatible decoding.
Useful? React with 👍 / 👎.
Per-app input selection routes through the direct Core Audio IOProc path, and
syncAudioDevicesWithSystem is forced true whenever Direct Audio Capture is off.
That makes the preferred-device guard in bindPreferred{Input,Output}DeviceIfNeeded
unreachable: the AVAudioEngine path always follows the macOS system default, so
the code below each guard never ran.
Collapse both functions to explicit, always-succeeding no-ops with accurate
comments, and remove the helpers only the dead paths reached
(tryBindToSystemDefault{Input,Output}, setEngineOutputDevice). Keep
setEngineInputDevice — still used by the reconnect path.
No behavior change (the removed code was unreachable); the full suite (132 tests)
passes. Addresses Greptile's dead-code / misleading-comment finding on altic-dev#642.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dc53e92f98
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| get: { SettingsStore.shared.syncAudioDevicesWithSystem }, | ||
| set: { SettingsStore.shared.syncAudioDevicesWithSystem = $0 } |
There was a problem hiding this comment.
When a fresh user turns this toggle off without first changing the input picker, this setter only stores false and can leave preferredInputDeviceUID nil. resolvedInputDeviceForCapture() only stays independent when that UID is present; otherwise it falls through to AudioDevice.getDefaultInputDevice(), so subsequent macOS input changes are still captured while the UI says sync is off. Seed the preference from the currently displayed/default input when disabling sync.
Useful? React with 👍 / 👎.
| private func bindPreferredInputDeviceIfNeeded() -> Bool { | ||
| DebugLogger.shared.debug("bindPreferredInputDeviceIfNeeded() - Starting input device binding", source: "ASRService") | ||
|
|
||
| guard SettingsStore.shared.syncAudioDevicesWithSystem == false else { | ||
| DebugLogger.shared.info("Sync mode enabled - using system default input device", source: "ASRService") | ||
| return true | ||
| } | ||
|
|
||
| guard let preferredUID = SettingsStore.shared.preferredInputDeviceUID, preferredUID.isEmpty == false else { | ||
| DebugLogger.shared.info("No preferred input device set - using system default", source: "ASRService") | ||
| return true | ||
| } | ||
|
|
||
| DebugLogger.shared.debug("Attempting to bind to preferred input device (uid: \(preferredUID))", source: "ASRService") | ||
|
|
||
| guard let device = AudioDevice.getInputDevice(byUID: preferredUID) else { | ||
| DebugLogger.shared.warning( | ||
| "Preferred input device not found (uid: \(preferredUID)). Falling back to system default input.", | ||
| source: "ASRService" | ||
| ) | ||
| // Try to use system default as fallback | ||
| return self.tryBindToSystemDefaultInput() | ||
| } | ||
|
|
||
| DebugLogger.shared.debug("Found preferred input device: '\(device.name)' (id: \(device.id))", source: "ASRService") | ||
|
|
||
| let ok = self.setEngineInputDevice(deviceID: device.id, deviceUID: device.uid, deviceName: device.name) | ||
| if ok == false { | ||
| DebugLogger.shared.warning( | ||
| "Failed to bind engine input to preferred device '\(device.name)' (uid: \(device.uid)). Trying system default input.", | ||
| source: "ASRService" | ||
| ) | ||
| // Try to use system default as fallback | ||
| return self.tryBindToSystemDefaultInput() | ||
| } | ||
|
|
||
| DebugLogger.shared.info("✅ Successfully bound input to '\(device.name)'", source: "ASRService") | ||
| DebugLogger.shared.info("AVAudioEngine input follows system default (per-app selection uses direct capture)", source: "ASRService") | ||
| return true |
There was a problem hiding this comment.
When sync is disabled but DirectCoreAudioInput cannot be used for the selected mic (for example the helper rejects devices with multiple input streams in CoreAudioCaptureSupport.c:54-57, or AudioDeviceStart fails), startPreferredAudioCapture() falls back to the AVAudioEngine path. This method now always succeeds without binding preferredInputDeviceUID, so the recording silently uses the macOS default while Settings/menu still show the dedicated mic; either fail/notify or bind/re-sync before using the fallback.
Useful? React with 👍 / 👎.
…eferenceCoordinator Upstream altic-dev#531 ("Add support for choosing custom audio input") landed while this branch was open and reworked the same code paths, so the conflicting files are resolved wholly in favour of upstream and this branch's contribution is re-expressed as a small delta on their model. altic-dev#531's `.manual` mode honours the preferred microphone by moving the macOS default input to it (`enforcePreferredInput` -> `setDefaultInputDevice`, reasserted at recording start and on hardware changes). That leaves the original goal of this branch uncovered: capturing a dedicated microphone for FluidVoice alone, without taking over the system input for every other app. So `MicrophoneSelectionMode` gains a third case, `.fluidVoiceOnly`: - `inputDeviceForCapture()` resolves the preference in both preferred modes. - `enforcePreferredInput()` returns the new `.skippedFluidVoiceOnlyMode` — never touching the system default is the entire feature. - Gated on `experimentalDirectAudioCaptureEnabled`; a stored `.fluidVoiceOnly` reads back as `.manual` when direct capture is off, so the preference is still honoured rather than silently ignored. - Two guards against silently recording the wrong microphone, since device resolution falls back to the macOS default by design: `preferredInputIsMissing()` refuses to start when the pinned device is unplugged, and `startPreferredAudioCapture()` refuses the AVAudioEngine fallback, which can only ever record the system default. - Hardware-change handlers skip `stabilizePreferredInputAfterHardwareChange` (it reasserts the default) and re-resolve the capture device instead. - `setMicrophoneSelectionMode()` captures the user's own input device when *entering* `.manual` from any other mode, and hands it back when leaving. Capturing only from `.system` (as before) missed the `.fluidVoiceOnly` -> `.manual` -> `.fluidVoiceOnly` round trip and left the system input stuck on FluidVoice's choice after the user opted back out — found on-device by walking that exact sequence. - Settings and the menu bar each gain one "use for FluidVoice only" switch. Superseded by upstream and dropped: the preferred-UID-clobbered-on-disconnect fix, the derived Picker binding, and the `syncAudioDevicesWithSystem` boolean. Local: swiftlint --strict clean; 161 tests pass.
|
Congrats on landing #531 — and apologies for the collision, we were clearly working the same problem in parallel. I've rebased this PR onto it, and in doing so it got a lot smaller: #531 already has the right structure, so this is now a mode that plugs into What the two do differently. #531's This PR adds a third mode, Why it's possible without the -10851 problem. It rides the How it composes with your code:
Tests are in A couple of things I deliberately left alone, happy to change either:
Also worth noting: your fix for the disconnect clobbering Local: Verified on-device (Apple Silicon), reading the device actually bound out of
The Bluetooth rows matter because that is the device class this feature originally broke on. Direct capture binds an AirPods mic (negotiating 24 kHz rather than the 48 kHz the other devices use), transcribes from it, and releases the device cleanly on stop — and pinning FluidVoice to a wired mic while Bluetooth is the system default leaves the Bluetooth default alone. The last row is the AVAudioEngine-fallback guard firing for real: Understood if you'd rather ship #531 alone for now and revisit — in that case I'd still suggest keeping the direct-capture seam in mind, since it's what makes per-app selection possible at all. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c55f060094
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // on aggregate and Bluetooth devices). Falling back to it in FluidVoice-only mode would | ||
| // silently capture the system default while Settings and the menu bar still show the | ||
| // dedicated microphone, so fail loudly instead. | ||
| if SettingsStore.shared.microphoneSelectionMode == .fluidVoiceOnly { |
There was a problem hiding this comment.
Fresh evidence: this new guard only covers the startup fallback path; when an already-running direct capture hits the duration-mismatch recovery, handleDirectCaptureDurationMismatch() calls startCompatibilityAudioCapture(reason: "duration_mismatch") directly at ASRService.swift:942, bypassing this check. In a FluidVoice-only session whose direct stream drifts, the app can switch to AVAudioEngine and capture/bind the macOS/default path while the UI still promises not to, so the runtime fallback needs the same fail-loudly behavior or should route through startPreferredAudioCapture().
Useful? React with 👍 / 👎.
|
A few cross-references, since this lands in code several open reports are about. Flagging, not claiming to fix any of them. #655 (Drain audio engine off main queue) — that PR and this one overlap in #644 (crash on audio source switch) — explicitly not claiming a fix; the build in that thread already has confirmations from two reporters. But since this PR rewrites those two handlers, it seemed worth recording what the branch does under that scenario. Recording from a USB mic, then disconnecting the Bluetooth device — which moves the macOS default input out from under the session — recovered cleanly, no crash or hang: #592 (AirPods/Bluetooth stale stream format) — related, and observable on this path: capture from AirPods prepares at 24 kHz / 480 frames where wired devices come up at 48 kHz / 512, so the HFP format is being read correctly rather than reused from the A2DP route. #663 and #617 (audio muffled after dictating over Bluetooth) — not fixed here, and worth being plain about why: capturing from a Bluetooth microphone forces the headset into HFP, which degrades playback for the duration. That is inherent to using a BT mic and this PR doesn't change it. It does make it avoidable, though, which may be the more useful outcome for those reporters: with a dedicated input selected, FluidVoice captures that device and never opens the AirPods' input stream, so the headset stays in A2DP and music keeps playing at full quality while you dictate. Under As before — happy to rebase, split the coordinator refactor back out, or rename any of it. |
|
Thanks for the PR! @grohith327 check this out - could be related and we can take it in for next-next update or so if relevant |
2 participants
Description
Adds a microphone mode that captures a chosen input device for FluidVoice alone, without changing the macOS system default input — dictate from a USB handset or Bluetooth headset while every other app keeps using the built-in mic.
Rebased onto #531. That PR landed while this was open and reworked the same code paths, so this branch was rebuilt as a small delta on top of it rather than a competing implementation. #531's
.manualmode honours the preferred microphone by moving the macOS default input to it (enforcePreferredInput()→setDefaultInputDevice, reasserted at recording start and on hardware changes). This PR adds the case that doesn't cover: leaving the system default alone entirely.It works because of the direct Core Audio path already in the tree (
DirectCoreAudioInput→AudioDeviceCreateIOProcID), which binds an arbitrary input device per-app. That sidesteps the AVAudioEngine-10851limitation that got the original independent mode removed in c8df54a.Key changes:
MicrophoneSelectionModegainscase fluidVoiceOnly, plususesPreferredInputDevice/changesSystemDefaultInputso call sites read as intent rather than mode-equality checks.enforcePreferredInput()returns a new.skippedFluidVoiceOnlyMode— never callingsetDefaultInputDeviceis the feature.experimentalDirectAudioCaptureEnabled. With direct capture off, a storedfluidVoiceOnlyreads back as.manual, so the preference is still honoured by the existing path rather than silently ignored..manual, which survives a disconnect by moving the default — but in this mode that fallback means capturing the system mic while Settings shows the dedicated one. SostartPreferredAudioCapture()refuses the AVAudioEngine fallback, andpreferredInputIsMissing()refuses to start when the pinned device is unplugged. This is a deliberate behaviour change from the earlier revision of this PR, which fell back silently in both cases.stabilizePreferredInputAfterHardwareChangein this mode (it reasserts the system default) and re-resolve the capture device instead.setMicrophoneSelectionMode()now captures the user's own input device when entering.manualfrom any other mode, and hands it back when leaving. Capturing only on.system→.manual(the previous rule) missed thefluidVoiceOnly→manual→fluidVoiceOnlyround trip — i.e. flipping the new switch off and back on — and left the system input stuck on FluidVoice's choice after the user had opted back out. Found on-device by walking that sequence; behaviour for.system↔.manualis unchanged..manual, which moves the system input — the exact side effect the user opted out of.Type of Change
Related Issue or Discussion
Testing
swiftlint --strict --config .swiftlint.yml Sources Tests(0 violations)xcodebuild test— 160 tests, 0 failures (incl. newFluidVoiceOnlyMicrophoneModeTests)Hardware-validated by reading the device
ASRServiceactually bound out of its own logging, and sampling the macOS default input before / during / after:fluidVoiceOnly, USB handsetfluidVoiceOnly, Bluetooth (AirPods, HFP @ 24 kHz)fluidVoiceOnly, USB handset while Bluetooth is the system defaultfluidVoiceOnly, virtual device (BlackHole)system/manual, with Bluetooth connectedmanual, USB handsetfluidVoiceOnly, preferred device unpluggedfluidVoiceOnly, multi-stream aggregate devicefluidVoiceOnlystored.manualbehaviour.manualshouldAlso exercised: disconnecting the Bluetooth device mid-recording while capturing from the handset — capture stayed on the handset (
Default input changed; FluidVoice-only capture keeps its own device→ route recovery → same device), and the system default was left to macOS rather than reasserted.Screenshots / Video
FluidVoice-only mode — Input Device set to the USB handset while the macOS system input stays on the built-in mic:
Notes
CoreAudioCaptureSupport.cby design; in this mode that now surfaces as a clear refusal rather than a silent fallback to the system mic.fluidVoiceOnlywon't decode on an older build — inherent to adding an enum case, flagging it in case that matters for rollback.🤖 Generated with Claude Code