Skip to content

feat: preserve raw input audio in frame userdata - #7269

Draft
chenghao-mou wants to merge 2 commits into
mainfrom
chenghao/feat/AGT-3508-raw-input-recording
Draft

chenghao-mou wants to merge 2 commits into
mainfrom
chenghao/feat/AGT-3508-raw-input-recording

Conversation

@chenghao-mou

@chenghao-mou chenghao-mou commented Sep 14, 2026

Copy link
Copy Markdown
Member

Warning

This should not be merged until backend can handle the new channel

Expose aligned raw input through lk.audio.raw and label Krisp output with lk.audio.processing (denoised or isolated). Consumers select audio before resampling.

Pre-connect audio is resampled before processing so the raw audio and processing label remain available at agent input.

Tests cover raw/processed alignment with buffered and mutating processors, pass-through frames, and pre-connect resampling.

Addresses AGT-3508 (Linear).

Initial prompt and agent context

Model: GPT-6

for noise cancellation, can we preserve a raw field in userdata + processing=denoised vs isolated? if the plugin is NC, we keep raw + denoised, if it is VF, we keep raw + isolated $grill-me

immediately before. Resampling is fine

at input is enough, downstream resampling should be on the target audio frame. anyway.

FrameProcessor first

we should name it lk.audio.raw and lk.audio.processing

okay, let's create a PR to add this support, and also include a 3rd channel in recorder io to include the raw user input.

I guess we can skip recorder changes for now: Native Opus fallback loses recordings

When libopus is unavailable, CHANNEL_LAYOUT sends three channels to the native Opus encoder. That encoder accepts only mono or stereo, so encoding fails and leaves the recording incomplete.

@chenghao-mou
chenghao-mou requested a review from a team as a code owner September 14, 2026 13:57
@chenghao-mou chenghao-mou added the review_effort:high Needs broad context and substantial validation label Sep 14, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

@chenghao-mou chenghao-mou changed the title feat: preserve raw input audio and record a third channel feat: preserve raw input audio in frame userdata Sep 14, 2026
@chenghao-mou
chenghao-mou marked this pull request as draft September 14, 2026 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review_effort:high Needs broad context and substantial validation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant