[BCI] QVAC-17071 feat: add BCI neural signal support (variable conv1 kernel + windowed attention)#10
Open
sharmaraju352 wants to merge 7 commits intomasterfrom
Open
[BCI] QVAC-17071 feat: add BCI neural signal support (variable conv1 kernel + windowed attention)#10sharmaraju352 wants to merge 7 commits intomasterfrom
sharmaraju352 wants to merge 7 commits intomasterfrom
Conversation
added 5 commits
April 16, 2026 19:57
Read n_audio_conv1_kernel from model hparams to allow BCI models to use a non-standard first convolution kernel size. Standard whisper models default to kernel size 3. Made-with: Cursor
- Add n_audio_window_size and n_audio_last_window_layer hparams - When present, encoder self-attention is restricted to a local window for layers up to last_window_layer - Bypass flash attention when windowed mask is active (Metal FA does not support custom F32 masks); flash attention remains enabled for non-BCI models and for the decoder - Populate window_mask data on the encoder graph (not the cross graph) - Add proper SOS token (language + transcribe) initialization for BCI models Backward-compatible: n_audio_window_size defaults to 0 and n_audio_last_window_layer defaults to -1, disabling windowed attention entirely for standard whisper models. Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
Made-with: Cursor
5326bf7 to
bbb3535
Compare
GustavoA1604
requested changes
Apr 16, 2026
added 2 commits
April 18, 2026 10:46
Address review feedback: 1. Guard read_safe for BCI-specific hparams (n_audio_conv1_kernel, n_audio_window_size, n_audio_last_window_layer) behind a n_mels > 256 check. Standard whisper models have n_mels <= 128 and do not contain these fields — reading them unconditionally would corrupt the file position and break model loading. 2. Add explicit is_bci flag to hparams struct, set when BCI fields are detected during loading. 3. Use is_bci flag (instead of n_audio_window_size > 0) to guard the BCI-specific decoder SOS token initialization. 4. Log BCI-specific hparams when a BCI model is detected. Made-with: Cursor
The windowed attention mask values depend only on n_ctx and window_size, both fixed after model load. Move the O(n_ctx^2) computation from whisper_encode_internal (called every encode) to whisper_init_state (called once). The encode path now just copies the precomputed data to the graph tensor. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two changes to whisper.cpp to support brain-computer interface (BCI) neural signal transcription. Based on v1.8.4.1.
1. Variable conv1 kernel size
n_audio_conv1_kernelfrom model hparams (defaults to 3 for standard whisper models)2. Windowed self-attention for encoder layers
n_audio_window_sizeandn_audio_last_window_layerhparamslast_window_layerBackward compatibility
Both changes are backward-compatible:
n_audio_conv1_kerneldefaults to3(standard whisper behavior)n_audio_window_sizedefaults to0andn_audio_last_window_layerdefaults to-1, which disables windowed attention entirelyContext
Required by the new @qvac/bci-whispercpp addon: tetherto/qvac#1583
Test plan
Test results
BCI package (
@qvac/bci-whispercpp)Standard whisper package (
@qvac/transcription-whispercpp)