DotPad: Add multi-button combination gesture support

[bramd]

Link to issue number:

None

Summary of the issue:

The DotPad driver only supports single-button input gestures. Users cannot press multiple buttons simultaneously to trigger custom gestures.

Description of user facing changes:

Multi-button input gestures are now supported on DotPad braille displays and can be mapped to NVDA functionality via the Input Gestures dialog. For example, pressing f1 and panLeft simultaneously produces the gesture f1+panLeft.

Description of developer facing changes:

Only internal changes in the DotPad driver. No public API changes.

Description of development approach:

• Added DP_KeyGroup enum to categorize key groups (FUNCTION, PERKINS, etc.) based on the second byte of notification commands.
• Added secondByte property to DP_Command for extracting the key group from command values.
• Implemented key press state tracking across multiple groups (_keysPressed, _keyGroupsReleased).
• Gestures fire only when all key groups are released, enabling multi-button combinations.
• Replaced the previous DPKeyGesture with DPInputGesture which builds gesture IDs from all pressed keys.
• Gesture names use camelCase (e.g., panLeft, panRight, navCenter) converted from the DP_PerkinsKey enum names.
• Bit order is reversed (LSB to MSB) to match BRLTTY and protocol documentation key numbering.

Testing strategy:

• Enabled input help and verified:
  • Single button gestures still work and are reported correctly
  • Multi-button presses are reported correctly with consistent key ordering
• Assigned functions to gestures in Input Gestures dialog, confirmed they work
• Verified existing panLeft/panRight scroll gestures still function

Known issues with pull request:

• Normalizing gesture names to camelCase may break existing user gesture mappings to panLeft/panRight if a user has redefined those.
• Long press support was removed from this PR per review feedback. It will be submitted as a separate PR that adds long press as a framework-wide abstraction for all braille gestures, with a configurable threshold setting.

Code Review Checklist:

• Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• API is compatible with existing add-ons.
• Security precautions taken.

[seanbudd]

Hi @bramd - I just want to confirm with the team whether or not we want long press support in NVDA properly or just for this device. I think we may want to ensure long-presses are available for all gestures and part of our framework

[bramd]

Hi @bramd - I just want to confirm with the team whether or not we want long press support in NVDA properly or just for this device. I think we may want to ensure long-presses are available for all gestures and part of our framework

Good point. If we want to make it more generic I could either extend the PR to implement this or take the logn press support out and separate that in it's own PR. Also, if we make it generic we might want to consider key repeat as well. The current implementation fires one longPress gesture after holding keys, but does not fire gestures on a certain time interval. People might want this for key repeat on braille scrolling gestures for example.

[bramd]

@seanbudd I'll put this back to draft for now until we have a decision on long press gestures.

[seanbudd]

@bramd - please refactor out the long press support into a different PR which adds the abstraction for all forms of gestures

[bramd]

@seanbudd To clarify on the follow up for long press support: I would propose this to be braille specific. It doesn't make sense for keyboard input and touch has it's own specific handling of hover/tap/hold. Was that your intent or where do you see other uses for long press support?

[SaschaCowley]

@bramd said:

To clarify on the follow up for long press support: I would propose this to be braille specific. It doesn't make sense for keyboard input

Why doesn't it make sense for keyboard input? I'm not saying we need to add it for keyboard input, but conceptually it makes sense to me, and would allow further flexibility (for instance, NVDA+t could speak the title on press, spell it on double-press, and copy it on long-press).

[seanbudd]

Thanks @bramd

[seanbudd]

@bramd - keeping it braille specific for now is fine too. Can you also make multi-button support available for all braille devices?

[seanbudd]

Can you also make multi-button support available for all braille devices?

Sorry I'm not sure if this is the case already or if we can make this more generic

[Copilot]

Pull request overview

Adds multi-button combination gesture support for the DotPad braille display driver, enabling NVDA users to bind simultaneous button presses (e.g., f1+panLeft) via the Input Gestures dialog.

Changes:

• Added key-group categorization (DP_KeyGroup) and command parsing helper (DP_Command.secondByte) for key notifications.
• Implemented multi-group key press state tracking and a new DPInputGesture that builds combo gesture IDs from all pressed keys.
• Updated default DotPad pan gesture identifiers and documented the new feature in user-facing change notes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File	Description
user_docs/en/changes.md	Documents multi-button gesture support for DotPad displays.
source/brailleDisplayDrivers/dotPad/driver.py	Implements key state tracking across notifications and generates combination gesture IDs; updates default gestureMap IDs.
source/brailleDisplayDrivers/dotPad/defs.py	Adds DP_KeyGroup enum and DP_Command.secondByte helper used for identifying key groups.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[bramd]

Why doesn't it make sense for keyboard input? I'm not saying we need to add it for keyboard input, but conceptually it makes sense to me, and would allow further flexibility (for instance, NVDA+t could speak the title on press, spell it on double-press, and copy it on long-press).

Correct, but currently we trigger a gesture on key down and holding a key would result in key repeat kicking in. For example, if you hold nvda+t for a while, you will get the title copied since the script count increases on every key repeat and the highest count (3) will copy the title. So to handle long presses, we would need to trigger the gesture on key up, which might feel sluggish and is different from how most programs handle keyboard input. It might also interfere with gestures that are useful with key repeat now, such as pressing and holding up/down arrow in a virtual buffer and scrolling through the text rapidly. If we want to explore long press keyboard gestures it might be better to do that in a separate issue since I think there are some design decisions and edge cases to work out if we want to go that route.

[bramd]

Can you also make multi-button support available for all braille devices?

Sorry I'm not sure if this is the case already or if we can make this more generic

As it is now, every BrailleDisplayDriver has it's own gesture class and the driver constructs that class with the expected input, which often is a list/set of keycodes. This is all fairly device specific and I think needs testing on real hardware if we refactor/change it. I am considering making some re-usable methods for key press/release tracking for long press support, so a braille display driver can start using those to have less code to manage and use the long press concept. I think this should be opt-in though, so we can ask existing braille driver maintainers to implement this and test it on real hardware.

To get back to your original question, multi button combinations are already implemented for the braille display drivers I'm familiar with. For Dot Pad it just had a very simple implementation to make single buttons work on press (not on release).

[Qchristensen]

Reads well, thanks!