Skip to content

Add direct click, which activates the focused item with a real touch - #159

Open
zywek123 wants to merge 1 commit into
trypsynth:masterfrom
zywek123:direct-click
Open

zywek123 wants to merge 1 commit into
trypsynth:masterfrom
zywek123:direct-click

Conversation

@zywek123

@zywek123 zywek123 commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Some controls do nothing on the accessibility click and react only to
touches, such as Add to favorites in TikTok, which works out the button
from where the finger lands. Until now, the only ways to activate them
were a double tap and hold released just after the hold is recognized,
with no signal of when, or the pass-through gesture and a finger that
hits the control.

Direct click is a new switch in the advanced settings, off by default,
for Android 13 and later with Backtalk's gesture detection. When it is
on, activating the focused item, by double tap, the activation key or a
braille display, touches it on the screen like a finger instead of
sending ACTION_CLICK. The setting is translated into Polish.

A gesture Backtalk dispatches goes through touch exploration like a
finger does, so its press reaches gesture detection rather than the
app, also in the 300 ms after a double tap while Android still counts
the touch as going on. DirectClick presses the middle of the part of
the item that shows, with a stroke that stays down (willContinue). When
TouchInteractionMonitor gets that press, which it tells apart by the
missing device and its point, it calls requestDelegating(), so Android
hands the press to the app, and only then does DirectClick lift the
finger by continuing the stroke. Both calls are synchronous, and Android
handles them on its main thread in the order they come, so the app gets
the press before the lift and holds it for 100 ms, however busy Backtalk
is. A single stroke with its own lift would not do: if the lift reached
Android before the hand-over, the hand-over would still send the app a
press with no lift to follow, which the app takes for a long press.
Waiting for TYPE_TOUCH_INTERACTION_END and touching through a one-pixel
pass-through region, as an earlier version of this change did, works
too, but holds every click for about 300 ms.

The usual click stays for text fields and sliders, where a touch would
put the cursor or set the value, on a magnified screen, on other
displays, for items out of sight, while a finger is on the screen, whose
touch the press would cancel, right after touch exploration, which
Android goes on with for a while after the finger lifts, in pass-through
regions, where Android would hand the press to the app before Backtalk
could lift it, and whenever the press doesn't reach the app. Double tap
and hold still goes to Android as a long press, as a double tap only
counts when the second tap lifts within 100 ms.

Gesture detection also stops counting a finger as down once a touch
that Android handed straight to the app ends, such as one in direct
touch's region, which reaches Backtalk as a down without its lift.
Direct click only presses while no finger is down.

DirectClickTest covers the press, the hand-over and the lift, their
timeouts, refusals and cancellations, and the point that is touched.
TouchInteractionMonitorTest runs a direct click right after a double tap
with Android's own controller: the press goes to the app and is neither
a finger nor a gesture. It also checks a press that can't be handed
over, one that Android handed over itself, when direct click may press,
and that a second tap held down is no double tap. Letting the press
through to gesture detection fails two of these tests, and letting
direct click press during touch exploration, or dropping the check for
a press that Android handed over, fails one each.

On an Android 16 emulator, with direct click on, an item in Settings
opens about 110 ms after the double tap is recognized, double tap and
hold on a launcher icon opens its menu, and a quick release after the
hold opens the app. With direct click off, the usual click opens the
item. [S10, Android 15: TikTok's Add to favorites only works correctly when direct click is enabled. When disabled, the BackTalk click action is ignored]

Limitations: direct click needs Backtalk's own gesture detection, so it
does nothing on Android 12 and earlier, or with gesture detection off.
If a finger touches the screen in the few milliseconds between the
press and the hand-over, Android hands that finger to the app instead.
The press lasts 100 ms, so direct click is about 100 ms slower than the
usual click. Moving the pass-through mode's timer to the main thread is
a separate PR #160.

  • DirectClick: presses the item, lifts the finger once gesture detection
    hands the press over, and clicks the usual way when the press doesn't
    reach the app
  • TouchInteractionMonitor: takes the press of a direct click before
    gesture detection, hands it to the app, tells when direct click may
    press, and clears the finger when a touch ends
  • FocusActor: activates the focused item with DirectClick while the
    switch is on
  • TalkBackService: gives DirectClick to the main display's monitor
  • PassThroughModeActor.State.passesThrough(): where Android passes a
    touch straight to the app
  • The switch, its Polish translation, and two MissingTranslation entries
    in lint-baseline.xml

  Some controls do nothing on the accessibility click and react only to touches, such as Add to favorites in TikTok, which works out the button from where the finger lands. Direct click, a new switch in the advanced settings for Android
  13 and later with Backtalk's gesture detection, off by default and translated into Polish, makes activating the focused item, by double-tap, the activation key or a braille display, touch it on the screen like a finger instead.

  Backtalk presses the item with a gesture of its own. Its gesture detection recognizes the press, has Android hand it to the app, and only then lifts the finger, so the app gets a 100 ms tap right after the double-tap, without waiting
  for Android to end the touch, and never holds the finger long enough for a long press. Double-tap and hold still goes to Android as a long press.

  The usual click stays for text fields and sliders, where a touch would put the cursor or set the value, on a magnified screen, on other displays, for items out of sight, while a finger is on the screen or right after touch exploration,
  in pass-through regions, and whenever the press doesn't reach the app.

  Gesture detection also stops counting a finger as down once a touch that Android handed straight to the app ends, such as one in direct touch's region, which reaches Backtalk as a down without its lift. Direct click only presses while
  no finger is down.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants