Skip to content

ios: input handling cleanup #3949

Merged
tvanderstad merged 8 commits intomasterfrom
ios-mtk-rewrite
Nov 16, 2025
Merged

ios: input handling cleanup #3949
tvanderstad merged 8 commits intomasterfrom
ios-mtk-rewrite

Conversation

@tvanderstad
Copy link
Copy Markdown
Contributor

@tvanderstad tvanderstad commented Nov 14, 2025

This PR makes cleanups to iOSMTK.swift, mostly to the text and drawing input wrappers but also to the metal view itself.

I wasn't careful to be sure that everyone would like every cleanup. As a significant motivation, this cleanup has helped me get hands-on experience with Swift. I also intend to maintain this logic going forward and wasn't shy about subjecting it to personal preferences.

Changes:

  • format the file (using Zed)
  • factor delegate objects out of the objects that use them
  • store all delegates, gesture recognizers, and interactions as properties of their view
  • rename various things (e.g. iOSMTKTextInputWrapper -> MdView) and move related definitions together
  • assorted cleanups (see code review comments)
  • add comments, including 'mark' comments that appear in XCode's minimap
  • disable app debug to shorten dev feedback loop

QA steps:

  • Md
    • iPhone
      • type into a document
      • scroll a document
      • undo an edit
      • floating cursor (long press spacebar)
      • floating cursor (long press cursor)
    • iPad
      • type into a document (virtual keyboard)
      • type into a document (physical keyboard)
      • type into a document (universal control)
      • scroll a document (touch)
      • scroll a document (trackpad)
      • scroll a document (universal control)
      • undo an edit
      • drop an image into the editor (universal control)
      • floating cursor (long press spacebar)
      • floating cursor (long press cursor)
  • Svg
    • iPhone
      • select, copy, then paste some strokes
      • pan and zoom at the same time
    • iPad
      • select, copy, then paste some strokes (touch)
        • already can't select with touch
      • select, copy, then paste some strokes (pencil)
        • already, paste menu cannot be opened with pencil
      • select, copy, then paste some strokes (trackpad)
      • select, copy, then paste some strokes (universal control)
        • already, paste menu cannot be opened with universal control
      • pan and zoom at the same time (touch)
      • pan and zoom at the same time (trackpad)
      • pan and zoom at the same time (universal control)
        • already effectively crashes the app
  • Misc
    • interact with the space where the toolbar usually is in compact mode
    • hover tab strip to show a tooltip (trackpad)
    • hover tab strip to show a tooltip (universal control)

Existing issues:

Comment on lines +63 to 79
for gestureRecognizer in self.gestureRecognizers ?? [] {
// receive touch events even if they are part of one of these recognized gestures
// this supports checkboxes and other interactive markdown elements in the text area (crudely)
if gestureRecognizer.name == "UITextInteractionNameSingleTap"
|| gestureRecognizer.name == "UITextInteractionNameTapAndAHalf"
|| gestureRecognizer.name == "UITextInteractionNameLinkTap"
{
gestureRecognizer.cancelsTouchesInView = false
}
}

for gesture in gestureRecognizers ?? [] {
let gestureName = gesture.name?.lowercased()

if gestureName?.contains("interactiverefinement") ?? false {
gesture.addTarget(self, action: #selector(longPressGestureStateChanged(_:)))

// send interactive refinements to our handler
// this is the intended way to support a floating cursor
if gestureRecognizer.name == "UITextInteractionNameSingleTap" {
gestureRecognizer.addTarget(
self, action: #selector(handleInteractiveRefinement(_:)))
}
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new version of this looks clearer to me. Investigation revealed that self.addInteraction(textInteraction) adds all gesture recognizers in textInteraction.gesturesForFailureRequirements to self.gestureRecognizers, allowing me to iterate them in one loop. I also call out the names of the recognizers specifically.

Todo: looks like I forgot to fill in the correct name for the interactive refinement recognizer

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Todo update: done ✅

Comment on lines +126 to 128
@objc func handlePan(_ sender: UIPanGestureRecognizer? = nil) {
mtkView.handleTrackpadScroll(sender)
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While intended for the trackpad, this is a pan handler, so this name makes more sense to me. Vaguely more google-able. I would rename the mtkView function too, but actually there's already a handlePan there. This one's used for markdown and that one for drawing. Expect more renames like this after I clean that up.

Comment on lines +130 to 147
@objc private func handleInteractiveRefinement(_ recognizer: UIGestureRecognizer) {
switch recognizer.state {
case .possible:
break
case .began:
isLongPressCursorDrag = true
case .cancelled, .ended:
isLongPressCursorDrag = false

interactiveRefinementInProgress = true
case .changed:
break
case .ended, .cancelled, .failed:
interactiveRefinementInProgress = false

inputDelegate?.selectionWillChange(self)
mtkView.drawImmediately()
inputDelegate?.selectionDidChange(self)
default:
break
}
}
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enumerate all enum variants (can you tell I am a Rust engineer?). Also rename fn and mutated property to be vaguely more google-able.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you didn't get rid of the default tho...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Swift issues a warning if I do because the set of variants could change

pan.allowedTouchTypes = [
NSNumber(value: UITouch.TouchType.direct.rawValue),
NSNumber(value: UITouch.TouchType.indirect.rawValue),
// NSNumber(value: UITouch.TouchType.pencil.rawValue),
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mentioning the disallowed touch type made this make more sense to me

NSNumber(value: UITouch.TouchType.indirectPointer.rawValue),
]
pan.allowedScrollTypesMask = .all
if UIPencilInteraction.prefersPencilOnlyDrawing { // todo: update when prefersPencilOnlyDrawing changes
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

possible bug discovered: updating the pencil only drawing preference does not update the min touches for a pan

Comment on lines +1556 to +1558
/// Returns whether the event should be forwarded up the inheritance hierarchy
func handleKeyEvent(_ presses: Set<UIPress>, with event: UIPressesEvent?, pressBegan: Bool)
-> Bool
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function was updating a class property that callers would check to see what to do next. Now it returns a bool indicating what to do next and the property has been removed.

selectedDebuggerIdentifier = "Xcode.DebuggerFoundation.Debugger.LLDB"
selectedLauncherIdentifier = "Xcode.DebuggerFoundation.Launcher.LLDB"
selectedDebuggerIdentifier = ""
selectedLauncherIdentifier = "Xcode.IDEFoundation.Launcher.PosixSpawn"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the corresponding issue for this

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my bad: #3950

@tvanderstad tvanderstad merged commit d9ece25 into master Nov 16, 2025
1 check passed
@tvanderstad tvanderstad deleted the ios-mtk-rewrite branch November 16, 2025 03:49
@tvanderstad
Copy link
Copy Markdown
Contributor Author

remaining QA will be async

ad-tra added a commit that referenced this pull request Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants