Problem Statement

The calculator app currently supports different input mechanisms (keyboard, mouse, touch, pen), and while the app interface has been tailored in some way for for the rest of them, pen input has received less attention: the pen is treated like a regular mouse pointer with no special consideration. This is less than ideal when trying to input long numbers in a device where pen is one of the primaty input methods (e.g. Surface Studio in canvas mode, Surface Go with no keyboard attached), where many users would find easier to just write down the numbers.

Evidence or User Insights

Many Windows surfaces have benefited lately from tighter ink integration, from the new inline text input canvas to the integration of ink in apps like the Mail client, the Office suite and Microsoft To-Do. Users of pen enabled devices have come to expect more intuitive ways to interact with their computer, with the pen allowing rich ink input rather than just behaving like a simple pointer. The community response to this suggestion has been very positive suggesting that at least high-confidence users would find it a welcome addition (no pun intended).

Proposal

The proposal is to allow user to enter calculations in handwriten form, which will be recognized, parsed and calculated as any other user input.

Goals

• Allow active pen (e.g. Surface Pen) users to enter handwritten calculations, recognize them and display the result
• Understand mathematical expressions that can be expressed in one line using the same limited expression format the calculator already recognizes for pasted strings. E.g.:
  • VALID: 1+ 2 * 3
  • NOT VALID: sin(1)

Non-goals

• Provide handwriten input for touch users
  • Accuracy wouldn't be satisfactory and it would conflict with the current touch usage patterns
• Provide feature parity with the Math Input Panel
  • It would be a nice follow-up but it would require additional work on the recognition logic
• Replace the current numeric pad. It should remain available so the user is free to use the pen as a pointer if they prefer it.

Low-Fidelity Concept:

I was playing around with the code and I added support for entering expressions using ink:

It works by overlaying a InkCanvas on top of the calculation result, and leveraging the existing OnPaste method to process the recognized string.

Is this a feature you would be interested in adding? If you do I can polish it and send a PR 😊

Note: A team member edited this comment for clarity based on conversation below.