The touch area for each digit needs to be much bigger - there is room in the existing design it just isn't under the semantics node.

For reference, our time picker compared to android native.