Future Apple Pencil could be used to trace 3D objects for easy AR generation

Future Apple Pencils could allow sketching or writing on any surface.

Following multiple patent applications regarding different ways of adding controls to the Apple Pencil, Apple has now looked outside the stylus to the surfaces that it can used on. Currently all Apple Pencils require a touch screen such as an iPad display, but future ones may become much more independent.

"Content creation using electronic input device on non-electronic surfaces," US Patent No. 10,613,666, proposes that the Pencil could be used on any surface at all. If the Pencil can be made to detect and then transmit when it is in contact with a surface, when it is moved, and so on, then it could be used anywhere.

Rather than being confined to a relatively small iPad screen, an artist could sketch out on a whole table, and that drawing would be accurately created on the device. Artists could trace out existing drawings or plans, or could produce 3D drawings of objects by tracing the Pencil over such an object in real life.

"In conventional systems including a touch-sensitive surface," says the patent, "content generation using an input device, such as an active stylus, can be based on a detected path between a touch-down location and a lift-off location."

"[The] touch-down and lift-off on a non-touch-sensitive surface can be monitored by a force sensor of the input device," it continues. "The position and/or motion of the input device can be tracked according to various methods including one or more of a motion and orientation sensor, a camera, or an electromagnetic- or sound-based triangulation scheme."

All of this can be used equally for drawing or writing. "The force data and position/motion data can be processed to generate content, including textual character input and three-dimensional objects," it continues.

It may even be possible to sketch in thin air. "In some examples, the content can be generated based on tracking position and/or motion of the input device without requiring contact with a surface," says the patent.

Apple isn't planning to replace the touch-sensitive display, however,, arguing instead that improving the Apple Pencil in this way will be useful in existing situations. "In particular, use of an active stylus capable of generating stylus stimulation signals that can be sensed by the touch-sensitive device can improve the precision of stylus input," it says.

The patent is credited to three inventors, Tyler S. Bushnell, Steven Cardinali, and Katherine E. Tong. Bushnell and Cardinali were previously listed together as two of the inventors on an Apple Watch patent that would see the Digital Crown gain touch-sensitive gestures.