|
Digital Inking with Tablet PCs |
|||||||||||||||
|
|
Tablet PC allows users to input data with a digital pen as well as a standard keyboard or mouse. Tablet PCs have special screens which use an active digitizer to enable users to write directly on the screen to control their PC and to input information as handwriting or drawing. This process, called inking, enables users to add "digital ink" to a full range of Windows applications, which appears as natural-looking handwriting on the screen. The digitized handwriting can be converted to standard text through handwriting recognition, or it can remain as handwritten text. Both the converted text in typeface and the cursive handwritten text function equally well as data formats in Windows applications and platforms -- that is, both forms of text can be sent as e-mail in Microsoft Outlook and exchanged as documents in Microsoft Word, and can be sent from the Tablet PC to a desktop computer or a Pocket PC, which can display the text in the same character format that it was sent in. More Intuitive To UsersIn many situations, handwriting is a more natural input method for placing data into applications. Inking makes these applications more convenient and available to users. Sitting at a meeting, working out of doors, wedged in a cramped airplane seat -- these are all situations where fiddling with a keyboard is challenging. Inking also allows users to insert a sketch or drawing, jot down a chart, take free-hand notes, and annotate an existing document such as a PowerPoint presentation. "Using a pen to input information into the Tablet PC is simply a more natural and productive way to work in many situations" says Microsoft's Tablet PC expert Alex Gounares. "Using digital ink is so much quicker and easier than using a keyboard." "Thinking in ink" -- maximizing the possibilities of ink as its own data format -- is the other great potential of the Tablet PC and inking, according to Gounares. "There are so many cool things that we can do with ink as ink," says Gounares. "If you consider the ability to take free-form notes, send those around in e-mail, annotate Microsoft Excel spreadsheets with ink comments and so forth, we really have opened up new scenarios and capabilities for end users. But this is just the beginning of what will be possible as software developers create new applications and functionalities for digital ink." How Digital Ink WorksInking is actually a broad term that represents a set of technologies. A number of technologies had to come together to make inking possible. Many teams across Microsoft -- including Microsoft Research teams on several continents -- were involved in solving the technological challenges that came together as inking. First, there were hardware considerations. The input screen required special features. On Tablet PCs, a digitizer overlain on the LCD screen creates an electromagnetic field. When the pen comes in contact with the screen's electromagnetic field, its motion is reflected on the screen as a series of data points. As the pen continues to move across the screen, the digitizer collects information from the pen movement in a process called "sampling." The Tablet PC digitizer is capable of sampling 130 "pen events" -- units of motion that correspond to data points -- per second. These electromagnetic pen events are then represented visually on the screen as pen strokes. Because of its high sampling rate, the Tablet PC is able to create the effect of "real-time inking;" that is, as the user writes on the LCD screen, digital ink appears to flow at the same speed that the pen writes, no matter how fast the pen moves. The high sampling rate also enables written ink to be displayed and stored with very high graphical resolution. Not only is this important for visual legibility on the screen, it is necessary for maximizing accuracy during the process of handwriting recognition. The more data points collected in the ink objects, the greater the accuracy when the data passes through the recognizer and is associated with words. "Microsoft is a leader in handwriting recognition," says Gounares. "Microsoft has invested in this technology for many years, and we have built up considerable expertise and technology. However, even though we have state-of-the-art handwriting recognition technology, it is important to understand that handwriting recognition is inherently a statistical process. For some users it will work really well, for others, not so well. As a rough rule of thumb, if you can't read your own writing, neither can the computer!" The Tablet PC does not require the user to "train" the software to provide immediate functionality -- the handwriting recognition engine can work right away, with any user. "We have developed this technology by studying and analyzing large numbers of handwriting samples," says Gounares. "Microsoft has collected and stored such a large sample of handwriting in the recognizer that the Tablet PC recognition software requires no additional training." Additionally, each of the supported languages -- American and International English, German, French, Korean, Japanese and Simplified and Traditional Chinese -- has its own recognizer engine with a bank of language-specific handwriting samples. If the Tablet PC employs the recognizer engine for the French language, for instance, it would use its log of French handwriting characteristics to interpret the handwritten ink objects for greatest accuracy. The Value of Digital InkThe inking process gives users the choice of converting the handwritten data to standard text through handwriting recognition. Users can also preserve the data in its ink format --keep ink as ink -- and lose none of its functionality. Hand-drawn or written ink need not be converted to a different format to be saved, sent, sized or otherwise manipulated by Windows-based applications or exported across Windows platforms. Handwriting preserved as ink objects are fully supported as a defined and recognized "native data type" across Windows-based applications and platforms. That is, digital ink can be passed from application to application as digital ink. For example, on a Tablet PC, users can respond to e-mail in Outlook by writing responses in ink by hand onto the screen. The response is sent and received as handwritten digital ink -- without the necessity of converting the handwritten ink objects to text. This means that the Tablet PC is more than just a handwriting recognition tool. Notes taken in ink, sketches drawn in ink, annotations made in ink -- none of these need to be converted to function as a data type in Windows applications. A handwritten document has the same value and versatility as a document that has been keyed in on a keyboard. Maintaining ink as ink is one of the most exciting breakthroughs of the Tablet PC. The beauty of ink is that it allows you to express yourself freely. It preserves the context and personality of communication. You're not restricted to any predefined structure: You don't need to stay within the lines. Inking technology will also serve as the foundation for many innovative new applications that will join the naturalness of handwriting and drawing with the power of the PC. Gounares spins out a few examples. "With today's Tablet PC technology, you can express yourself freely with ink in diagrams and drawings -- you can capture anything you could do with a pen. And in the future, these technologies can be greatly expanded upon. The University of Waterloo in Canada, for example, is building on the Tablet PC foundation to create a mathematics recognizer. Long and complex equations can be written quickly in longhand and solved automatically," he says. "Over the course of the next few years, amazing applications will be developed for the Tablet PC that capitalize on the promise of ink as ink," says Gounares. "Our mantra while developing the inking technology was that it had to be better than paper. We wanted the naturalness of paper with the power of the PC. There are few barriers to expressing yourself in ink."
| |||||||||||||
