Use Any Surface as a Touchscreen - OmniTouch - Video

Watch the video to see how users can create a screen, move an image on the screen, zoom in and out, paint using a palette of colors, select buttons on a menu.

This week, Microsoft and Carnegie Mellon researchers are unveiling OmniTouch, a system that turns any surface into a touchscreen.

Composed of a depth-sensing camera and laser-based pico projector (more on that in a second), the contraption looks like a flattened-pancake version of R2D2 and sits on the user’s shoulder.

The user first creates a screen wherever he or she likes — such as a hand, arm, wall, table or pad of paper — and then, just like with any other touchscreen, uses fingers to navigate it.

The screen is created with the laser-based pico projector, which is an electronic system placed on a tiny chip. It transforms the image into an electronic signal, which then sparks laser lights of different colors.

Those in turn get copied and projected by mirrors, pixel-by-pixel, onto the selected surface.

The depth-sensing camera then detects when a user’s fingers are touching the “screen” (the surface the user has designated) and “pressing” a button on it.

To get the technology to work, the researchers needed to teach the system how to recognize fingers, which are crucial to the technology because they create screens, select buttons, zoom in and so on.

They dissected the qualities of fingers and developed a method for determining when a finger was “clicking” on something.

Researcher Hrvoje Benko said:

In this case, we’re detecting proximity at a very fine level. The system decides the finger is touching the surface if it’s close enough to constitute making contact. This was fairly tricky, and we used a depth map to determine proximity. In practice, a finger is seen as “clicked” when its hover distance drops to one centimeter or less above a surface, and we even manage to maintain the clicked state for dragging operations.

The technology was unveiled at the 2011 UIST, a conference on innovations in the ways humans interact computer in Santa Barbara, Calif., from October 16-19. Benko and Carnegie Mellon Ph.D. student Chris Harrison and Andrew D. Wilson of Microsoft wrote about the technology in this paper.