Pranav Mistry is a PhD student in the Fluid Interfaces Group at MIT’s Media Lab. Before his studies at MIT, he worked with Microsoft as a UX researcher; he’s a graduate of IIT. Mistry is passionate about integrating the digital informational experience with our real-world interactions.
Gestures are everything, and come naturally to us. Pranav asks why we can’t interact with computers in the same we we interact. He experimented with different input systems for computers
- a hacked mouse was turned into a glove – allowing the computer read hand movements
- sticky notes that could be written and read by the computer, then either sent on as sms or treated as an input to the computer.
- A pen that can draw in 3 dimensions
- A computer map built into a table
People are interested in information, not necessarily the computers or pixels that show them. His next step was to try to eliminate the computer. SixthSense is a helmet mounted computer projected to a wall, that tracks your fingers using a camera. You can make gestures at any wall to use the computer. One gesture immediately takes a photo, another allows sending it as an email. Some extra features, acting as an interface between physical and digital world are
- can recognise an object such as a book, and project a review onto it (3star etc)
- can project videos or images onto newspapers
- looking at a boarding pass and seeing if the flight is delayed
- playing pong on the ground with your feet
- can project onto a piece of paper and use it as a touch screen (play racing games, draw with finger, browse web)
- ‘copy and paste’ from physical world onto the paper screen
SixthSense has the potential to keep us more connected to the physical world, and keep us human rather than a machine in front of another machine.
At the end of the video, he announced the software will be made open source for others to experiment with. The hardware is relatively cheap at ~$300.