At the 2009 Computer-Human Interaction (CHI) conference in Boston, Dr. Pattie Maes and her doctoral student, Pranav Mistry, the Fluid Interfaces Group at MIT’s Media Lab unveiled the prototype of SixthSense, a wearable gestural interface that augments our physical world with digital information, and lets us use natural hand gestures to interact with that information.

Basically, the SixthSense prototype is mobile projector coupled with a Webcam and a cell phone. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the Webcam recognizes and tracks user’s hand gestures and physical objects using computer-vision based techniques. SixthSense uses simple computer-vision techniques to process the video-stream data captured by the camera and follows the locations of colored markers on the user’s fingertips (which are used for visual tracking). In addition, the software interprets the data into gestures to use for interacting with the projected application interfaces.

The current SixthSense prototype supports several types of gesture-based interactions, demonstrating the usefulness, viability, and flexibility of the system.  It allows the user to project information from the phone onto any surface — walls, the body of another person or even your hand.  Interestingly, the current prototype system is quite inexpensive and it costs approximate $350 to build.

Pranav Mistry

In the following videos, Dr. Pattie Maes and Pranav Mistry demonstrated SixthSense technology and showed how this technology creating a mobile interface that will integrate into many parts of our life, giving an access to information for making optimal decisions throughout our day.

I am very impressed with the smart ideas and potentials of SixSense technology.  I can’t wait when such technology becomes available for consumers.  I anticipate SixSense technology will significant changes the way we work as well as the way we teach and learn in the classrooms.

About Dr. Steve Yuen

I am Dr. Steve Yuen, a Professor Emeritus of Instructional Technology and Design at The University of Southern Mississippi. This is my personal blog on the use of emerging technologies in teaching and learning. Hope you find this blog interesting, stimulating, and educational. Please feel free to social bookmark this page.
This entry was posted in Mobile Technology, technology and tagged , , , , , . Bookmark the permalink.

6 Responses to SixthSense

  1. 鄭伊真 says:

    最近看了一本書”從零開始的人脈術”, 書中提到過去人們建立人脈傾向透過社團等的面對面方式, 而今愈來愈多的人(尤其是網路世代)習慣使用facebook、twitter等social network平台交朋友、尋求幫助、建立聲望、、、, 恰如文章中所述, 由於網路的發明, 技術的進步, 大大改變了人們的社交模式.

    但是我也發現兩者間的確存有差異,記得老師曾在上課提過台灣與西方網路使用的偏好, 以facebook來說, 西方用它來溝通, 台灣人加入是因為有好玩的小遊戲, 我想這跟我們文化中的見面三分情多少有關, 也可能跟民情有關, 台灣人在意見或情感的表達上仍是屬於較含蓄與保留的.

    不否認social network平台有許多的優點, 然而溝通不只是言語上的, 肢體或表情同樣會提供許多訊息, 是心口合一還是口是心非? 這不是加上幾個表情符號就可以辦到的.

  2. vivi says:


  3. Patrick Chiang says:




  4. 奕嬛 says:

    記得上個學期裡修習一門課”數位學習”,才剛了解虛擬實境(Virtual reality,VR)和擴增實境(augmented reality,AR)這二者的差異,並對於科技技術的進展快速感到十分訝異之時,卻又在上週的課堂中受到一記重擊:貼近真實生活的產品誕生了-sixth sense。
    而觀看完由老師 您推薦的影片後,我真的有種不可思議的感覺,人類的發明創意如何能始終源源不絕,使得我們的生活型態不斷地在改變,變得越來越便利;尤其是運用在教學方面,更是一大助益,教學場域不再侷限在教室裡,教學領域不再侷限在教學大綱上,教學形式不再侷限在固有的樣式,師生互動將會有更多變化,這是我們所期待與樂見到的結果。


  5. Ahu says:

    I would applaud the discovery and development of “sixth sense” if I knew that it had a philanthropic basis or inspiration, but to me it seems like a bunch of “bells and whistles” that will only end up causing students and teachers to day-dream, interrupt the decision-making process by adding a distracting interface, and literally lead to a loss of focus or ability to concentrate on the task at hand. A similar example, although one in which the technology is of practical use, is the recent development of flight pilot helmets which have the capability of providing similar functions, which include the provision of ground and radar information in one eye, and which interprets gestures for missile guidance. In this case, it provides vital survival information and keeps the pilot in a state of increased vigilance.

    Unfortunately, certain pilots can never adjust to this type of sensory input and others require a great deal of training to use it successfully and safely. Why would we introduce a form of “computer vision” to students which will additionally tax their attentional resources, interfere with ongoing awareness of the classroom environment, and which, to my knowledge, have yet to provide any real informational value beyond what could be provided by traditional means.

    When considering the flexibility of children’s minds and the plasticity which they demonstrate towards the integration of technology, it seems even more dangerous to toy around with this “computer-human interface” technology at early ages. The current multi-media, internet generation has “evolved” to multi-task and demonstrate technological intuition (positive outcomes), but also pose increasing challenges to educators and demonstrate decreasing attention spans and motivation towards learning. Let’s try to avoid more of the same. Let this technology be used in an assisting manner to those with special needs, for example, where greater good can be done.

  6. Michael T. says:

    I can remember the first time that I saw this I thought it was a joke. The title of the video was something about some MIT students with an Augmented reality multi-device. When I first saw it I think I actually laughed out loud because the person wearing the device looked absolutely ridiculous. The person was wearing a camera, a projector, a mini-computer, and something else all strapped to his/her ( I can’t remember) chest. Then they started playing with it.

    I was fascinated by their ability to interact with things without touching them. They interacted with a digital news paper, took pictures with gestures, and even found out “deals” in the supermarket by interacting with the computer in its own reality.

    The thing that really impressed (and scared) me though, was that they were able to scan a person’s face and it pulled up all their information that was online *facebook, twitter, blogs, google searches, etc.*

    I’m sure that they have improved their device since then and even since this post. There have been a great many improvements in the technology, the size of the hardware, and sophistication of the software.

    This was my first experience with Augmented Reality and that technology. It seems to be catching on more and more. Even now Microsoft has released a SDK for its Kinect device that allows people to do some amazing things with motions, gestures, and AR. This is an exciting time to watch the progression of this technology and others. These technologies combined doesn’t make movies like “Minority Report” such a far-fetched future anymore.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>