ShowMe ++ is an immersive mobile collaboration system that allows a remote user to communicate with a peer using video, audio and hand gestures. The system uses two Head Mounted Displays, cameras and depth sensors to enable a remote expert to be immersed in a local user's point of view and control devices in that user’s environment. The remote expert can provide guidance through the use of hand gestures that appear in real-time in the local user’s field of view as superimposed 3D hands. In addition, the remote expert is able to operate Internet connected devices in the novice’s environment and bring about physical changes by using the same hand gestures the novice would use. We also explored how to create a system that makes invisible data from our environment visible, combining widely accessible hardware to visualize layers of information on top of the physical world. Using our implemented prototype, the user can visualize, interact and modify properties of sound waves in real time by using intuitive hand gestures. As such the system supports experiential learning about certain physics phenomena through observation and hands-on experimentation.

ShowMe ++ consists in three different projects:

  • ShowMe: A Remote Collaboration System that Supports Immersive Gestural Communication
  • RemotIO: A System for Reaching into the Environment of a Remote Collaborator
  • Invisibilia: Revealing Invisible Data Using Augmented Reality and Internet Connected Devices
  • Publications

    Amores . J, Benavides. X , Maes. P, ShowMe: A Remote Collaboration System that Supports Immersive Gestural Communication, Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, April 18-23, 2015, Seoul, Republic of Korea  [PDF]

    Benavides, X, Amores, J, & Maes,. Invisibilia: revealing invisible data using augmented reality and internet connected devices. (UbiComp '15:) International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers   [PDF]

    Benavides. X, Amores. J, Maes. P. Remot-IO: a System for Reaching into the Environment of a Remote Collaborator. (UIST´15) 28th ACM User Interface Software and Technology Symposium.  [PDF]