The Touched Project: 2007-2010
Abstract
Video Coding and Processing for Improved Telepresence Experience in the Performing Arts
The aim of this project is to create new tools for networked dance performance in a telepresence environment, for 3D video acquisition, compression and interactive 3D display, which can also be utilized in a broader setting, i.e., teleconferencing, gaming, TeleHealth, connecting communities, education, surveillance, sightseeing, and sports.
The term telepresence refers to “the experience of being fully present at a live real world location remote from one's own physical location”. This project utilizes innovative technologies to create unique, in-person experiences between people, places, and events in their work and personal lives— over the network.
Objectives
Conduct a series of exploratory exercises in telepresence technology as an assessment of the state of the art. These will be carried out in the IRMACS facility (Interdisciplinary Research in the Mathematical and Computational Sciences) at SFU, which is a benchmark facility for distributed collaborative interaction.
Explore the limitations of the current technology, and determine the amounts of audio and video delay that can be tolerated in distributed dance performances utilizing contemporary music and movement techniques as well as traditional indigenous music and movement techniques. These delay tolerances are different for different dance styles and forms.
Create and/or capture examples of different dance styles and/or forms as case studies, analyze their motion vocabularies, and from them create suitable motion models for low delay video compression.
Create algorithms for low delay video encoding and predictive video decoding based on the aforementioned motion models. Use these new video-coding algorithms in short choreographed telepresence performance works.
Develop a multi-view video coding and 3D displaying system by integrating multiple stereo cameras, the latest multi-core Cell processor from IBM, the H.264-based multiview video coding, and auto-stereoscopic LCD display.
Develop interactive 3D displaying software to improve the viewing, training and motion analysis of dance performance and explore the application of this system and software to live performance, telepresence activities and other 3D-related areas.
Develop two major interdisciplinary and intercultural new media dance works out of the aforementioned experimental tool building and optimization projects that fully challenge the range of telepresence activities.
Train graduate and undergraduate students in the new interdisciplinary methods we use.
Collaborators
Dr. Henry Daniel (Artistic PI)
Dr. Ivan V. Bajić (Scientific Co-PI) Assistant Professor, School of Engineering Science, Simon Fraser University. The engineering component of the project is carried out in the Multimedia Communications Lab (MCL) at SFU. MCL is co-directed by Dr. Bajić and Dr. Liang. Dr. Bajić acts as the PI for the NSERC part of the project, and is involved in the development of low-delay video coding and super-resolution video display.
Dr. Jie Liang (NSERC Co-PI) Assistant Professor, School of Engineering Science, Simon Fraser University. Dr. Liang is the Co-PI for the NSERC part of the project, and leads the efforts in multi-view video coding and display.
Dr. Peter Borwein (Scientific Co-Investigator) Professor of Mathematics & Burnaby Mountain Chair. Dr Borwein is Executor Director of IRMACS. His research in classical analysis and number theory provides some of the analytic tools, mathematical and computational, for the project.
Resources
Imprint II
Imprint II is a daring, site-specific work for dancers, large music ensemble and telepresence technology. It explores the resonance of cultural artifacts and stories as transmitted from place-to-place, time-to-time, and person-to-person. Music by Owen Underhill is paired with Henry Daniel’s spectacular combination of modern technology and movement. Given the site-specific nature of the piece and subject matter, this second version of the project created for SFU Woodward’s and the Fei and Milton Wong Experimental Theatre is a profoundly transformed work from its precursor. Imprint II has almost entirely new choreography, new music for the TPE and music that has been expanded from 7 to 16 instruments. New languagescapes have also been created based on interviews with participants in Imprint II, and many other original elements including the boxes designed by visual artist Alan Storey and an installation in the Woodward’s Audain gallery designed by Majid Bagheri that the dancers interact with during the performance.
Beyond Boxes
Beyond Boxes (2011) is a short documentary film on the research and rehearsal process surrounding Imprint II (2010), a choreographic work by Henry Daniel with music by Owen Underhill. The film itself was directed and edited by Sonia Suvagau with cinematography by Felix Oltean and an original sound score by Sebastian Hugeneck.
Imprint I
An interdisciplinary work created by choreographer Henry Daniel and composer Owen Underhill for the Celebration of Creativity and the launch of the new MOA. This live performance took place as part of the opening day celebrations at 7:00 pm on Saturday January 23, 2010. A collaborative process involving the principal creators, MOA Curator Jill Baird, Kwakwaka’wakw participants Chief Robert Joseph and William Wasdan, and the Turning Point Ensemble, has led to a unique contemporary work that responds and extrapolates from Kwakwala language and song to new music, dance and technology. The intention of the creators is to explore the imprint of cultural artifacts and stories as transmitted from place to place, time-to-time, and person-to-person.
T2
"T2: Telepresence, Co-Location and Simultaneity". The choreographic work T2 explores notions of co-location and simultaneity in dance using telepresence technology.
It is an attempt to break up and redistribute the continuous flow of images audiences are accustomed to seeing onstage, the aim being to have the viewer reconsider not only what the dance means, but also how the images we are accustomed to become strange and even alien in these new settings.
T2_echo
Touched
Touched is a performance research project that investigates the twin concepts of Creation and creativity. It features human beings and semiautonomous robotic systems as “performers” in a constructed environment. The work takes two popular images from two very different religious/philosophical traditions as its starting point. The first is a detail from Michelangelo’s fresco on the ceiling of the Sistine Chapel, The Hands of God and Adam. This image symbolizes the transference of the spark of consciousness from the divine to the human in the Christian religious tradition. The second image is the dancing form of Lord Shiva in the Nataraj. Shiva, a key figure in Hindu religion, represents the source of all movement in the cosmos as (s)he dances the world into and out of existence.
Scientists Richard Vaughan (robotics) and Arthur Kirkpatrick (haptic interfaces) from Simon Fraser University’s School of Computing Science, guest media artists Mark Coniglio and Dawn Stoppiello of Troika Ranch (NYC), and choreographer Henry Daniel from the School for the Contemporary Arts collaborate to bring this new work to the stage.
