DRA 3907HF: Theatre and Emerging Technologies
CSC 2524: Topics in Interactive Computing: Graphics, Interaction and Performance in Immersive Environments (AR/VR/XR)
Ariel (Coleen MacPherson) engages the audience in interactively creating the sounds of the island as Prospero (Trevor Jablonowski) sleeps. Selections from The Tempest, class production for the graduate course “Collisions and Common Ground: Art - Technology – Performance.” BMO LAB for Creative Research in AI, the Arts and Performance, Centre for Drama, Theatre and Performance Studies, Luella Massey Studio Theatre, University of Toronto. Toronto, 2019. (Photo by David Rokeby)
This interdisciplinary graduate course brings together scholars, artists, and students from Drama/Theatre, Visual Studies, Music, Comparative Literature, Engineering and Computer Science who are interested in the intersection between theatre and emerging technologies. The course is taught: David Rokeby (media artist and Director of the BMO LAB) and Professors Pia Kleber (Drama/Theatre/Comparative Literature) and Karan Singh (Computer Science).
In the class, we explore technologies such as Augmented Reality, Virtual Reality, real-time motion tracking and motion capture, video and depth sensor-based immersive installations using multi-channel sound and video projection, robotics and Artificial Intelligence. The class designs and implements theatrical scenes, animated and augmented through the use of the explored technologies for public presentation at the end of the term. These productions will be directed by Brendan Healy (Artistic Director of Canadian Stage) and Alan Dilworth (Artistic Director of Necessary Angel).
For students coming from an arts background, the course offers direct experience working with immersive and emerging technologies and a chance to explore their applications to their research. For students with a technology background, the course provides instruction into interactive graphics techniques in immersive environments and the opportunity to integrate their research into an art-based publicly presented project. The course exposes all of the students to rigorous interdisciplinary practices and their conceptual, practical and theoretical challenges through group discussions, concept generation, practical experimentation and research, and engagement with visiting artists. We focus on establishing a successful engagement across disciplinary lines, encouraging good communication and an atmosphere of respect and trust with those who think, work, define objectives and measure success in different ways.
Augusto Bitter rehearses his role as Ariel creating the island in Virtual Reality for a presentation of scenes from the Tempest, class production for the graduate course “Collisions and Common Ground: Art - Technology – Performance.” BMO LAB for Creative Research in AI, the Arts and Performance, Centre for Drama, Theatre and Performance Studies, Luella Massey Studio Theatre, University of Toronto. Toronto, 2018.
Ariel (Coleen MacPherson) spreads fire throughout the ship through her digitally-tracked body movements. Selections from The Tempest, class production for the graduate course “Collisions and Common Ground: Art - Technology – Performance.” BMO LAB for Creative Research in AI, the Arts and Performance, Centre for Drama, Theatre and Performance Studies, Luella Massey Studio Theatre, University of Toronto. Toronto, 2019. (Photo by David Rokeby)
The goals of this course are to:
- Encourage creative applications of new digital technologies in the arts with applications for theatre, performance and installation using technologies including but not limited to real-time motion capture, robotics, artificial intelligence, and interactive sound and image technologies for dynamic staging
- Encourage a balance between enthusiasm and criticality in these explorations;
- Develop effective cross-disciplinary practice and an understanding of the challenges it poses, and encourage the cultivation of mutual respect across the disciplines;
- Challenge students to venture outside their field of expertise to gain the experience and language necessary to communicate effectively across disciplinary boundaries;
- Stress-test emerging technological research outside the lab in an unpredictable and unbounded environment;
- Encourage consideration of the potentials, challenges, and theoretical and ethical implications of these technologies beyond the scope of the course and its projects;
- Develop a cohort of students experienced with the arts and the technologies and comfort with working across disciplines for future collaborations.
- Teach students the fundamental principles behind the technology, perception and interactive techniques for immersive environments AR/VR/XR
- Read, understand, present and critique state of the art research papers relevant to the course
- Build prototypes to enable theatrical performance using immersive technology
David Rokeby, Director of the BMO LAB, Experimental Interactive Installation Artist, software and hardware developer
Office: BMO lab
Prof. Pia Kleber, Drama & Comparative Literature
Helen and Paul Phelan Chair in Drama
Office: UC, A302
Prof. Karan Singh, Computer Science
- General Introduction to Course
- Introduction of Faculty and Students
- Discussion of the Structure of the Course, the Assignments and process of Evaluation
- Introduction to historical and contemporary ways of representing rain on stage.
- An initial tutorial on the MaxMSP software that we use for a lot of our interaction design
- Reading: David Chapman - ‘How should we evaluate progress in AI?’
- 12:00-2:00: presentation by Friedrich Kirschner, director of puppetry and software developer at the Ernst Busch University of Performing Arts, Berlin
- Introduction to the work and methodologies of theatre director Robert Wilson
- Class trip to the production of Puccini's Turandot directed by Robert Wilson at the Canadian Opera Company
Robert Wilson in Conversation
- 12:00-2:00: Conversation between Alexander Neef (Director of COC) and Robert Wilson with reception following (at BMO Lab)
Augmented and Virtual Reality
- Presentation of documentation of existing performance work using AR and VR
- Experiential presentation of AR and VR
- Discussion of applications of AR and VR to our production
- An initial tutorial on the Unity software for 3D world design for VR.
- Reading: Steve Dickson: 'A History of virtual reality in performance'
- Documentation of relevant existing performance work using Projection Mapping
- Experiential presentation of Projection Mapping technology
- Discussion of applications of Projection Mapping to our production
- Reading: Steve Dickson: 'Multimedia Theater 1911-1959'
Interaction and Immersive Environments
- Documentation of existing works
- Experiential presentation of example immersive technologies
- Discussion of applications of interaction and immersion to our production
- Reading: Robert Wechsler: 'Artistic Considerations in the Use of Motion Tracking'
- Documentation of existing works using sound in performance
- Experiential presentation of example sound technologies
- Discussion of applications of sound to our production
- Documentation of existing works using motion capture in performance
- Experiential presentation of example motion capture technologies
- Discussion of applications of motion capture to our production
- Reading: Kaisu Koski - ‘Performing an Avatar’
Students are encouraged to continue developing their ideas and implementations for productions in their groups during Reading Week.
Production Refinement and Rehearsal
Production Refinement and Rehearsal
Production Refinement and Rehearsal
Public presentation of Production of the Chosen Scenes