Is this an avatar I see before me? Audience takes to stage in virtual Shakespeare play

Video game software enables actors to interact with viewers
in real-time fantasy

Playhouses may be dark, but the Royal Shakespeare Company is to bring immersive theatre to audiences wherever they are in the world by combining revolutionary virtual reality and video games technologies with traditional live performances.

Real-time animation will be created by actors interacting with a live audience, who will in turn influence a story inspired by A Midsummer Night’s Dream.

Mischievous Puck and his fellow fairies will be conjured up by actors whose every movement will be translated by cutting-edge technology into virtual avatars that mirror them as extraordinary creatures.

A fantasy world inspired by Shakespeare’s imagination will be recreated on an audience’s computers and mobile phones in a way that allows them to interact directly with the narrative and music. They could, for example, become an iridescent firefly, lighting up a virtual enchanted forest – along with thousands of other fireflies – and guiding Puck along the way. Or they could trigger layers of instrumental sound, adding to a symphonic score performed by the London-based Philharmonia Orchestra.

In a pioneering project, to be announced tomorrow, specialists from the worlds of theatre, music and gaming have shared their expertise, developing a “prototype” that could be adopted for future live performances.

In a groundbreaking RSC production of The Tempest in 2016, actor Mark Quartley wore sensors that captured his movements, rendering Ariel, the sprite, as an animated ethereal character on stage.

It used real-time performance-capture technology that has developed dramatically since then. Now – in a production entitled Dream, to be premiered next month – motion capture and facial rigging will reproduce the movements of seven RSC performers for virtual avatars that are more detailed, including faces and hands, which could not be achieved for Ariel.

RSC artistic director Gregory Doran
RSC artistic director Gregory Doran: ‘There’s so much opportunity. It’s as if technology is now providing us with this amazing paint-box.’ Photograph: RSC

Unlike a regular live stream, this is interactive, showing how the excitement of a live performance can be recreated beyond the confines of a physical location.

Headed by the RSC, the project’s key players include the Philharmonia Orchestra, Marshmallow Laser Feast, virtual reality creators, and Epic Games, developers of the Unreal Engine video games software.

The RSC’s artistic director, Gregory Doran, spoke of his excitement at the project’s potential.

He told the Observer: “This is a 21st-century reimagining of Shakespeare’s play, which is giving us a completely different vocabulary of imagery. That’s extraordinary. When we did The Tempest in 2016, we explored the opportunities that the digital world could give to us.

“I really did think then – quoting the play – this is a ‘brave new world’. There’s so much opportunity out there. It’s as if technology is now providing us with this amazing paintbox. We have a lot of very exciting talent that can find uses for those new tools.”

He added: “What I love about this potential for an interactive experience is it’s not the play of A Midsummer Night’s Dream, but it’s the world of A Midsummer Night’s Dream. If that makes you want to read or watch this play, then that’s great.”

He recalled that in his youth he “got hooked on the Greek classics” by watching the extraordinary creatures created by Ray Harryhausen, who pioneered animation effects in films such as Jason and the Argonauts, the 1963 masterpiece.

He believes that the technology could inspire interest in the theatre and Shakespeare from young people who are already so familiar with the world of video games: “This gives audiences of different ages a shared point of connection across the generations and, if that experience also has Shakespeare’s imagination behind it, that’s got to be a good thing.”

Luke Ritchie, the Philharmonia’s head of innovations, said: “We’ve never had a collaborative process like this. It’s a game-changer. You’ll be seeing real-time live animation which is being made live by live performers on a stage, creating virtual beings.”

Sarah Ellis, the RSC’s director of digital, said: “We are expanding our theatre-making toolkit. This is a pioneering piece of stage production. It allows us to look at exciting possibilities.”

She added: “One of the things we were most proud of with The Tempest was the inter-generational moment where we had young and old audiences responding to it. Through gaming technology we’re now creating a piece of live performance, not a game – and we hope it will once again bring together generations, either through the Shakespeare or the technology.”

The production will include two new orchestral pieces by Esa-Pekka Salonen, the Philharmonia’s principal conductor, and by Swedish composer Jesper Nordin, creator of a groundbreaking interactive music software called Gestrument, which enables Dream to have an interactive musical soundtrack affected by the actors’ movements.

Dream is an “Audience of the Future Demonstrator” project, part of the government’s Industrial Strategy Challenge Fund. It will be performed between 12 and 20 March. Audiences can either buy a £10 ticket, to interact directly, or view the performance passively for free. Tickets will be available via


Dalya Alberge

The GuardianTramp

Related Content

Article image
‘Mine’s lobster and champagne’: Judi Dench reveals secret onstage supper at Shakespeare play
The Oscar-winning star discusses her Shakespearean roles and the characters she brought to life in watercolours in her new book

Dalya Alberge

18, Jun, 2023 @6:00 AM

Article image
A happy ending for King Lear? Trauma of plague caused Shakespeare to change play’s finale
The Bard, like us, lived through a period of trauma. One expert now believes it coloured his later plays

Vanessa Thorpe

13, Dec, 2020 @8:45 AM

Article image
‘This is another crack in the glass ceiling’: RSC casts disabled actors in new season
Artistic director Greg Doran hopes to ‘reflect the nation’ in boost to stage diversity

Dalya Alberge

26, Jan, 2019 @12:00 PM

Article image
Gregory Doran: ‘Shakespeare defines things when you can’t’
As the RSC’s artistic director steps down after 35 years and 50 shows, he talks about the productions he’s most proud of, his new memoir, and the loss of his husband, the actor Antony Sher

Kate Kellaway

23, Apr, 2023 @12:00 PM

Article image
RSC gives musicians a central role at Stratford
Evelyn Glennie scores Troilus and Cressida as musicians get greater prominence in next Shakespeare season

Dalya Alberge

16, Sep, 2018 @6:00 AM

Article image
Shakespeare gets the starring role in cultural celebration alongside Olympics
Experts ask if the Bard is Britain's only exportable brand as leading organisations recruit playwright for Games

Vanessa Thorpe

28, May, 2011 @11:07 PM

Article image
Simon Russell Beale: it’s fine to take liberties with Shakespeare
Acclaimed actor says we shouldn’t be purists about the text – as long as it still makes coherent, emotional sense

Dalya Alberge

14, Feb, 2015 @10:00 PM

Article image
Shakespeare inspired to write Othello after being booed off stage
The bard learned from his humiliating experiences as an actor at the Globe to hone his playwright skills, academic says

Dalya Alberge

27, Aug, 2022 @12:00 PM

Article image
Royal Shakespeare Company under fire for not casting enough Asian actors
RSC criticised for producing The Orphan of Zhao, regarded as Chinese equivalent of Hamlet, with predominantly white cast

Matt Trueman

19, Oct, 2012 @3:43 PM

Article image
I don’t care who wrote Shakespeare, says RSC artistic director
Theatre to stage adaptation of Henry VI series, the subject of disputes over authorship

Mark Brown Arts correspondent

04, Feb, 2020 @2:30 PM