Game of Thrones in Augmented Reality (Unity C# scripting, Maya 3D modeling and animation)

As showcased in:
NYU IDM Spring Showcase 2017
May 3, 2017
Media and Games Network, 2 MetroTech Center, Brooklyn, NY, 11201 US

NYC Media Lab Summit 2017
September 28, 2017
The New School, Parsons School of Design, 55 W 13th Street New York, NY 10011 US

BRIC Arts Media: BK to the Future
September 23, 2017
647 Fulton Street Brooklyn, NY 11217 US

Exploring Future Realities
November 30, 2017
NYU Rosenthal Pavilion 60 Washington Square South, New York, NY 10012 US

Augmented Reality Asia Expo 2017
China National Conference Center, Beijing, China
July 19-21, 2017
No.7 Tianchen East Road, Chaoyang District, Beijing, China

This slideshow requires JavaScript.

Social Media:

Software: Unity, Vuforia SDK, MonoDevelop
Programming Languages: C#
Hardware: Android
Download the AR marker
Download GOT AR APK

Together with Subigya Basnet, a fellow graduate student at NYU, we created an augmented reality experience for Game of Throne fans. The Game of Thrones series is rich in narrative, and the core concept is to produce an augmented reality experience using 3d assets and animation. Our inspiration comes from putting in the history and lore series made by fans which are supplementary to those who watch the HBO series.   Here, stories of how dragons came to be, how the old Valyria was destroyed by doom and how Aegon Targaryen conquered the seven kingdoms leading to the usurp the iron throne are popular themes that were told in History and Lore. We then decided to construct a first prototype of an interactive map.

Technology

The best way to create our AR experience is through Unity game engine and the Vuforia plugin, one of industry’s leading plugins that deliver marker augmentation. We wanted the marker to be part of the story and be seamless in terms of User Experience.

The end goal is to use a 24″x36″ physical map of Westeros.

Concept

Below is an example of the 3d animation integrated into Unity. The tree is a generative script.

To those familiar with the opening credits of Game of Thrones, this is an extremely iconic scene that we hope to recreate in augmented reality.

prototype

Interactive Wireframes
//invis.io/5FAZYGEW4

3D Asset Production

We’re using Maya to create the 3D models for each of the key places in Westeros. One of our design considerations is to keep the number of polygons or faces at a given time at 100,000 maximum. Anything beyond that will crash most devices and Unity. It’s our goal to keep the models with an optimal number of polygons but still keeping the aesthetics good.

To illustrate the 3D modeling process, here is an example using The Wall. The goal of this 3D model is to keep it as low poly as possible while still maintaining the aesthetic appeal that Game of Thrones viewers recognize.

I started from this shot from the intro of the TV show.

got_wall_final

I chose this shot because of its potential to be a low poly model, and with a low poly model, we’ll be able to allocate what we have saved for other important models such as the model of a dragon in our AR application.

I modeled the Wall using Maya. The textures are applied in Unity to make our process more efficient.

thewall_maya

Integrating the FBX model into Unity, here is the following result:

giphy (1)

Dragon Animation

We also had to customize animations for the sigils and also the glide of the dragon. Animations are done in every bone of the dragon’s tail to create a subtle glide. This is done using keyframe animation on Maya.

Integrating into Unity

Over the past couple of weeks, we started integrating the 3D assets into Unity 3D game engine. We used the leading marker-based technology Vuforia by PTC to implement the marker-based tracking.

Overhead

Our prototype is composed of two levels: Exploration and Narrative. The Exploration phase allows the user to discover the world of Westeros and know more about the backgrounds of each house (ie. Lannister, Martell, Tyrell, Targaryen, Stark, Arryn, Tully) The narrative level introduces the user to the story of how Aegon Targaryen conquered Westeros, that became the trigger for the happenings in the TV show.

The narrative animation in Unity is also done using custom scripts in C#.

Monodevelop_GOT

Sound Design

Sound can truly bring the experience to life, beyond animation. It creates the illusion and enhances the imagination of the user, as if he is actually inside Westeros. With this, I integrated the background song of Game of Thrones, not Ramin Djawadi’s iconic Game of Thrones original theme, but a music that serves to heighten the anticipation of the user as he goes along in the journey. Along with this is the voice over of my colleague narrating the story, and sound effects that I have designed and mixed to bring it to life. Below is the sample of my first prototype that I created.

Final Project

After 2 months of non-stop coding, 3D modeling and animating dragons, we finally finished the Game of Thrones in Augmented Reality app! To complement the amount of work we have put in, we decided to produce a trailer, which included us incorporating green screen technology and produced a Behind the Scenes footage as well. During the NYU IDM Spring Showcase, we received positive feedback to push this project beyond the world of Game of Thrones.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s