It seems that the depth-sensing front camera on the iPhone X is good for more than just unlocking your device and animating emojis. Apparently, it can also be used as a low-cost animation tool for game developers.
Next Games has created an iPhone app that can be used to capture facial data so that it may be used on a 3D model within a game. The studio offered The Verge a demonstration of the tool at GDC 2018 using their upcoming game The Walking Dead: Our World (autoplay video).
Typically there are two ways to animate a game character. The first is to do it using manual animation where an artist sets the movements frame by frame. This process is very time-consuming, especially when doing facial animations.
The second option is to use motion capture rigs. Animations using mo-cap is much faster than manual animation but requires specialized equipment that is prohibitively expensive and bulky.
"We're using the power of the iPhone X, combined with ARKit's facial recognition tech, to do what was once a time-consuming undertaking."
With Next’s new tool, animators can quickly capture multiple facial animations just using an iPhone X. The app utilizes an Apple ARKit API that gives developers access to the facial capture capabilities of the camera. Face animations can be recorded and then imported into Unity to be used on any model.
Not only that, Next plans to implement the tool into the Our World interface to allow players to create avatars that look like them that can then be used in the social aspect of the game. Players may also be able to animate zombies and upload them for others to use. Of course, this feature will be exclusive to the iPhone version and presumably only for X users.
Our World is a location-based AR game that is slated for a second-quarter launch.
Since the API is openly available from Apple, it is quite likely that we will see other developers creating similar tools and coming up with unique ways to incorporate the feature into future games.