FaceSnap: Real-Time Personalized Lightstage Facial Performance Capture

Rukhshanda Hussain^1,2, Noé Artru^1,2, Emeline Got^2,*, Luiz Gustavo Hafemann^2,*, Alexandre Messier², Brandon Dearlove², Rafael M. O. Cruz^1,†, Abdallah Dib^2,†, Eric Granger^1,†

¹École de Technologie Supérieure ²Ubisoft La Forge

^*This work was done while the authors were at Ubisoft La Forge. ^†Equal supervision.

Accepted to WACV 2027

Paper (coming soon) arXiv Benchmark

TL;DR: FaceSnap enables real-time personalized, production-quality facial performance capture from a single monocular camera using a one-time lightstage personalization step — 83 FPS, 4K appearance.

Overview

FaceSnap — a fully automated, end-to-end pipeline for facial performance capture. From a one-time multi-view Lightstage capture: (i) multi-stage optimization obtains registered facial animation sequences, (ii) a personalized model is trained to estimate geometry and dynamic appearance, and enables (iii) real-time monocular 4D face capture of new performances at 4K (83 FPS).

Geometric Fidelity & Skin Detail

Reference View S2 reference view

Zoom In

Reference View S1 reference view

Zoom In

FaceSnap captures clean edge flows for geometry in challenging expressions and recovers fine skin detail, from pores to wrinkles — zoom in for details.

One Time Optimization

FaceSnap recovers faithful geometry with fine details and accurate lip movements for challenging expressions in range of motion, on both Multi4D and our custom Lightstage dataset.

Real Time Tracking

Once the personalized model is built, FaceSnap produces production-quality meshes (at 0.6 ms per frame) and 4K textures (2.9 + 8.5 ms) — 25,000× faster than traditional artist-dependent production pipelines.

Comparison of Geometry Reconstruction with State-of-the-Art Methods

FaceSnap's one-time optimization recovers geometry that is more faithful to the subject's true shape than monocular feed-forward models (DECA, SEREP) and per-frame multi-view fitting methods (FLAME fitting, Topo4D).

Comparison of Appearance Prediction with State-of-the-Art Methods

FaceSnap recovers fine-grained wrinkles and pore-level details in 4K that general upscalers (e.g., ESRGAN fine-tuned on face data) struggle with. FaceSnap also achieves perceptual comparability to multi-camera optimization methods like Topo4D, which directly bake the texture from multi-view input.

Multi4D Benchmark

We introduce Multi4D, a benchmark based on Multiface that enables direct geometry comparison using each method’s native mesh topology, without requiring retargeting to a common topology. Multi4D establishes a standardized protocol for evaluating dynamic 4D facial reconstructions on range-of-motion sequences, facilitating fair, topology-invariant comparisons between approaches.

To obtain the raw reference meshes used for evaluation, we apply a photogrammetry tool on 38 camera views from Multiface, providing 360-degree coverage around the head. Multi4D comprises six identities, with approximately 1,100–1,500 frames per subject. We release high-resolution raw meshes with approximately 240k vertices and 399k faces per frame, together with camera parameters and 1K-resolution textures. We also provide benchmark evaluation code, demo data for one sample frame to demonstrate point-to-surface distance calculation, and data visualization scripts.