Synthetic data has frequently been used for a range of computer vision tasks, such as object identification, scene comprehension, eye tracking, hand tracking, and complete body analysis. However, the development of full-face synthetics for face-related machine learning has been substantially hindered by the difficulty of modeling the human skull. Although realistic digital humans have been produced for films and video games, each character typically requires much artistic time. Because of this, the synthesis of facial training data in literature has been accompanied by simplifications or a focus on specific facial features, like the area around the eyes or the hockey mask.
Due to the disparity in distributions between actual and artificial face data, generalization is difficult due to the domain gap. Because of this, it is believed that synthetic data cannot wholly replace real data for jobs that must be performed in the field. Domain adaptation and adversarial domain training, where models are urged to ignore domain differences, have been the essential methods to close this domain gap.
Procedural sampling can generate and render innovative 3D faces at random without human assistance. The technology does this by overcoming a significant scaling limitation in the methods used by the Visual Effects (VFX) industry to synthesize lifelike individuals. By creating synthetic looks with unparalleled realism, Wood et al. aimed to directly address the issue by minimizing the domain gap at the source. Their approach procedurally blends a parametric 3D face model with a vast collection of excellently crafted elements from artists, such as textures, hair, and apparel.
Machine learning algorithms trained on the synthetic data for landmark localization and face parsing achieved performance on par with the state-of-the-art without employing a single genuine image. However, one drawback of this technique is the absence of dynamic, expression-dependent wrinkles. In this paper, they provide an easy-to-use but efficient plan for implementing expression-based wrinkles. The method creates textures only from neutral-expression scans, which stay static during all expression-related deformations of the underlying face geometry.
Their main goal is to use high-resolution scans of posed faces to extract intricate wrinkling effects for identification. To store these potential wrinkles, they create wrinkle maps for albedo and displacement textures. They mix the crease and neutral surfaces during synthesis for any expression other than those depicted in the source scans, utilizing the tension in the face mesh to create dynamic wrinkling effects.
Animations are also a part of the Supplementary Material. Early VFX techniques initially utilized wrinkle maps to describe artist-defined bumps or normal maps for imitating dynamic wrinkles. These strategies, however, have three flaws. First, the shadows and silhouettes, essential for face-related tasks like landmark localization, are unaltered and imitate changes in underlying geometry. Second, neither the approaches impact the diffuse textures nor the albedo. Scale is the last and most significant flaw.
The techniques involve manually creating wrinkle maps and masks for each character’s blendshape. In contrast, their automated mesh-tension-driven approach incorporates genuine wrinkles for albedo and displacement textures from scans and organically grows with the number of identities and expressions. Additionally, they manage identities without expression scans by copying believable wrinkles from the most comparable neutral textures.
They make the following specific contributions to further the development of synthetics for face-related tasks:
- A readily scalable system with growing identities and expressions for dynamic, expression-based wrinkles.
- An empirical comparison of surface-normal estimation and face key point localization performance vs. the SOTA synthetics system.
- New assessment measures and data for locating critical points in the area of the eye where wrinkles are significant for learning activities
This Article is written as a research summary article by Marktechpost Staff based on the research paper 'Mesh-Tension Driven Expression-Based Wrinkles for Synthetic Faces'. All Credit For This Research Goes To Researchers on This Project. Check out the paper and project.
Please Don't Forget To Join Our ML Subreddit
Aneesh Tickoo is a consulting intern at MarktechPost. He is currently pursuing his undergraduate degree in Data Science and Artificial Intelligence from the Indian Institute of Technology(IIT), Bhilai. He spends most of his time working on projects aimed at harnessing the power of machine learning. His research interest is image processing and is passionate about building solutions around it. He loves to connect with people and collaborate on interesting projects.