![\"Flame](\"assets/images/post003-flame-fitting-pipeline.png\")
Figure 1: Fitting FLAME to an unconstrained single image.
In this post, we explore how to recover the 3D geometry of a human face using the FLAME model [1]. FLAME is a parametric 3D face model trained on a large dataset of diverse human faces. It represents faces with a compact set of coefficients for identity, expressions, and head pose, making face reconstruction a problem of regressing the right parameters rather than building a mesh from scratch. This approach is both versatile and robust, capable of capturing fine variations efficiently.
We focus on single-image fitting with FLAME, using the open-source flame-head-tracker [3] codebase. This repository provides a modular pipeline for both video-based tracking and single-image fitting. Our discussion centers on fitting the FLAME model to recover a 3D representation from a single unconstrained image.
Figure 1: Fitting FLAME to an unconstrained single image.
FLAME is a mesh-based parametric model that represents the human face using approximately 5023 vertices and 9976 faces. Instead of reconstructing a dense mesh from scratch, FLAME encodes each face as a compact set of parameters controlling shape, expression, and pose:
![\"Nearest](\"assets/images/post002_flame_model_variations.gif\")
Figure 2: FLAME variation [2].
This approach enables efficient and realistic modeling of facial geometry. However, FLAME is designed for bare facial structure and does not explicitly model hair, glasses, or other accessories, which can introduce challenges when fitting to real images.
Fitting a 3D model like FLAME to a single image is difficult due to limited cues:
These factors mean single-image fitting must rely on strong priors, landmarks, and optimization rather than raw pixels alone.
The flame-head-tracker pipeline for single-image fitting consists of three main steps:
Initialization: Pre-trained models DECA [5] and MICA [6] provide initial estimates of FLAME parameters—DECA for expressions, pose, texture, and lighting; MICA for identity shape coefficients. This initialization step is crucial: a good starting point significantly accelerates convergence and improves the stability of the optimization process.
Before fitting the FLAME model, the input image must be carefully prepared to reduce noise and isolate the relevant facial information. The preprocessing step includes:
As a first step, matting is applied to remove the background, isolating the human subject for more accurate and focused subsequent processing.
![\"FLAME](\"assets/images/post003-mat-figure.png\")
Figure 3: Matting applied to the input image.
The face is detected and cropped, then aligned to a canonical orientation. This normalization helps the model deal with variations in scale and rotation.
![\"Aligned](\"assets/images/post003-aligned.png\")
Figure 4: Example of face alignment before FLAME fitting.
Using MediaPipe, dense 2D facial landmarks are extracted. These serve as geometric anchors for initializing and optimizing the 3D model.
A segmentation step separates facial regions from non-facial elements such as hair, background, or clothing. By ignoring these irrelevant pixels, the fitting process becomes more robust against occlusions.
![\"Face](\"assets/images/post003-parsing-mask.png\")
Figure 5: Example of face parsing mask used in preprocessing.
Together, these steps ensure that the pipeline starts with a clean, well-aligned face region where key features are accurately localized. This significantly improves the reliability of the subsequent initialization and optimization stages.
Once the FLAME parameters are initialized, the fitting process is refined through a two-stage optimization strategy. This approach first secures the global alignment between the 3D model and the image, then fine-tunes local details for realism.
The first stage focuses on aligning the FLAME model with the input image using detected landmarks. Here, the system estimates the 6 degrees of freedom (6DoF) camera pose — yaw, pitch, roll, and 3D translation (x, y, z).
With the camera fixed, the second stage optimizes the full set of FLAME parameters, including:
This stage balances accuracy and stability by applying learning rates and regularization tuned for each parameter group.
| Parameter | Variable Name | Learning Rate | Regularization |
|---|---|---|---|
| Expression | d_exp | 0.01 → 0.005 | L2: 0.025 |
| Jaw Pose | d_jaw | 0.025 → 0.01 | – |
| Eye Pose | eye_pose | 0.03 → 0.01 | – |
| Shape (photometric) | d_shape | 0.001 | L2: 0.1 |
| Texture (photometric) | d_tex, d_texture | 0.005 | – |
| Lighting (photometric) | d_light | 0.005 | – |
Through this staged refinement, the system moves from a coarse but stable alignment to a detailed and realistic reconstruction, while preventing overfitting to noise or occlusions in the input image.
The fitting process can be performed in two modes:
The outcome of single-image fitting is a set of FLAME parameters that reconstruct the 3D mesh of the input face, capturing identity, expression, pose, texture, and lighting.
![\"Single-image](\"assets/images/post003-output_total.png\")
Figure 6: Example of single-image FLAME fitting. Input image (left) → fitted FLAME mesh (right).
Limitations:
We walked through fitting the FLAME model to a single image using the flame-head-tracker pipeline, covering preprocessing, initialization, optimization, and fitting modes.Applications include avatars, telepresence, facial animation, healthcare, and biometrics.
By combining parametric priors with modern optimization and learning techniques, FLAME remains a strong foundation for 3D face reconstruction research and applications.
[1] Li, Tianye, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. “Learning a model of facial shape and expression from 4D scans.” ACM Trans. Graph. 2017.
[2] https://github.com/TimoBolkart/FLAME-Universe
[3] https://github.com/PeizhiYan/flame-head-tracker
[4] https://github.com/huseyintemiz/flame-head-tracker (my fork with educational materials)
[5] DECA: Feng, Yao, Haiwen Feng, Michael J. Black, and Timo Bolkart. “Learning an Animatable Detailed 3D Face Model from In-The-Wild Images.” ACM Transactions on Graphics (Proc. SIGGRAPH), vol. 40, no. 8, 2021.
[6] MICA: Zielonka, Wojciech, Timo Bolkart, and Justus Thies. “Towards Metrical Reconstruction of Human Faces.” European Conference on Computer Vision, 2022.
", "url": "https://temiz.ai/recovering-human-face-geometry", "date_published": "2025-09-22T00:00:00+03:00", "date_modified": "2025-09-22T00:00:00+03:00", "author": { "name": "Hüseyin Temiz", "url": null, "avatar": null } }, { "id": "https://temiz.ai/3d-gaussian-splats-avatars-utilizing-uv-mapping", "title": "3d Gaussian Splats Avatars Utilizing Uv Mapping", "summary": null, "content_text": "In this blog post, we examine how four recent methods utilize UV mapping within Gaussian Splat avatar pipelines. Many modern approaches convert the 3D mesh representation into a 2D UV-mapped domain to address the limitations of working directly with 3D meshes. By leveraging UV mapping, these pipelines achieve more uniform sampling, intuitive 2D editing, and efficient data compression—advantages not easily attainable with raw 3D geometry. Each method in the literature integrates UV mapping differently to maximize these benefits. Here, we explore and compare their strategies.UV Mapping in Each Method Method How UV Is Used FlashAvatar [1] • Canonical field initialization by sampling Gaussians uniformly over the UV map yields a far more even distribution across face, hair, and neck regions.• Mesh-driven motion uses UV→barycentric mapping to attach Gaussians to mesh and apply dynamic offsets. FATE [2] • Sampling-based densification ensures optimal Gaussian placement by sampling uniformly in UV space.• Neural baking converts discrete Gaussians into continuous attribute maps (position, color, opacity) in UV, enabling intuitive texture-style editing of the splats. GEM [3] • Gaussian eigenbasis construction: a UNet predicts multi-channel Gaussian parameter maps in the FLAME UV domain, where each pixel encodes one splat’s full parameters.• Linear distillation builds compact eigenbases over these UV maps for real-time synthesis. MeGA [4] • UV displacement map driven by FLAME parameters captures per-vertex geometric detail.• Neural textures (diffuse, view-dependent, dynamic) live in UV space for high-fidelity appearance.• Deferred rendering rasterizes the refined UV mesh + textures before blending with Gaussian hair. Similarities & DifferencesUse of UV SpaceAll four methods decouple 3D Gaussian placement and attribute editing by projecting onto a shared 2D UV domain, avoiding the irregular sampling and distortion issues of raw mesh faces. Sampling vs. Baking FATE & FlashAvatar perform direct UV sampling to place Gaussians; FATE further bakes trained splats into continuous UV maps for editing. GEM uses UV as an intermediate training representation, distilling it into linear eigenbases, whereas MeGA uses UV both for geometry refinement (displacement) and texture creation. Mesh DependenceExcept for pure mesh-driven strategies, all UV-based pipelines still rely on a parametric mesh (usually FLAME) to establish UV→3D correspondences and drive animation via blendshapes or skinning. Attachment Strategies: FlashAvatar attaches splats directly to mesh vertices (via UV → barycentric weights) then learns offsets. FATE samples points purely in UV and reconstructs their 3D positions via barycentric projection. GEM requires the mesh only at inference to deform distilled eigen-coefficients. MeGA hybridizes: the mesh handles global facial motion, UV displacement refines geometry, and Gaussian splats model hair. Editing WorkflowUV domains unlock powerful 2D editing: painting, diffusion-model edits, or neural filters—far more intuitive than manipulating unstructured 3D point clouds. Continuous vs. Discrete: FATE creates continuous UV attribute maps, enabling pixel-space edits. GEM keeps discrete splats but trains via UV maps for compactness. MeGA splits geometry (UV displacement) from appearance (neural textures), offering fine-grained control in both domains. Advantages of UV Mapping over Direct Mesh Sampling Uniform Density ControlUV sampling lets you enforce an even coverage of splats (e.g., FlashAvatar), sidestepping the uneven vertex spacing or face-size distortions of direct 3D sampling. 2D Editing ToolchainAttribute maps in UV space can be edited with mature 2D tools—painting, diffusion models, or neural filters—making appearance tweaks as straightforward as texture work on polygonal models. Compact Representation & AccelerationUV maps lend themselves to image-based compression or CNN/UNet processing (as in GEM), enabling low-dimensional bases and lightning-fast inference on commodity hardware. Decoupling Geometry & AppearanceBy separating UV displacement for shape from UV textures for color and dynamics (MeGA), pipelines simplify rendering and editing complexity. Stable Barycentric MappingSampling in UV then projecting via barycentric coordinates (FATE) avoids mesh discretization artifacts and ensures smooth, differentiable optimization of Gaussian parameters. ConclusionIn sum, UV parameterizations transform complex 3D Gaussian splat operations into well-behaved 2D workflows, yielding more uniform sampling, intuitive editing, and powerful opportunities for compression and acceleration that mesh-only approaches simply cannot match.References[1] FlashAvatar: Xiang, Jun, et al. “Flashavatar: High-fidelity head avatar with efficient gaussian embedding.” CVPR 2024.[3] FATE: Zhang, Jiawei, et al. “Fate: Full-head gaussian avatar with textural editing from monocular video.” CVPR 2025.[4] GEM: Zielonka, Wojciech, et al. “Gaussian eigen models for human heads.” CVPR 2025.[2] MEGA: Wang, Cong, et al. “Mega: Hybrid mesh-gaussian head avatar for high-fidelity rendering and head editing.” CVPR 2025.", "content_html": "In this blog post, we examine how four recent methods utilize UV mapping within Gaussian Splat avatar pipelines. Many modern approaches convert the 3D mesh representation into a 2D UV-mapped domain to address the limitations of working directly with 3D meshes. By leveraging UV mapping, these pipelines achieve more uniform sampling, intuitive 2D editing, and efficient data compression—advantages not easily attainable with raw 3D geometry. Each method in the literature integrates UV mapping differently to maximize these benefits. Here, we explore and compare their strategies.
| Method | How UV Is Used |
|---|---|
| FlashAvatar [1] | • Canonical field initialization by sampling Gaussians uniformly over the UV map yields a far more even distribution across face, hair, and neck regions. • Mesh-driven motion uses UV→barycentric mapping to attach Gaussians to mesh and apply dynamic offsets. |
| FATE [2] | • Sampling-based densification ensures optimal Gaussian placement by sampling uniformly in UV space. • Neural baking converts discrete Gaussians into continuous attribute maps (position, color, opacity) in UV, enabling intuitive texture-style editing of the splats. |
| GEM [3] | • Gaussian eigenbasis construction: a UNet predicts multi-channel Gaussian parameter maps in the FLAME UV domain, where each pixel encodes one splat’s full parameters. • Linear distillation builds compact eigenbases over these UV maps for real-time synthesis. |
| MeGA [4] | • UV displacement map driven by FLAME parameters captures per-vertex geometric detail. • Neural textures (diffuse, view-dependent, dynamic) live in UV space for high-fidelity appearance. • Deferred rendering rasterizes the refined UV mesh + textures before blending with Gaussian hair. |
All four methods decouple 3D Gaussian placement and attribute editing by projecting onto a shared 2D UV domain, avoiding the irregular sampling and distortion issues of raw mesh faces.
Sampling vs. Baking
Except for pure mesh-driven strategies, all UV-based pipelines still rely on a parametric mesh (usually FLAME) to establish UV→3D correspondences and drive animation via blendshapes or skinning.
Attachment Strategies:
UV domains unlock powerful 2D editing: painting, diffusion-model edits, or neural filters—far more intuitive than manipulating unstructured 3D point clouds.
Continuous vs. Discrete:
Uniform Density ControlUV sampling lets you enforce an even coverage of splats (e.g., FlashAvatar), sidestepping the uneven vertex spacing or face-size distortions of direct 3D sampling.
2D Editing ToolchainAttribute maps in UV space can be edited with mature 2D tools—painting, diffusion models, or neural filters—making appearance tweaks as straightforward as texture work on polygonal models.
Compact Representation & AccelerationUV maps lend themselves to image-based compression or CNN/UNet processing (as in GEM), enabling low-dimensional bases and lightning-fast inference on commodity hardware.
Decoupling Geometry & AppearanceBy separating UV displacement for shape from UV textures for color and dynamics (MeGA), pipelines simplify rendering and editing complexity.
Stable Barycentric MappingSampling in UV then projecting via barycentric coordinates (FATE) avoids mesh discretization artifacts and ensures smooth, differentiable optimization of Gaussian parameters.
In sum, UV parameterizations transform complex 3D Gaussian splat operations into well-behaved 2D workflows, yielding more uniform sampling, intuitive editing, and powerful opportunities for compression and acceleration that mesh-only approaches simply cannot match.
[1] FlashAvatar: Xiang, Jun, et al. “Flashavatar: High-fidelity head avatar with efficient gaussian embedding.” CVPR 2024.
[3] FATE: Zhang, Jiawei, et al. “Fate: Full-head gaussian avatar with textural editing from monocular video.” CVPR 2025.
[4] GEM: Zielonka, Wojciech, et al. “Gaussian eigen models for human heads.” CVPR 2025.
[2] MEGA: Wang, Cong, et al. “Mega: Hybrid mesh-gaussian head avatar for high-fidelity rendering and head editing.” CVPR 2025.
", "url": "https://temiz.ai/3d-gaussian-splats-avatars-utilizing-uv-mapping", "date_published": "2025-07-06T00:00:00+03:00", "date_modified": "2025-07-06T00:00:00+03:00", "author": { "name": "Hüseyin Temiz", "url": null, "avatar": null } }, { "id": "https://temiz.ai/3d-meshes-and-uv-mapping", "title": "3d Meshes And Uv Mapping", "summary": null, "content_text": "Meshes and UV maps are core to 3D graphics—they define shape and texture placement. From gaming and film to AR/VR and avatars, understanding how they work together is key to creating realistic and efficient digital content. This blog covers the basics and shows how they’re used in practice.3D Mesh and FLAMEA 3D mesh is a collection of vertices, edges, and faces that define the surface of a 3D object—like a wireframe sculpture. It forms the structural backbone of digital models used in games, animation, simulations, and 3D avatars. Common file formats include .obj, .ply, and .glb. For example, a human face mesh may consist of thousands of vertices and triangular faces to capture fine geometric detail.The FLAME [1] model (Faces Learned with an Articulated Model and Expressions) is a widely used parametric 3D head mesh model designed for realistic facial animation. It represents the face with 5023 vertices and 9976 triangles, controlled by identity, expression, and pose parameters. FLAME uses linear blend skinning (LBS) and blendshapes for expressive deformation, making it compact and efficient for applications like 3D avatars, facial reenactment, and neural rendering from 2D images or videos. Figure 1: FLAME head mesh with different poses.[2] FLAME’s parametric design uses 300 shape parameters (from PCA of 3D face scans) for identity control, 100 expression parameters for dynamic facial animation, and 6 pose parameters (3 neck, 3 jaw) for head/jaw rotation. By adjusting or tuning these parameters, we effectively modify the positions of FLAME’s 5023 vertices, enabling the generation of diverse facial shapes, expressions, and poses. This compact 406-parameter representation enables diverse, realistic faces with computational efficiency for real-time applications, as demonstrated in Figure 2 showing various shape, pose, and expression variations. Figure 2: FLAME variation. What Is UV Mapping?UV mapping is a 2D representation of a 3D surface used for texture mapping. Think of it as creating a flat blueprint of a curved 3D object—like unfolding a globe into a world map or unwrapping a chocolate bar to see its flat wrapper layout.Understanding U and V CoordinatesThe letters U and V represent 2D coordinates in texture space, similar to how X and Y work in regular 2D graphics: U corresponds to the horizontal axis (0 to 1, left to right) V corresponds to the vertical axis (0 to 1, bottom to top) These coordinates are completely independent from the 3D XYZ coordinates of the meshThe Chocolate Bar AnalogyImagine unwrapping a chocolate bar: 3D Object: The wrapped chocolate bar has a complex curved surface UV Map: When you carefully unfold the wrapper, you get a flat 2D layout Texture Mapping: The printed design on the wrapper (logos, text, patterns) corresponds to how textures are applied to the 3D surfaceThis unwrapping process is exactly what UV mapping does—it takes the complex 3D surface and “unfolds” it into a flat 2D space where textures can be painted or applied.Why UV Coordinates?UV coordinates are normalized (0 to 1 range), making them resolution-independent. Whether your texture is 256×256 pixels or 4096×4096 pixels, the same UV coordinates will work perfectly. Figure 3: UV MAP of FLAME. How FLAME Mesh and UV Map Work TogetherThe FLAME model provides a 3D mesh of the human head, defined by 5023 vertices and 9976 triangular faces, while the pre-calculated UV map offers a 2D representation of that mesh surface. The matching between the FLAME mesh and its UV map is defined by a per-vertex correspondence, where each vertex in 3D space has an associated (u, v) coordinate in 2D UV space.Here’s how the matching works: Each triangle in the FLAME mesh consists of 3 vertices indexed in 3D space. The same triangle has corresponding UV coordinates for those 3 vertices in 2D. When rendering or texturing, the 3D triangle is mapped to its 2D counterpart using these UV coordinates. This mapping allows a 2D texture image to wrap seamlessly over the 3D mesh surface.In practice, you often have: A vertex list: 3D coordinates (x, y, z) for each vertex. A face list: triplets of vertex indices (defining triangles). A UV list: (u, v) coordinates per vertex. A UV face list: triplets of UV indices (corresponding to vertex indices in face list).So, the structure allows the renderer or processing tool to: Match each triangle’s 3D geometry to its corresponding UV triangle. Sample texture values from the UV space during rendering or neural feature mapping (e.g., in 3DGS or GEM).Techniques for UV MappingUV mapping can be done manually using tools like Blender or Maya for precise control, or automatically for faster results. Good UV maps minimize distortion and avoid stretching by carefully placing seams and organizing UV islands. Clean mapping is key for accurate texture placement and visual quality.Why UV Mapping MattersUV mapping is essential for applying textures, PBR materials, baked normals, and lightmaps, enabling detailed and stylized 3D visuals. It also plays a key role in neural rendering, such as mapping 3D Gaussians onto UV space for efficient and high-fidelity avatar synthesis.Tools and Code SnippetsTools like Trimesh and Open3D allow easy access to mesh geometry and UV coordinates in code. You can use them to load meshes, extract UVs, and sample points on the UV map for tasks like texture transfer or feature alignment. Advanced techniques like Poisson disk sampling on the UV map enable uniform point distribution, useful for generative models and neural rendering pipelines.ConclusionMesh and UV mapping form the foundation of 3D graphics, enabling detailed geometry and rich texture placement across digital models. Whether you’re working on games, films, or avatars, mastering these concepts is essential. Try loading a simple mesh and unwrapping it yourself to see how UVs bring textures to life—tools like Blender or code notebooks make it easy to get started.References[1] Li, Tianye, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. “Learning a model of facial shape and expression from 4D scans.” ACM Trans. Graph. 2017.[2] https://github.com/TimoBolkart/FLAME-Universe", "content_html": "Meshes and UV maps are core to 3D graphics—they define shape and texture placement. From gaming and film to AR/VR and avatars, understanding how they work together is key to creating realistic and efficient digital content. This blog covers the basics and shows how they’re used in practice.
A 3D mesh is a collection of vertices, edges, and faces that define the surface of a 3D object—like a wireframe sculpture. It forms the structural backbone of digital models used in games, animation, simulations, and 3D avatars. Common file formats include .obj, .ply, and .glb. For example, a human face mesh may consist of thousands of vertices and triangular faces to capture fine geometric detail.
The FLAME [1] model (Faces Learned with an Articulated Model and Expressions) is a widely used parametric 3D head mesh model designed for realistic facial animation. It represents the face with 5023 vertices and 9976 triangles, controlled by identity, expression, and pose parameters. FLAME uses linear blend skinning (LBS) and blendshapes for expressive deformation, making it compact and efficient for applications like 3D avatars, facial reenactment, and neural rendering from 2D images or videos.
![\"Nearest](\"assets/images/post002_flame_head.png\")
Figure 1: FLAME head mesh with different poses.[2]
FLAME’s parametric design uses 300 shape parameters (from PCA of 3D face scans) for identity control, 100 expression parameters for dynamic facial animation, and 6 pose parameters (3 neck, 3 jaw) for head/jaw rotation. By adjusting or tuning these parameters, we effectively modify the positions of FLAME’s 5023 vertices, enabling the generation of diverse facial shapes, expressions, and poses. This compact 406-parameter representation enables diverse, realistic faces with computational efficiency for real-time applications, as demonstrated in Figure 2 showing various shape, pose, and expression variations.
Figure 2: FLAME variation.
UV mapping is a 2D representation of a 3D surface used for texture mapping. Think of it as creating a flat blueprint of a curved 3D object—like unfolding a globe into a world map or unwrapping a chocolate bar to see its flat wrapper layout.
The letters U and V represent 2D coordinates in texture space, similar to how X and Y work in regular 2D graphics:
Imagine unwrapping a chocolate bar:
This unwrapping process is exactly what UV mapping does—it takes the complex 3D surface and “unfolds” it into a flat 2D space where textures can be painted or applied.
UV coordinates are normalized (0 to 1 range), making them resolution-independent. Whether your texture is 256×256 pixels or 4096×4096 pixels, the same UV coordinates will work perfectly.
![\"Nearest](\"assets/images/post002_flame_uv_wireframe.png\")
Figure 3: UV MAP of FLAME.
The FLAME model provides a 3D mesh of the human head, defined by 5023 vertices and 9976 triangular faces, while the pre-calculated UV map offers a 2D representation of that mesh surface. The matching between the FLAME mesh and its UV map is defined by a per-vertex correspondence, where each vertex in 3D space has an associated (u, v) coordinate in 2D UV space.
Here’s how the matching works:
In practice, you often have:
So, the structure allows the renderer or processing tool to:
UV mapping can be done manually using tools like Blender or Maya for precise control, or automatically for faster results. Good UV maps minimize distortion and avoid stretching by carefully placing seams and organizing UV islands. Clean mapping is key for accurate texture placement and visual quality.
UV mapping is essential for applying textures, PBR materials, baked normals, and lightmaps, enabling detailed and stylized 3D visuals. It also plays a key role in neural rendering, such as mapping 3D Gaussians onto UV space for efficient and high-fidelity avatar synthesis.
Tools like Trimesh and Open3D allow easy access to mesh geometry and UV coordinates in code. You can use them to load meshes, extract UVs, and sample points on the UV map for tasks like texture transfer or feature alignment. Advanced techniques like Poisson disk sampling on the UV map enable uniform point distribution, useful for generative models and neural rendering pipelines.
Mesh and UV mapping form the foundation of 3D graphics, enabling detailed geometry and rich texture placement across digital models. Whether you’re working on games, films, or avatars, mastering these concepts is essential. Try loading a simple mesh and unwrapping it yourself to see how UVs bring textures to life—tools like Blender or code notebooks make it easy to get started.
[1] Li, Tianye, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero. “Learning a model of facial shape and expression from 4D scans.” ACM Trans. Graph. 2017.
[2] https://github.com/TimoBolkart/FLAME-Universe
", "url": "https://temiz.ai/3d-meshes-and-uv-mapping", "date_published": "2025-07-04T00:00:00+03:00", "date_modified": "2025-07-04T00:00:00+03:00", "author": { "name": "Hüseyin Temiz", "url": null, "avatar": null } }, { "id": "https://temiz.ai/placing-gaussians-splats-on-mesh-surface", "title": "Placing Gaussians Splats On Mesh Surface", "summary": null, "content_text": "In recent years, 3D Gaussian Splatting (GS) has emerged as a powerful technique for photorealistic avatar rendering and animation. Optimizing Gaussian splats without any geometric prior is challenging, as they often fail to regress accurate geometry. However, when the geometry of the object is known, fitting a mesh in advance and using its surface as a geometric prior provides a significant advantage. For example, if the object to be reconstructed is a human head, leveraging the FLAME mesh model as a geometric prior can greatly improve results.After fitting the FLAME model, we obtain a reasonable geometric surface on which to place GSs. At this stage, how we distribute the GSs on the mesh becomes an important problem. Using a smart strategy to place the GSs on the surface can significantly aid the subsequent GS optimization process.Naïve Random SamplingAs a baseline, GSs can be randomly distributed across the mesh surface. However, this naive approach often leads to clustering, irregular spacing, and oversampling, which degrades both visual quality and optimization stability.Poisson Disk SamplingTo avoid the performance degradations of random sampling, Poisson Disk Sampling — a technique that enforces a minimum distance between sampled points — to robustly distribute Gaussians on the FLAME mesh.Poisson Disk Sampling Algorithm (Bridson’s Algorithm): Initialize a grid with cell size = r/√2 (where r is minimum distance) Start with a random initial point, add to active list While active list is not empty: Pick random point from active list Generate k candidates (typically k=30) in annulus [r, 2r] For each candidate: Check if it’s far enough from all neighbors If valid, add to output and active list - If no valid candidates found, remove point from active list Key Properties: Guarantees minimum distance r between all points Maximum distance approximately 2r O(n) time complexity Produces uniform, organic-looking distributions Figure 1: A side-by-side comparison of random sampling (left) and Poisson disk sampling (right) on a 2D surface.Figure 1 shows the comparison between the two sampling methods, highlighting how Poisson Disk Sampling achieves more consistent nearest neighbor distances (lower standard deviation) and avoids extreme clustering (higher minimum distance) compared to random sampling. Figure 2: Comparison between random sampling (left) and Poisson disk sampling (right) on a sphere surface.As Figure 2 shows, Poisson disk sampling provides significantly better surface coverage compared to random sampling. The uniform distribution of sample points ensures that no regions of the mesh are left sparsely covered, while simultaneously preventing overcrowding in other areas. This improved coverage translates directly to more consistent Gaussian placement across the entire FLAME mesh surface, resulting in better geometric representation and more stable optimization during the 3D Gaussian splatting process. Figure 3: Nearest neighbor distance distributions for different samplings on sphere surface. Metric Poisson Disk Sampling Random Sampling n_points 500 500 nn_mean 0.1187 0.0787 nn_std 0.0160 0.0418 nn_min 0.1001 0.0073 nn_max 0.1853 0.2509 Table 1: Nearest neighbor distance statistics comparison between Poisson Disk Sampling and Random Sampling methods.The quantitative analysis in Table 1 and the distribution visualization in Figure 3 clearly demonstrate the superiority of Poisson Disk Sampling over random sampling. Most notably, Poisson Disk Sampling maintains a significantly higher minimum distance (0.1001 vs 0.0073), preventing the extreme clustering that occurs with random sampling. The lower standard deviation (0.0160 vs 0.0418) indicates much more consistent spacing between neighboring Gaussians, while the constrained maximum distance (0.1853 vs 0.2509) ensures no large gaps in coverage. This uniform distribution is crucial for stable 3D Gaussian Splatting optimization, as it prevents both overcrowded regions that cause rendering artifacts and sparse areas that miss geometric detail.Sampling on FLAME meshFigures 2 and 3 illustrate performance on a spherical surface with a more uniform geometry. In real-world applications, geometry is far from trivial—the FLAME human head mesh is both highly realistic and sufficiently complex. We will evaluate the sampling methods’ performance on the FLAME model. Figure 4: Nearest neighbor distance distributions for different samplings. Figure 4 visually compares random and Poisson disk sampling on the FLAME mesh for different point counts. Poisson disk sampling (bottom row, red) produces a much more uniform and evenly spaced distribution of points across the surface, as reflected by higher uniformity scores. In contrast, random sampling (top row, blue) results in visible clustering and uneven coverage, regardless of the number of points. This demonstrates that Poisson disk sampling maintains consistent regularity and superior surface coverage. Figure 5: Nearest Neihbor distance distribution of different sampling experiments. In Figure 5, Poisson disk sampling consistently enforces a minimum distance between neighboring samples, as shown by the sharp cutoff on the left side of each distribution. This property prevents samples from clustering too closely or overlapping, resulting in a more uniform and controlled distribution across the surface. In contrast, random sampling produces a broader spread of nearest neighbor distances, with many samples packed very closely together, leading to potential overlaps and uneven coverage. As the number of points increases, Poisson sampling maintains this regularity, while random sampling continues to exhibit significant variability and clustering. This highlights the robustness of Poisson disk sampling for applications requiring uniform coverage. Figure 6: flame_sampling_statistics.png. The Uniformity Score is a metric that quantifies how evenly points are distributed in a sampling. It is calculated as 1 minus the coefficient of variation (CV) of the nearest neighbor distances, where CV is the standard deviation divided by the mean. A score closer to 1 indicates a more uniform (evenly spaced) distribution, while a lower score means greater variability and less uniformity. This makes it easy to compare different sampling methods: for example, Poisson disk sampling yields higher uniformity scores than random sampling, reflecting its more consistent spacing between points. Points Random Sampling Poisson Disk Sampling 1000 0.4906 0.7752 5000 0.4760 0.8107 10000 0.4795 0.8086 Table 2: Uniformity scores (higher is better) for random and Poisson disk sampling at different point counts.Table 2 clearly shows that Poisson disk sampling consistently achieves much higher uniformity scores than random sampling across all point counts. This indicates that Poisson disk sampling produces a more evenly spaced and regular distribution of points on the mesh surface, while random sampling results in greater variability and less uniform coverage. The higher uniformity of Poisson disk sampling is especially important for applications that require consistent surface representation and stable optimization.ConclusionFor placing Gaussians on mesh surfaces, uniform coverage, minimal overlap, adaptive density, and mesh-awareness are essential. Naïve random sampling often leads to clustering, gaps, and unstable optimization. In contrast, Poisson Disk Sampling enforces even spacing and respects mesh geometry, resulting in a more regular and robust distribution. Having better geometric priors at the start of Gaussian splat optimization leads to more accurate and consistent recovery of visual details. This approach enables more accurate and stable neural avatar representations, improving rendering quality and optimization convergence—especially for complex models and advanced techniques such as GEM, RGBAvatar, or 3D Gaussian Blendshapes.", "content_html": "In recent years, 3D Gaussian Splatting (GS) has emerged as a powerful technique for photorealistic avatar rendering and animation. Optimizing Gaussian splats without any geometric prior is challenging, as they often fail to regress accurate geometry. However, when the geometry of the object is known, fitting a mesh in advance and using its surface as a geometric prior provides a significant advantage. For example, if the object to be reconstructed is a human head, leveraging the FLAME mesh model as a geometric prior can greatly improve results.
After fitting the FLAME model, we obtain a reasonable geometric surface on which to place GSs. At this stage, how we distribute the GSs on the mesh becomes an important problem. Using a smart strategy to place the GSs on the surface can significantly aid the subsequent GS optimization process.
As a baseline, GSs can be randomly distributed across the mesh surface. However, this naive approach often leads to clustering, irregular spacing, and oversampling, which degrades both visual quality and optimization stability.
To avoid the performance degradations of random sampling, Poisson Disk Sampling — a technique that enforces a minimum distance between sampled points — to robustly distribute Gaussians on the FLAME mesh.
Poisson Disk Sampling Algorithm (Bridson’s Algorithm):
Key Properties:
![\"Algorithm](\"assets/images/post001_algorithm_comparison.png\")
Figure 1: A side-by-side comparison of random sampling (left) and Poisson disk sampling (right) on a 2D surface.
Figure 1 shows the comparison between the two sampling methods, highlighting how Poisson Disk Sampling achieves more consistent nearest neighbor distances (lower standard deviation) and avoids extreme clustering (higher minimum distance) compared to random sampling.
![\"Sampling](\"assets/images/post001_sampling_comparison.png\")
Figure 2: Comparison between random sampling (left) and Poisson disk sampling (right) on a sphere surface.
As Figure 2 shows, Poisson disk sampling provides significantly better surface coverage compared to random sampling. The uniform distribution of sample points ensures that no regions of the mesh are left sparsely covered, while simultaneously preventing overcrowding in other areas. This improved coverage translates directly to more consistent Gaussian placement across the entire FLAME mesh surface, resulting in better geometric representation and more stable optimization during the 3D Gaussian splatting process.
![\"Nearest](\"assets/images/post001_nn_distance_comparison.png\")
Figure 3: Nearest neighbor distance distributions for different samplings on sphere surface.
| Metric | Poisson Disk Sampling | Random Sampling |
|---|---|---|
| n_points | 500 | 500 |
| nn_mean | 0.1187 | 0.0787 |
| nn_std | 0.0160 | 0.0418 |
| nn_min | 0.1001 | 0.0073 |
| nn_max | 0.1853 | 0.2509 |
Table 1: Nearest neighbor distance statistics comparison between Poisson Disk Sampling and Random Sampling methods.
The quantitative analysis in Table 1 and the distribution visualization in Figure 3 clearly demonstrate the superiority of Poisson Disk Sampling over random sampling. Most notably, Poisson Disk Sampling maintains a significantly higher minimum distance (0.1001 vs 0.0073), preventing the extreme clustering that occurs with random sampling. The lower standard deviation (0.0160 vs 0.0418) indicates much more consistent spacing between neighboring Gaussians, while the constrained maximum distance (0.1853 vs 0.2509) ensures no large gaps in coverage. This uniform distribution is crucial for stable 3D Gaussian Splatting optimization, as it prevents both overcrowded regions that cause rendering artifacts and sparse areas that miss geometric detail.
Figures 2 and 3 illustrate performance on a spherical surface with a more uniform geometry. In real-world applications, geometry is far from trivial—the FLAME human head mesh is both highly realistic and sufficiently complex. We will evaluate the sampling methods’ performance on the FLAME model.
![\"Nearest](\"assets/images/post001_flame_sampling_comparison.png\")
Figure 4: Nearest neighbor distance distributions for different samplings.
Figure 4 visually compares random and Poisson disk sampling on the FLAME mesh for different point counts. Poisson disk sampling (bottom row, red) produces a much more uniform and evenly spaced distribution of points across the surface, as reflected by higher uniformity scores. In contrast, random sampling (top row, blue) results in visible clustering and uneven coverage, regardless of the number of points. This demonstrates that Poisson disk sampling maintains consistent regularity and superior surface coverage.
![\"Nearest](\"assets/images/post001_flame_nn_distance_histograms.png\")
Figure 5: Nearest Neihbor distance distribution of different sampling experiments.
In Figure 5, Poisson disk sampling consistently enforces a minimum distance between neighboring samples, as shown by the sharp cutoff on the left side of each distribution. This property prevents samples from clustering too closely or overlapping, resulting in a more uniform and controlled distribution across the surface. In contrast, random sampling produces a broader spread of nearest neighbor distances, with many samples packed very closely together, leading to potential overlaps and uneven coverage. As the number of points increases, Poisson sampling maintains this regularity, while random sampling continues to exhibit significant variability and clustering. This highlights the robustness of Poisson disk sampling for applications requiring uniform coverage.
![\"Nearest](\"assets/images/post001_flame_sampling_statistics.png\")
Figure 6: flame_sampling_statistics.png.
The Uniformity Score is a metric that quantifies how evenly points are distributed in a sampling. It is calculated as 1 minus the coefficient of variation (CV) of the nearest neighbor distances, where CV is the standard deviation divided by the mean. A score closer to 1 indicates a more uniform (evenly spaced) distribution, while a lower score means greater variability and less uniformity. This makes it easy to compare different sampling methods: for example, Poisson disk sampling yields higher uniformity scores than random sampling, reflecting its more consistent spacing between points.
| Points | Random Sampling | Poisson Disk Sampling |
|---|---|---|
| 1000 | 0.4906 | 0.7752 |
| 5000 | 0.4760 | 0.8107 |
| 10000 | 0.4795 | 0.8086 |
Table 2: Uniformity scores (higher is better) for random and Poisson disk sampling at different point counts.
Table 2 clearly shows that Poisson disk sampling consistently achieves much higher uniformity scores than random sampling across all point counts. This indicates that Poisson disk sampling produces a more evenly spaced and regular distribution of points on the mesh surface, while random sampling results in greater variability and less uniform coverage. The higher uniformity of Poisson disk sampling is especially important for applications that require consistent surface representation and stable optimization.
For placing Gaussians on mesh surfaces, uniform coverage, minimal overlap, adaptive density, and mesh-awareness are essential. Naïve random sampling often leads to clustering, gaps, and unstable optimization. In contrast, Poisson Disk Sampling enforces even spacing and respects mesh geometry, resulting in a more regular and robust distribution. Having better geometric priors at the start of Gaussian splat optimization leads to more accurate and consistent recovery of visual details. This approach enables more accurate and stable neural avatar representations, improving rendering quality and optimization convergence—especially for complex models and advanced techniques such as GEM, RGBAvatar, or 3D Gaussian Blendshapes.
", "url": "https://temiz.ai/placing-gaussians-splats-on-mesh-surface", "date_published": "2025-07-03T00:00:00+03:00", "date_modified": "2025-07-03T00:00:00+03:00", "author": { "name": "Hüseyin Temiz", "url": null, "avatar": null } }, { "id": "https://temiz.ai/sample-math-latex-code", "title": "Sample Math Latex Code", "summary": null, "content_text": "This is experimental page. All contents are garbage. Set of spherical harmonics mapped to the surface of a sphereWhen rendering 3D scenes using Gaussian splats, each point (or “splat”) needs to represent not just color, but how that color changes depending on the viewing direction. Spherical harmonics allow encoding this angular variation compactly.This is important for achieving photorealistic rendering with effects like: Specular highlights Soft shadows View-dependent lighting changes🔸 What Are Spherical Harmonics?Spherical Harmonics (SH) are a series of orthogonal basis functions defined on the surface of a sphere. You can think of them as the 3D analog of Fourier series for functions on a sphere.They are indexed by: Degree (l): controls the frequency (higher means more detail) Order (m): varies from -l to +lExample Levels: SH level 0 (SH0): Only constant color (no view dependence) SH level 1 (SH1): Basic directional lighting (like Lambertian) SH level 2 or 3+: Captures more complex angular dependencies🔸 SH in Gaussian SplattingEach 3D Gaussian stores: Position, scale, rotation, opacity Spherical Harmonics coefficients for RGB appearanceInstead of storing just a single RGB color, we store a set of SH coefficients per color channel. When rendering: Compute the viewing direction v for each Gaussian. Evaluate the SH basis functions at direction v to get a vector SH_basis(v). Compute the final color as a dot product:\\[C(v) = \\sum_{l=0}^{L} \\sum_{m=-l}^{l} c_{l,m} \\cdot Y_{l,m}(v)\\]Where: \\(C(v)\\) : The final color for the viewing direction \\(v\\). \\(c_{l,m}\\): The SH coefficients for each degree \\(l\\) and order \\(m\\). \\(Y_{l,m}(v)\\): The SH basis functions evaluated at the direction \\(v\\).🔸 Key Spherical Harmonics EquationsSH Basis FunctionsThe real-valued spherical harmonics basis functions are defined as:\\[Y_{l}^{m}(\\theta, \\phi) = N_{l}^{m} \\cdot P_{l}^{m}(\\cos\\theta) \\cdot \\begin{cases}\\sqrt{2} \\sin(|m|\\phi) & \\text{if } m < 0 \\\\1 & \\text{if } m = 0 \\\\\\sqrt{2} \\cos(m\\phi) & \\text{if } m > 0\\end{cases}\\]Where: \\(\\theta\\) : inclination (angle from the z-axis) \\(\\phi\\): azimuth (angle from the x-axis in the x-y plane) \\(N_{l}^{m}\\): normalization constant \\(P_{l}^{m}\\): associated Legendre polynomialNormalization Constant\\[N_{l}^{m} = \\sqrt{\\frac{(2l+1)}{4\\pi} \\cdot \\frac{(l - |m|)!}{(l + |m|)!}}\\]Example: SH0 and SH1 SH0 (l=0, m=0): \\(Y_0^0 = \\frac{1}{2} \\sqrt{\\frac{1}{\\pi}}\\) SH1 (l=1, m=-1,0,1): \\(Y_1^{-1} = \\sqrt{\\frac{3}{4\\pi}} \\sin\\theta \\sin\\phi\\) \\(Y_1^{0} = \\sqrt{\\frac{3}{4\\pi}} \\cos\\theta\\) \\(Y_1^{1} = \\sqrt{\\frac{3}{4\\pi}} \\sin\\theta \\cos\\phi\\) 🔸 Benefits of Using SH Compact representation of angular variation. Efficient evaluation during rendering. Scalable to higher levels for more detail.", "content_html": "This is experimental page. All contents are garbage.
![\"Set](\"assets/images/post000_set-of-spherical-harmonics-mapped-to-the-surface-of-a-sphere.png\")
Set of spherical harmonics mapped to the surface of a sphere
When rendering 3D scenes using Gaussian splats, each point (or “splat”) needs to represent not just color, but how that color changes depending on the viewing direction. Spherical harmonics allow encoding this angular variation compactly.
This is important for achieving photorealistic rendering with effects like:
Spherical Harmonics (SH) are a series of orthogonal basis functions defined on the surface of a sphere. You can think of them as the 3D analog of Fourier series for functions on a sphere.
They are indexed by:
-l to +lEach 3D Gaussian stores:
Instead of storing just a single RGB color, we store a set of SH coefficients per color channel. When rendering:
v for each Gaussian.v to get a vector SH_basis(v).Where:
The real-valued spherical harmonics basis functions are defined as:
\\[Y_{l}^{m}(\\theta, \\phi) = N_{l}^{m} \\cdot P_{l}^{m}(\\cos\\theta) \\cdot \\begin{cases}\\sqrt{2} \\sin(|m|\\phi) & \\text{if } m < 0 \\\\1 & \\text{if } m = 0 \\\\\\sqrt{2} \\cos(m\\phi) & \\text{if } m > 0\\end{cases}\\]Where:
SH0 (l=0, m=0): \\(Y_0^0 = \\frac{1}{2} \\sqrt{\\frac{1}{\\pi}}\\)
When rendering 3D scenes using Gaussian splats, each point (or “splat”) needs to represent not just color, but how that color changes depending on the viewing direction. Spherical harmonics allow encoding this angular variation compactly.
![\"Spherical](\"assets/images/post000_Spherical_Harmonics.png\")
Left: Spherical harmonics mapped to a sphere | Right: Spherical harmonics visualization
This is important for achieving photorealistic rendering with effects like:
Spherical Harmonics (SH) are a series of orthogonal basis functions defined on the surface of a sphere. You can think of them as the 3D analog of Fourier series for functions on a sphere.
They are indexed by:
-l to +lEach 3D Gaussian stores:
Instead of storing just a single RGB color, we store a set of SH coefficients per color channel. When rendering:
v for each Gaussian.v to get a vector SH_basis(v).color = Σ (c_i * SH_basis_i(v)) for i = 1..nwhere c_i are SH coefficients.
| SH Level | # Coefficients (per color) | What it Captures |
|---|---|---|
| 0 | 1 | Constant color |
| 1 | 4 | Directional diffuse lighting |
| 2 | 9 | Soft shadows, simple specular |
| 3 | 16 | Rich angular detail |
So if SH=2:
Imagine a glossy red ball:
import numpy as npfrom scipy.special import sph_harm# Dummy view direction (theta, phi) in spherical coordinatestheta = np.pi / 4 # angle from z-axisphi = np.pi / 3 # angle from x-axis in x-y plane# Compute SH basis up to degree l=2def compute_sh_basis(theta, phi, l_max=2): basis = [] for l in range(l_max + 1): for m in range(-l, l + 1): Y_lm = sph_harm(m, l, phi, theta).real # Only real part used in graphics basis.append(Y_lm) return np.array(basis)# Example SH coefficients for RGB (assuming SH2 = 9 coeffs per channel)sh_coeffs_r = np.random.rand(9)sh_coeffs_g = np.random.rand(9)sh_coeffs_b = np.random.rand(9)# Evaluate SH basissh_basis = compute_sh_basis(theta, phi)# Final RGB color (dot product of SH coefficients and SH basis)r = np.dot(sh_coeffs_r, sh_basis)g = np.dot(sh_coeffs_g, sh_basis)b = np.dot(sh_coeffs_b, sh_basis)print(f\"RGB color from SH: R={r:.2f}, G={g:.2f}, B={b:.2f}\")This code shows:
scipy.special.sph_harmIn Gaussian Splatting, Spherical Harmonics:
You can trade off quality vs speed by choosing the SH level (e.g., SH=0 for fast rendering, SH=2+ for better realism).
", "url": "https://temiz.ai/spherical-harmonics-in-gaussian-splats", "date_published": "2025-05-19T00:00:00+03:00", "date_modified": "2025-05-19T00:00:00+03:00", "author": { "name": "Hüseyin Temiz", "url": null, "avatar": null } } ] }