How to save 3D face

[icegaomu]

Thank you for finsh this great job!
I have some questions about how to get 3D mesh and texture in your code?
I want to same then individual as obj or plt format and get 3D normal ,I don’t see those code but your paper show those 3D shapes .
I look forward to hearing from you.thanks again!

[philgras]

Hi,

in nha_optimizer.py, there is a method for rendering the shaded mesh

@torch.no_grad()
    def predict_shaded_mesh(self, batch, tex_color=np.array((188, 204, 245)) / 255, light_colors=(0.4, 0.6, 0.3)):
        K = batch["cam_intrinsic"]
        RT = batch["cam_extrinsic"]
        H, W = batch["rgb"].shape[-2:]

        flame_params_offsets = self._create_flame_param_batch(batch)
        offsets_verts, _ = self._forward_flame(flame_params_offsets)

        vertex_colors = torch.ones_like(offsets_verts) * torch.tensor(tex_color, device=self.device).float().view(1, 3)
        # define meshes and textures
        tex = TexturesVertex(vertex_colors)
        mesh = Meshes(
            verts=offsets_verts,
            faces=self._flame.faces[None].expand(len(offsets_verts), -1, -1),
            textures=tex,
        )

        return render_shaded_mesh(mesh, K, RT, (H, W), self.device, light_colors)

You can use this as a starting point. The mesh object can be used to compute per-vertex normals or can be saved to .obj or other formats using pytorch.io.*

Hope this helps!

[icegaomu]

Thank you for the answer,but I want to save the whole RGB image or called UV maps ,but there is a little complex to save the object with texture, because i only find the render_rgba program. And it's for the novel view images generation, is there have a more easy way to save the texture ? Thanks again!

[icegaomu]

def predict_shaded_mesh(self, batch, tex_color=np.array((188, 204, 245)) / 255, light_colors=(0.4, 0.6, 0.3))
And this program give a const tex_color ,but now i can't save the whole texture (not from one view) can you give me some advice?

[qiuyuzhao]

Hi,

in nha_optimizer.py, there is a method for rendering the shaded mesh

@torch.no_grad()
    def predict_shaded_mesh(self, batch, tex_color=np.array((188, 204, 245)) / 255, light_colors=(0.4, 0.6, 0.3)):
        K = batch["cam_intrinsic"]
        RT = batch["cam_extrinsic"]
        H, W = batch["rgb"].shape[-2:]

        flame_params_offsets = self._create_flame_param_batch(batch)
        offsets_verts, _ = self._forward_flame(flame_params_offsets)

        vertex_colors = torch.ones_like(offsets_verts) * torch.tensor(tex_color, device=self.device).float().view(1, 3)
        # define meshes and textures
        tex = TexturesVertex(vertex_colors)
        mesh = Meshes(
            verts=offsets_verts,
            faces=self._flame.faces[None].expand(len(offsets_verts), -1, -1),
            textures=tex,
        )

        return render_shaded_mesh(mesh, K, RT, (H, W), self.device, light_colors)

You can use this as a starting point. The object can be used to compute per-vertex normals or can be saved to .obj or other formats using pytorch.io.*mesh

Hope this helps!

This seems to only save before rendering,how to get the rendered?

[icegaomu]

I try to save the texture uv map first by change the render_rgba program ,but it also get some problem because of there are not certain parameter about pix_to_face,and it's vertex weight . @qiuyuzhao this is my email [email protected] we can have a communicate or if you have some correct change or save the rendered obj successful please tell me ! thanks a lot

[yangqing-yq]

I try to save the texture uv map first by change the render_rgba program ,but it also get some problem because of there are not certain parameter about pix_to_face,and it's vertex weight . @qiuyuzhao this is my email [email protected] we can have a communicate or if you have some correct change or save the rendered obj successful please tell me ! thanks a lot

hey bro, any solution here?

[icegaomu]

I‘m sorry not.but i don't try the way sampling in the whole 3d space.maybe this method is work,and the whole time to genarate a face is too long,so i will not fellow this work.