Correctness of FoVPerspectiveCameras's compute_project_matrix()

[zhifanzhu]

pytorch3d/pytorch3d/renderer/cameras.py

Lines 622 to 638 in 59972b1

	h1 = (max_y + min_y)/(max_y - min_y)
	w1 = (max_x + min_x)/(max_x - min_x)
	tanhalffov = tan((fov/2))
	s1 = 1/tanhalffov
	s2 = 1/(tanhalffov * (aspect_ratio))
	# To map z to the range [0, 1] use:
	f1 = far / (far - near)
	f2 = -(far * near) / (far - near)
	# Projection matrix
	K = [
	[s1, 0, w1, 0],
	[0, s2, h1, 0],
	[0, 0, f1, f2],
	[0, 0, 1, 0],
	]

In FoVPerspectiveCameras the projection_matrix is computed as :

            K = [
                    [s1,   0,   w1,   0],
                    [0,   s2,   h1,   0],
                    [0,    0,   f1,  f2],
                    [0,    0,    1,   0],
            ]

where s1 = 2*znear / (max_x - min_x), w1 = (max_x - min_x) / (max_x - min_x) ,

However, if we verify this K by setting (x, y, z, w) = (min_x, min_y, znear, w=1) in world coordinates,
after projection it becomes

x_clip = (3 * znear * min_x + znear * max_x) / (max_x - min_x)
w_clip = znear

then after dividing x_ndc = x_clip / w_clip, clearly x_ndc does not get mapped to -1.

But if w1 is - (max_x - min_x) / (max_x - min_x), then min_x can be mapped to -1.

This is currently not an issue because inFoVPerspectiveCameras , max_x always equals - min_x, so w1 and h1 are always zeros.