Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling of persons without keypoints in OKS annotation #655

Open
kschlegel opened this issue Aug 1, 2023 · 0 comments
Open

Handling of persons without keypoints in OKS annotation #655

kschlegel opened this issue Aug 1, 2023 · 0 comments

Comments

@kschlegel
Copy link

I am struggling to see the reasoning in dealing with person objects without keypoint annotations in the OKS computation:

xg = g[0::3]; yg = g[1::3]; vg = g[2::3]
k1 = np.count_nonzero(vg > 0)
bb = gt['bbox']
x0 = bb[0] - bb[2]; x1 = bb[0] + bb[2] * 2
y0 = bb[1] - bb[3]; y1 = bb[1] + bb[3] * 2
for i, dt in enumerate(dts):
d = np.array(dt['keypoints'])
xd = d[0::3]; yd = d[1::3]
if k1>0:
# measure the per-keypoint distance if keypoints visible
dx = xd - xg
dy = yd - yg
else:
# measure minimum distance to keypoints in (x0,y0) & (x1,y1)
z = np.zeros((k))
dx = np.max((z, x0-xd),axis=0)+np.max((z, xd-x1),axis=0)
dy = np.max((z, y0-yd),axis=0)+np.max((z, yd-y1),axis=0)
e = (dx**2 + dy**2) / vars / (gt['area']+np.spacing(1)) / 2
if k1 > 0:
e=e[vg > 0]
ious[i, j] = np.sum(np.exp(-e)) / e.shape[0]

Here is how I read this:
k1 is the number of labelled keypoints for the person, x0 is to the left of the bounding box, x1 to the right and similarly for y0,y1 above and below.
Now if k1==0 i.e. we do not have any annotated keypoints for a person object then dx will be non-zero if and only if the predicted keypoint lies to the left of x0 or to the right of x1 or in other words if it lies significantly outside the persons tight bounding box. Similarly for dy. Filtering of the exponents for visible keypoints the nis also skipped when k1==1.

In particular if I am reading this correctly this means anything within vague proximity of the bounding box is considered "a perfect match", regardless of where around there it is. Sort of like a "well it correctly realised there is a person somewhere around here, can't blame it if it got the keypoints completely wrong". It feels like I must be missing something here, please do correct me if that is the case!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant