I have a question about sampling offset in the pose decoder.

[Yamameeee]

In FIgure 3. in your PETR paper, each referencepoint has multi sampling points.
But the shape of input sampling_locations for MultiScaleDeformableAttnFunction is (bs, num_query, num_heads, num_levels, num_leypoints, 2).

Did you set the number of sampling offsets for one reference point (i.e. one keypoint) as 1?

opera/opera/models/utils/transformer.py

Line 409 in a7cadf6

sampling_locations = reference_points_reshape \

[dae-sun]

yes, the number of sampling offsets for one reference point (i.e. one keypoint) is set as 1 for each head and feature level
So there are 32 sampling offsets for one reference point.

[dae-sun]

It could be visualized like this. green is the reference point and other points are sampling points. (low attentions are dismissed)