isaac_ros_foundationpose on zed camera

[kc645]

Hi, I am trying to run foundationpose on zed camera.

In the documentation, I don't see a column for zed camera, but there is for rosbag, realsense camera, and hawk camera. I managed to look at centerpose and compared the launch arguments.
So I ran this launch command

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=zed_mono_rect_depth,foundationpose mesh_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/Mac_and_cheese_0_1/Mac_and_cheese_0_1.obj texture_path:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_foundationpose/Mac_and_cheese_0_1/materials/textures/baked_mesh_tex0.png score_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/score_trt_engine.plan refine_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/foundationpose/refine_trt_engine.plan rt_detr_engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/synthetica_detr/sdetr_grasp.plan interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_centerpose/zed2_quickstart_interface_specs.json

to run zed mono rect depth.
To visualize, I ran rviz2 -d $(ros2 pkg prefix isaac_ros_foundationpose --share)/rviz/foundationpose_zed.rviz, to visualize

The correct feed shows up but somehow when it detects my tumblr (even though I am running Mac and cheese?) the program shuts down by saying below

What would be the solution to this problem?

Also, do we have to only use DETR object detection? What if we want to use yolo model or other object detection models, how would I go about this?

Thank you so much for your help!

[kc645]

so by changing the ZED setting of pub resolution from CUSTOM to NATIVE, I was able to let the program not quit.
However I get an error from zed_wrapper-2 Error as following

[zed_wrapper-2] Error: TF_DENORMALIZED_QUATERNION: Ignoring transform for child_frame_id "fp_object" from authority "Authority undetectable" because of an invalid quaternion in the transform (-nan nan nan -nan)

What would be the solution to this?