Request for Guidance: Adding a New Head in Detectron 2 for Weather Prediction and Object Occlusion Detection

[matg41]

Hello everyone,

I am attempting to add a new head to the base model to predict weather and determine if an object is occluded or not. The dataset I am using is coco-like, and I have already registered the new field in the platform. As a beginner in using detectron 2, I understand that I need to modify the forward function in the model and create a custom loss function. However, I am unsure of the proper way to do this in detectron 2.

I have come across issue #3321 (in detectron 2 repo), where a similar problem is being addressed. It seems that to create a new head, I need to define a regular nn.Module or nn.Sequential class, such as mask_head and deeplab. Subsequently, in roi_heads.py, I should implement the StandardROIHeads or the superclass ROIHeads and add the new head. Additionally, a yaml file needs to be created in the MaskDINO/configs folder to include the additional heads.

I would appreciate any guidance on how to properly add another mask-type head and make the necessary modifications in detectron 2.

Thank you for your assistance.