NMS time limit issue on MPS, Works in CPU

[shanalikhan]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hi,
I'm using yolo11n

This line works

results = model.train(data="/Users/shan/Documents/iqlogy/047/yolo8obb_2347/yoloobb_main.yaml", epochs=1, imgsz=512)#,device="mps")

Generates the following output:

train: Scanning [/Users/shan/Documents/iqlogy/047/yolo8obb_2347/labels/train.cache...](https://file+.vscode-resource.vscode-cdn.net/Users/shan/Documents/iqlogy/047/yolo8obb_2347/labels/train.cache...) 2117 images, 500 backgrounds, 0 corrupt: 100%|██████████| 2117/2117 [00:00<?, ?it/s]
val: Scanning [/Users/shan/Documents/iqlogy/047/yolo8obb_2347/labels/test.cache...](https://file+.vscode-resource.vscode-cdn.net/Users/shan/Documents/iqlogy/047/yolo8obb_2347/labels/test.cache...) 230 images, 60 backgrounds, 0 corrupt: 100%|██████████| 230/230 [00:00<?, ?it/s]
Plotting labels to runs/detect/train132/labels.jpg... 

optimizer: 'optimizer=auto' found, ignoring 'lr0=0.01' and 'momentum=0.937' and determining best 'optimizer', 'lr0' and 'momentum' automatically... 
optimizer: AdamW(lr=0.002, momentum=0.9) with parameter groups 81 weight(decay=0.0), 88 weight(decay=0.0005), 87 bias(decay=0.0)
TensorBoard: model graph visualization added ✅
Image sizes 512 train, 512 val
Using 0 dataloader workers
Logging results to runs/detect/train132
Starting training for 1 epochs...

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
        1/1         0G      2.141      1.816      1.171         27        512: 100%|██████████| 133/133 [07:45<00:00,  3.50s/it]
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 8/8 [00:22<00:00,  2.84s/it]
                   all        230       1052      0.562      0.417       0.43      0.158

1 epochs completed in 0.137 hours.
Optimizer stripped from runs/detect/train132/weights/last.pt, 5.4MB
Optimizer stripped from runs/detect/train132/weights/best.pt, 5.4MB

Validating runs/detect/train132/weights/best.pt...
WARNING ⚠️ validating an untrained model YAML will result in 0 mAP.
Ultralytics 8.3.13 🚀 Python-3.12.7 torch-2.4.0 CPU (Apple M3)
YOLO11n summary (fused): 238 layers, 2,582,347 parameters, 0 gradients, 6.3 GFLOPs
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 8/8 [00:21<00:00,  2.63s/it]
                   all        230       1052      0.564      0.417      0.429      0.157
Speed: 0.4ms preprocess, 86.0ms inference, 0.0ms loss, 0.8ms postprocess per image
Results saved to runs/detect/train132

Now converting from CPU to mps shows warning WARNING ⚠️ NMS time limit 3.600s exceeded

results = model.train(data="/Users/shan/Documents/iqlogy/047/yolo8obb_2347/yoloobb_main.yaml", epochs=1, imgsz=512, optimizer='SGD',device="mps",lr0=0.001,verbose=True)

Transferred 448/499 items from pretrained weights
TensorBoard: Start with 'tensorboard --logdir runs/detect/train14', view at http://localhost:6006/
Freezing layer 'model.23.dfl.conv.weight'
train: Scanning /Users/shan/Documents/iqlogy/047/yolo8obb_2347/labels/train.cache... 2117 images, 500 backgrounds, 0 corrupt: 100%|██████████| 2117/2117 [00:00<?, ?it/s]
val: Scanning /Users/shan/Documents/iqlogy/047/yolo8obb_2347/labels/test.cache... 230 images, 60 backgrounds, 0 corrupt: 100%|██████████| 230/230 [00:00<?, ?it/s]
Plotting labels to runs/detect/train14/labels.jpg...
optimizer: SGD(lr=0.001, momentum=0.937) with parameter groups 81 weight(decay=0.0), 88 weight(decay=0.0005), 87 bias(decay=0.0)
TensorBoard: model graph visualization added ✅
Image sizes 512 train, 512 val
Using 0 dataloader workers
Logging results to runs/detect/train14
Starting training for 1 epochs...

  Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
    1/1      4.06G      2.419      3.446      1.311         27        512: 100%|██████████| 133/133 [02:19<00:00,  1.05s/it]
             Class     Images  Instances      Box(P          R      mAP50  mAP50-95):   0%|          | 0/8 [00:00<?, ?it/s]

WARNING ⚠️ NMS time limit 3.600s exceeded
Class Images Instances Box(P R mAP50 mAP50-95): 12%|█▎ | 1/8 [00:12<01:29, 12.76s/it]
WARNING ⚠️ NMS time limit 3.600s exceeded
Class Images Instances Box(P R mAP50 mAP50-95): 25%|██▌ | 2/8 [00:21<01:00, 10.16s/it]
WARNING ⚠️ NMS time limit 3.600s exceeded
Class Images Instances Box(P R mAP50 mAP50-95): 38%|███▊ | 3/8 [00:29<00:46, 9.26s/it]
WARNING ⚠️ NMS time limit 3.600s exceeded
Class Images Instances Box(P R mAP50 mAP50-95): 50%|█████ | 4/8 [00:38<00:38, 9.60s/it]

Am i missing something, how to optimise in MPS

Additional

No response

[pderrenger]

@shanalikhan the NMS time limit warning on MPS might be due to hardware or software constraints. Ensure you're using the latest YOLOv5 version and PyTorch with MPS support. You can try reducing the batch size or image size to see if it helps. For more details, check the multi-GPU training tutorial.