SlurmGCP. Make delete instances status tracking "asynchronous"

[mr0re1]

Instead of waiting for delete-instance operation to finish, make record files for slurmsync to process them later.

$ time ./suspend.py tst-debugnodeset-[0-99]
real    1m21.206s  # current
...
real    0m3.223s # new

In a span of 3 invocations of slurmsync all watched operations has been processed with 0 requests to GCP API.

$ taif -f slurmsync.log
2024-12-17 19:49:48,830 DEBUG: Watching delete-instance op=operation-1734464885946-6297c9366fa42-410fdda2-c56c09f6. VM tst-debugnodeset-65 status=STOPPING                                                                  
2024-12-17 19:49:48,830 DEBUG: Skipping op operation-1734464885946-6297c9366fa42-410fdda2-c56c09f6, VM tst-debugnodeset-65 is STOPPING                                                                                     
2024-12-17 19:49:48,830 DEBUG: Watching delete-instance op=operation-1734464885947-6297c9366fd72-b3796c6f-ad11ffa7. VM tst-debugnodeset-67 status=STOPPING                                                                  
2024-12-17 19:49:48,830 DEBUG: Skipping op operation-1734464885947-6297c9366fd72-b3796c6f-ad11ffa7, VM tst-debugnodeset-67 is STOPPING                                                                                     
2024-12-17 19:49:48,830 DEBUG: Watching delete-instance op=operation-1734464885948-6297c9366ff16-7aa22a0a-9058adb4. VM tst-debugnodeset-68 status=STOPPING                                                                  
2024-12-17 19:49:48,830 DEBUG: Skipping op operation-1734464885948-6297c9366ff16-7aa22a0a-9058adb4, VM tst-debugnodeset-68 is STOPPING                                                                                     
2024-12-17 19:49:48,831 DEBUG: Watching delete-instance op=operation-1734464885949-6297c93670269-7a67f1ec-1cfa35e9. VM tst-debugnodeset-70 status=STOPPING                                                                  
2024-12-17 19:49:48,831 DEBUG: Skipping op operation-1734464885949-6297c93670269-7a67f1ec-1cfa35e9, VM tst-debugnodeset-70 is STOPPING                                                                                              
...
2024-12-17 19:50:48,770 DEBUG: Stop watching op operation-1734464885955-6297c93671cab-f982e4f9-a338798b, VM tst-debugnodeset-86 appears to be deleted
2024-12-17 19:50:48,771 DEBUG: Stop watching op operation-1734464885956-6297c93671e4c-e140a2fe-8a3f7d98, VM tst-debugnodeset-87 appears to be deleted
2024-12-17 19:50:48,771 DEBUG: Stop watching op operation-1734464885956-6297c93671ff1-9000a074-adf1b44c, VM tst-debugnodeset-88 appears to be deleted
2024-12-17 19:50:48,771 DEBUG: Stop watching op operation-1734464885957-6297c9367219c-dfc51021-6ad339fc, VM tst-debugnodeset-89 appears to be deleted
2024-12-17 19:50:48,771 DEBUG: Stop watching op operation-1734464885957-6297c93672337-b0fe4c8c-54d83688, VM tst-debugnodeset-90 appears to be deleted

Before sending out PR:
[ ] Add unit tests;
[ ] Setup code to create watched ops directory