WIP API for Yolact, python package #323

breznak · 2020-02-07T12:37:49Z

WIP
Aim of this PR is to have Yolact used (useful) as a python package, interfacing with other py code with API (ideally as close to pytorch.nn as possible)

init() improvements
eval()
train()
- might be too complicated
get rid of global variables
make .cuda() optional, depending on device_type chosen (cpu,gpu,tpu)

Fixes #299
For #300

so it's done automatically, not by hand after init.

breznak · 2020-02-07T12:39:16Z

yolact.py

+        self.init_weights(backbone_path='weights/' + cfg.backbone.path)
+
+        # GPU
+        #TODO try half: net = net.half()


Off topic, but has anyone tried running the models as half/fp16? Generally improves performance,mem footprint.

and imports. Cfg /config/ is not global, but passed as self.cfg, or as arg to methods.

from Config

breznak

Please review if you have time.

The 1st commit improves constructor.
the other commits separate local nn.Module classes to standalone files, and should improve/avoid global cfg requirement.
in the following commits I'll focus on bringing eval.py to Yolact.eval()

data/config.py

yolact.py

breznak · 2020-02-07T14:23:33Z

yolact.py

@@ -564,20 +235,21 @@ def freeze_bn(self, enable=False):
    def forward(self, x):
        """ The input should be of size [batch_size, 3, img_h, img_w] """
        _, _, img_h, img_w = x.size()
+        cfg = self.cfg
        cfg._tmp_img_h = img_h


I sometimes use this "hack" instead of changing the whole block from cfg to self.cfg, is that ok, style-wise?

Should be okay, since self.x is always so ugly (I really wish Python worked like other OOP languages in that regard).

Though it looks like you use both cfg and self.cfg in the same function here. I'd pick one and stick with it.

abhigoku10 · 2020-02-09T02:23:18Z

@breznak is this done cna we use yolact as an api , is it the same for yolact++ also ?? great work

breznak · 2020-02-09T14:36:43Z

can we use yolact as an api , is it the same for yolact++ also ?

yes, this is intended for both yolact, yolact++

dbolya

I think the direction so far is good, but I'm wondering if it's worth it to have all the modules own the config object vs. just having one global config object. There's so much you'd need to change in order to be able to run two completely distinct configs at the same time, that it's probably just not worth it. And who really needs to have to separate versions of Yolact running at once?

Ok I guess maybe it would actually be useful if you have two different types of images I guess. Hmm, should we expend the effort to support that?

Apart from that, let's also take this oppertunity to reduce the global side effects and to add arbitrary device support (basically change all the .cuda() and fix all the tensor initializations). I'll also try to contribute to this PR when I have time.

yolact.py

dbolya · 2020-02-10T22:09:14Z

yolact.py

@@ -396,18 +44,31 @@ class Yolact(nn.Module):
        - pred_aspect_ratios: A list of lists of aspect ratios with len(selected_layers) (see PredictionModule)
    """

-    def __init__(self):
+    def __init__(self, 
+                 config_name="yolact_base_config",


Ideally, you wouldn't have to modify config.py to use YOLACT as a pip package. Thus, I think this should accept either the name of the config or a config object (i.e., Union[str, Config]). This would require a similar change in set_cfg too.

accept either the name of the config or a config object (i.e., Union[str, Config]).

isn't that what's there now? Name of the config. I can change to the (str, Config) but that seems the same to me. I was thinking about ommiting the set_cfg() and expecting the user to import/create the config they want and just pass it here directly. So from data.config import yolact_base; net = Yolact(yolact_base) ?

yolact.py

dbolya · 2020-02-10T22:14:01Z

yolact.py

+
+        # GPU
+        #TODO try half: net = net.half()
+        self.cuda()


We should probably either pass in the device as an argument or let the user handle this externally. Just calling .cuda() has pytorch pick a GPU, and it restricts future CPU or TPU support.

and to add arbitrary device support (basically change all the .cuda() and fix all the tensor initializations).

I'll need some guidance on this. Do you mean just add Yolact(device="gpu/cpu/tpu") and handle these accordingly?

I like this and prepared support for the feature is in 236352d ,
But there's a lot of .cuda() calls in the code so far, even in external DCNv2. I'd prefer to finish this feature in a separate PR later.
Btw, the alternative to .cuda() is .to(device="gpu") ..if so, that's what we'd use.

dbolya · 2020-02-10T22:14:31Z

yolact.py

+        # GPU
+        #TODO try half: net = net.half()
+        self.cuda()
+        torch.set_default_tensor_type('torch.cuda.FloatTensor')


If this didn't break anything, then it's fine, but this should probably go at the start of init not here.

dbolya · 2020-02-10T22:15:38Z

yolact.py

@@ -564,20 +235,21 @@ def freeze_bn(self, enable=False):
    def forward(self, x):
        """ The input should be of size [batch_size, 3, img_h, img_w] """
        _, _, img_h, img_w = x.size()
+        cfg = self.cfg
        cfg._tmp_img_h = img_h


Should be okay, since self.x is always so ugly (I really wish Python worked like other OOP languages in that regard).

dbolya · 2020-02-10T22:16:21Z

yolact.py

@@ -564,20 +235,21 @@ def freeze_bn(self, enable=False):
    def forward(self, x):
        """ The input should be of size [batch_size, 3, img_h, img_w] """
        _, _, img_h, img_w = x.size()
+        cfg = self.cfg
        cfg._tmp_img_h = img_h


Though it looks like you use both cfg and self.cfg in the same function here. I'd pick one and stick with it.

data/config.py

dbolya · 2020-02-10T22:21:46Z

layers/modules/fast_mask_iou.py

+    print('Multiple GPUs detected! Turning off JIT.')
+
+ScriptModuleWrapper = torch.jit.ScriptModule if use_jit else nn.Module
+script_method_wrapper = torch.jit.script_method if use_jit else lambda fn, _rcn=None: fn


This part should probably go in its own file, since it should only be loaded once and might need to be used in multiple places.

not sure why that's important, but like this fccd89b ?

layers/modules/fpn.py

to be continued in following commits

currently only "gpu" is supported. In future, cpu,tpu. Use `net = Yolact(device_type="gpu")` `

as suggested

solving a FIXME which I thought I cannot access the object. Thanks dboya for explanation: it's an object wrapping the dict, and I can access it, just my mistake was relying on an IDE, which cannot see through it.

and imported where needed

initial python package for YOLACT. run `pip install .` And use as `from yolact import Yolact`

breznak · 2020-05-27T17:50:54Z

@dbolya when you have time, could we please proceed with this PR and #456 ? With those, the code is already quite useful for API-use, while still not yet a proper python package.

move config and init weights to __init__

f75754d

so it's done automatically, not by hand after init.

breznak commented Feb 7, 2020

View reviewed changes

breznak added 8 commits February 7, 2020 13:54

separate Concat nn.Module

286c496

move PredictionModule to separate class

e6815e1

import fixes to PredictionModule

e752308

FPN as a separate module

a0905eb

separate FastMaskIoUNet module

d824333

Yolact class fix self.cfg

913045f

and imports. Cfg /config/ is not global, but passed as self.cfg, or as arg to methods.

fix PredictionModule args, workaround imports

0aecc3d

from Config

last fixes

39739f4

breznak requested review from dbolya and chongzhou96 February 7, 2020 14:16

breznak commented Feb 7, 2020

View reviewed changes

dbolya mentioned this pull request Feb 10, 2020

how to test on a single image and get masks? #39

Closed

breznak mentioned this pull request Feb 10, 2020

Enh: move train and eval scripts to methods, pip package #299

Open

dbolya reviewed Feb 10, 2020

View reviewed changes

breznak added 5 commits February 12, 2020 08:55

review feedback

eb61db9

to be continued in following commits

prepare support for selecting device_type

236352d

currently only "gpu" is supported. In future, cpu,tpu. Use `net = Yolact(device_type="gpu")` `

move setting to CUDA in init to top

2f73774

as suggested

access mask_type.direct directly,

cf2d4e6

solving a FIXME which I thought I cannot access the object. Thanks dboya for explanation: it's an object wrapping the dict, and I can access it, just my mistake was relying on an IDE, which cannot see through it.

ScriptModuleWrapper in a sepatate file

fccd89b

and imported where needed

This was referenced Feb 19, 2020

YOLACT as a package? #343

Open

Any c++ code implementation to use in production with OpenCV? #354

Open

breznak added 2 commits May 16, 2020 23:09

Merge branch 'master' into eval_api

5a5d81d

python package

550d83a

initial python package for YOLACT. run `pip install .` And use as `from yolact import Yolact`

breznak mentioned this pull request May 27, 2020

API for eval, prep_display #456

Open

breznak requested a review from dbolya May 27, 2020 17:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP API for Yolact, python package #323

WIP API for Yolact, python package #323

breznak commented Feb 7, 2020 •

edited

Loading

breznak Feb 7, 2020

breznak left a comment

breznak Feb 7, 2020

dbolya Feb 10, 2020

dbolya Feb 10, 2020

abhigoku10 commented Feb 9, 2020

breznak commented Feb 9, 2020

dbolya left a comment

dbolya Feb 10, 2020

breznak Feb 12, 2020

dbolya Feb 10, 2020

breznak Feb 12, 2020

breznak Feb 12, 2020

dbolya Feb 10, 2020

dbolya Feb 10, 2020

dbolya Feb 10, 2020

dbolya Feb 10, 2020

breznak Feb 12, 2020

breznak commented May 27, 2020

WIP API for Yolact, python package #323

Are you sure you want to change the base?

WIP API for Yolact, python package #323

Conversation

breznak commented Feb 7, 2020 • edited Loading

Choose a reason for hiding this comment

breznak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abhigoku10 commented Feb 9, 2020

breznak commented Feb 9, 2020

dbolya left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

breznak commented May 27, 2020

breznak commented Feb 7, 2020 •

edited

Loading