Comments (82)
@abhigoku10 @dbolya EfficientNet is really efficient!!! I used the implementation and pretrained weights from EfficientNet-PyTorch. To validate the availability, i did two experiments:
-
EfficientNet-B0 has similar Top-5 Acc.(93.2%) with ResNet-50(93.0%) on ImageNet. When i change the backbone of YOLACT to EfficientNet-B0, the convergence rate and mAP between them almost is same. But the total model sizes are YOLACT550-EfficientNet-B0(47MB) and YOLACT550-ResNet-50(120MB).
-
EfficientNet-B4 has similar Top-5 Acc.(96.3%) with SENet(96.2%) on ImageNet, and its parameters size(19M) is less than ResNet-50(26M). When i change the backbone of YOLACT to EfficientNet-B4, i found it may achieve a better performance beyond the previous network. After 55 epochs training, now it have achieved 30.95 mAP on COCO test-dev, and it seems not converge completely. It is important to note that the YOLACT550-EfficientNet-B4 model size is only 101MB.
from yolact.
Sounds good! If it ends up working well, I might just make an official MobileNet-v2 version for YOLACTv1.1. That seems like it would be nice to have.
from yolact.
@dbolya Hi, I have done a preliminary test on my own simple dataset. And the experiment result made me feel excited~! With the help of MobileNet-v2, I successfully reduced the model size from over 100 MB to 31 MB, but without a significant performance loss on my own dataset. Next, i will try it on MS-COCO dataset, and i hope for the fantastic performance^_^
from yolact.
OK, i try it.
What do you mean by "anchors setting"? how do i need to change it?
What was the accuracy/map on you'r network?
What was the FPS? on what GPU?
Can you tell me what was the FPS on one of the "default" YOLACT backbone?Thanx!!
26.8 mAP for B0, 29.6mAP for B3.
GTX 1080Ti, base YOLACT 37 FPS
EfficientNet-B0 41FPS
from yolact.
THX~!
I used the implementation and the pretrained weights from this repo:https://github.com/tonylins/pytorch-mobilenet-v2
I did the experiment follow these steps bellow:
- Modify the network architecture by removing the final global average pooling layer and the final 1x1 conv layer;
- Make a custom
init_backbone()
method in which i translated the node name and discarded the unuseful tensor; - Add a new config-dict to the 'config.py', and set some helpful parameters.
Here is my MobileNet-v2 implementation:
class MobileNetV2(nn.Module):
def __init__(self):
super(MobileNetV2, self).__init__()
block = InvertedResidual
input_channel = 32
interverted_residual_setting = [
# t, c, n, s
[1, 16, 1, 1],
[6, 24, 2, 2], # useful
[6, 32, 3, 2], # useful
[6, 64, 4, 2],
[6, 96, 3, 1], # useful
[6, 160, 3, 2],
[6, 320, 1, 1], # useful
]
# Select the output layers
# idx-1 => 56 * 56 * 24
# idx-2 => 28 * 28 * 32
# idx-4 => 14 * 14 * 96
# idx-6 => 7 * 7 * 320
select_idxs = [1, 2, 4, 6]
# building first layer
input_channel = int(input_channel)
self.conv2d = conv_bn(3, input_channel, 2)
self.layers = nn.ModuleList()
self.channels = []
# building inverted residual blocks
features = []
for idx, (t, c, n, s) in enumerate(interverted_residual_setting):
output_channel = int(c)
for i in range(n):
if i == 0:
features.append(block(input_channel, output_channel, s, expand_ratio=t))
else:
features.append(block(input_channel, output_channel, 1, expand_ratio=t))
input_channel = output_channel
if idx in select_idxs:
self.layers.append(nn.Sequential(*features))
self.channels.append(output_channel)
features = []
self.backbone_modules = [m for m in self.modules() if isinstance(m, nn.Conv2d)]
# random initial
self._initialize_weights()
def forward(self, x):
x = self.conv2d(x)
outs = []
for layer in self.layers:
x = layer(x)
outs.append(x)
return tuple(outs)
def _initialize_weights(self):
modules = self.modules()
for m in modules:
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
if m.bias is not None:
m.bias.data.zero_()
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()
def init_backbone(self, path):
""" Initializes the backbone weights for training. """
state_dict = torch.load(path, map_location='cpu')
# Replace featuresXXX -> layers.x.xx(/conv2d.x) etc.
keys = list(state_dict)
for key in keys:
if key.startswith('features'):
idx = int(key.split('.')[1])
if idx <= 17:
if idx == 0:
new_key = 'conv2d.' + key[11:]
elif (idx >= 1 and idx <= 3):
new_key = "layers.0." + str(idx - 1) + key[10:]
elif (idx >= 4 and idx <=6):
new_key = "layers.1." + str(idx - 4) + key[10:]
elif (idx >= 7 and idx <= 13):
if idx <= 9:
new_key = "layers.2." + str(idx - 7) + key[10:]
else:
new_key = "layers.2." + str(idx - 7) + key[11:]
else:
new_key = "layers.3." + str(idx - 14) + key[11:]
state_dict[new_key] = state_dict.pop(key)
else:
state_dict.pop(key)
else:
state_dict.pop(key)
# Note: Using strict=False is berry scary. Triple check this.nn.ModuleList()
self.load_state_dict(state_dict, strict=False)
And there is the corresponding config:
mobilenetv2_backbone = resnet50_backbone.copy({
'name': 'MobileNetV2',
'path': 'xxx.pth',
'type': MobileNetV2,
'args': (),
# 'transform': resnet_transform,
'selected_layers': [1, 2, 3],
'pred_scales': [[24], [48], [96], [192], [384]],
'pred_aspect_ratios': [[[1.414]]] * 5,
'use_pixel_scales': False,
})
yolact_mobilenetv2_coco_config = yolact_resnet50_config.copy({
'name': 'yolact_mobilenetv2_coco',
'backbone': mobilenetv2_backbone.copy(),
'use_semantic_segmentation_loss': True
})
from yolact.
@abhigoku10 EfficientNets and MobileNet-v3 is coming~~~
from yolact.
@abhigoku10 can you share your yolact-efficient code?
from yolact.
@abhigoku10 @dbolya EfficientNet is really efficient!!! I used the implementation and pretrained weights from EfficientNet-PyTorch. To validate the availability, i did two experiments:
- EfficientNet-B0 has similar Top-5 Acc.(93.2%) with ResNet-50(93.0%) on ImageNet. When i change the backbone of YOLACT to EfficientNet-B0, the convergence rate and mAP between them almost is same. But the total model sizes are YOLACT550-EfficientNet-B0(47MB) and YOLACT550-ResNet-50(120MB).
- EfficientNet-B4 has similar Top-5 Acc.(96.3%) with SENet(96.2%) on ImageNet, and its parameters size(19M) is less than ResNet-50(26M). When i change the backbone of YOLACT to EfficientNet-B4, i found it may achieve a better performance beyond the previous network. After 55 epochs training, now it have achieved 30.95 mAP on COCO test-dev, and it seems not converge completely. It is important to note that the YOLACT550-EfficientNet-B4 model size is only 101MB.
Can you please share your EfficientNet backbone?
from yolact.
Hmm, can you still do a sanity check and change self.load_state_dict(state_dict, strict=False)
to self.load_state_dict(state_dict, strict=True)
and see if there's anything unexpected is missing? I think even with one anchor per layer, you shouldn't get < 1 mAP on the first validation epoch.
from yolact.
@Wilbur529 i was able to train the model using mobilenet architecture, thanks for sharing the reference implementation , when i tested it on images its able to detect with low accuracy but when i run it for a video the detection are not happening at all . Did u face the similar issue , should we make any changes with the code
from yolact.
@abhigoku10 I let it ran on only one single GPU, so it needed long time for training. At 200 epochs, the mAP are 14.36(box) and 15.02(mask).
from yolact.
I am reading both of their paper, and i haven't found a suitable pytorch implementation of them~ Once I succeed in my experiment, i will show the modified parts.
from yolact.
@Wilbur529 thanks for the update eagers waiting for this
from yolact.
What does 'selected_layers': [1, 2, 3] mean?
from yolact.
@zhangbaoj I replied in your new issue.
from yolact.
@abhigoku10 Can you make a pull request for it? Then we can all review it.
from yolact.
@Wilbur529
I meet the warning " Warning: Moving average ignored a value of inf " During the training , when I change the backbone of resnet to mobilenetv2 . Did you met this warning ?
from yolact.
@xscjun I have a patch coming as a result of #222 that might fix that issue for you.
from yolact.
Thanks!
And what about the anchors? How can i know the best settings?
from yolact.
@Auth0rM0rgan i was able to implement efficient net as backbone the performance was lower compared to resnet for objects like pedistrains and the fps was higher than resent 50 , i just used the same pred_scales, ascpect ratios similar to mobile net not sure is that the right approach , i could not share the implementation due to firm polices , @Wilbur529 @dbolya can you suggest 'selected_layers': [1, 2, 3],
'pred_scales': [[24], [48], [96], [192], [384]],
'pred_aspect_ratios': [[[1.414]]] * 5,
'use_pixel_scales': False, is this the rigthvalues selected
from yolact.
That would be a good test, though I will warn that we've tried VGG16+FPN before and got sub-par performance (though maybe that's to be expected). If you want to change out the backbone head over to backbone.py
and check out how I implemented Resnet
and VGG
. If you need any help, let me know.
from yolact.
@dbolya Hi, could you show me your train log file of resnet&coco?
from yolact.
This is my recently train log with mobilenetv2&coco...
[ 16] 59810 || B: 2.489 | C: 3.295 | M: 2.861 | S: 0.897 | T: 9.542 || ETA: 19 days, 23:21:26 || timer: 2.437
[ 16] 59820 || B: 2.505 | C: 3.321 | M: 2.840 | S: 0.898 | T: 9.565 || ETA: 19 days, 22:54:13 || timer: 2.125
[ 16] 59830 || B: 2.500 | C: 3.299 | M: 2.802 | S: 0.910 | T: 9.511 || ETA: 19 days, 22:34:59 || timer: 1.966
[ 16] 59840 || B: 2.504 | C: 3.271 | M: 2.787 | S: 0.900 | T: 9.461 || ETA: 19 days, 22:51:36 || timer: 2.071
[ 16] 59850 || B: 2.480 | C: 3.282 | M: 2.751 | S: 0.881 | T: 9.394 || ETA: 19 days, 22:43:14 || timer: 2.415
[ 16] 59860 || B: 2.462 | C: 3.247 | M: 2.752 | S: 0.883 | T: 9.344 || ETA: 19 days, 22:27:50 || timer: 2.144
[ 16] 59870 || B: 2.460 | C: 3.247 | M: 2.726 | S: 0.881 | T: 9.315 || ETA: 19 days, 22:22:21 || timer: 2.091
[ 16] 59880 || B: 2.489 | C: 3.290 | M: 2.696 | S: 0.891 | T: 9.366 || ETA: 19 days, 22:38:05 || timer: 2.101
[ 16] 59890 || B: 2.478 | C: 3.300 | M: 2.712 | S: 0.892 | T: 9.382 || ETA: 19 days, 22:48:17 || timer: 2.301
[ 16] 59900 || B: 2.485 | C: 3.299 | M: 2.711 | S: 0.889 | T: 9.384 || ETA: 19 days, 22:53:21 || timer: 2.046
[ 16] 59910 || B: 2.514 | C: 3.316 | M: 2.724 | S: 0.880 | T: 9.435 || ETA: 19 days, 23:27:14 || timer: 1.973
[ 16] 59920 || B: 2.505 | C: 3.308 | M: 2.721 | S: 0.877 | T: 9.410 || ETA: 19 days, 23:25:19 || timer: 2.257
[ 16] 59930 || B: 2.496 | C: 3.290 | M: 2.710 | S: 0.863 | T: 9.360 || ETA: 19 days, 23:15:29 || timer: 2.061
[ 16] 59940 || B: 2.467 | C: 3.269 | M: 2.694 | S: 0.864 | T: 9.293 || ETA: 19 days, 22:23:27 || timer: 1.924
[ 16] 59950 || B: 2.473 | C: 3.249 | M: 2.702 | S: 0.868 | T: 9.292 || ETA: 19 days, 22:55:27 || timer: 2.374
[ 16] 59960 || B: 2.497 | C: 3.261 | M: 2.702 | S: 0.858 | T: 9.318 || ETA: 19 days, 23:18:36 || timer: 2.055
[ 16] 59970 || B: 2.532 | C: 3.274 | M: 2.726 | S: 0.862 | T: 9.393 || ETA: 19 days, 23:51:10 || timer: 2.111
[ 16] 59980 || B: 2.493 | C: 3.262 | M: 2.723 | S: 0.861 | T: 9.339 || ETA: 19 days, 23:32:02 || timer: 1.991
[ 16] 59990 || B: 2.500 | C: 3.253 | M: 2.718 | S: 0.860 | T: 9.331 || ETA: 19 days, 23:44:44 || timer: 3.396
The loss value looks like going down. But the convergence speed looks so slowly. And i got a poor evaluated result:
box | 0.78 | 2.17 | 1.76 | 1.39 | 1.08 | 0.75 | 0.43 | 0.19 | 0.07 | 0.01 | 0.00 |
box | 1.31 | 3.44 | 2.92 | 2.37 | 1.78 | 1.31 | 0.78 | 0.33 | 0.14 | 0.04 | 0.00 |
box | 1.47 | 3.88 | 3.23 | 2.54 | 2.05 | 1.47 | 0.82 | 0.44 | 0.21 | 0.04 | 0.00 |
box | 2.17 | 5.26 | 4.63 | 3.84 | 3.12 | 2.26 | 1.42 | 0.77 | 0.27 | 0.11 | 0.00 |
box | 2.41 | 5.79 | 5.10 | 4.26 | 3.46 | 2.54 | 1.74 | 0.82 | 0.34 | 0.06 | 0.00 |
box | 2.65 | 6.06 | 5.34 | 4.67 | 3.76 | 2.90 | 1.96 | 1.05 | 0.51 | 0.20 | 0.03 |
box | 2.82 | 6.62 | 5.94 | 4.82 | 4.05 | 2.94 | 2.04 | 1.17 | 0.53 | 0.12 | 0.00 |
mask | 0.80 | 1.91 | 1.61 | 1.35 | 1.09 | 0.83 | 0.58 | 0.38 | 0.21 | 0.07 | 0.00 |
mask | 1.44 | 3.13 | 2.71 | 2.28 | 1.95 | 1.62 | 1.20 | 0.81 | 0.55 | 0.12 | 0.01 |
mask | 1.73 | 3.73 | 3.28 | 2.81 | 2.35 | 1.88 | 1.43 | 0.96 | 0.57 | 0.26 | 0.03 |
mask | 2.38 | 4.86 | 4.39 | 3.80 | 3.27 | 2.65 | 2.09 | 1.49 | 0.95 | 0.29 | 0.03 |
mask | 2.76 | 5.46 | 5.04 | 4.48 | 3.84 | 3.15 | 2.43 | 1.70 | 0.96 | 0.44 | 0.06 |
mask | 3.01 | 5.81 | 5.34 | 4.82 | 4.16 | 3.53 | 2.72 | 1.97 | 1.10 | 0.56 | 0.07 |
mask | 3.31 | 6.41 | 5.85 | 5.27 | 4.56 | 3.94 | 2.99 | 2.13 | 1.31 | 0.58 | 0.06 |
Hope to discuss more with you ^_^
from yolact.
Hmm with the mAP starting that low (< 1), it looks likes the behavior we get when we train from scratch. Are you using imagenet pretrained weights in your MobileNet implementation? If not, it'll take way longer to converge.
There's also a gotcha when loading pretrained weights because I had to set strict=False
in the torch.load
for loading backbone weights. Set that back to True
if you are using pretrained weights but suspect it's not loading properly.
from yolact.
haha, i found the problem...Just after i commented...
When i do test on my own dataset, i removed some other anchors on purpose... and i forgot to recover them.
from yolact.
Thx, i'm just going to do it!
from yolact.
So,what's the accuracy and fps with mobilenet v2 now?@Wilbur529
from yolact.
@andeyeluguo
It has run almost 200 epochs now, and the mAP are 14.36(box) and 15.02(mask) respectively. But I found that the model does not converge completely, so keep waiting...
from yolact.
@Wilbur529 wonderful work on the modification on the network backbone . keep going !!!!!!!!!!
from yolact.
@Wilbur529 hi is it possible to share your mobilenet implementation py ?? how good is the accuracy and fps compared to resenet 50,101
from yolact.
@abhigoku10 You could find the model and the accuracy in my previous answers, and i will do a benchmark test later. As far as I can see from the training log, i think it would be vary fast.
from yolact.
@abhigoku10 Good job! I think it may not be a difficult problem. If you couldn't find the error code, why not write a for-loop upon your image inference function to simulate the video inference:)
from yolact.
@Wilbur529 yup that was my next step , but the video execution uses threading and mutiframe buffer functionality so just trying to check the feasibility on that , but did u face the same issues with the video
from yolact.
@abhigoku10 Sry i did not face this problem. Based on your description, i think the model has no mistake. So just pay more attention to the evalvideo
method, and check the postprocess procedure, good luck for u~
from yolact.
@Wilbur529 sure i shall do it , how many iterations you had trained ur mobilenet model , since i trained with 2 lakh iterations the results on the validation set is not that great though the fps is high
from yolact.
@Wilbur529 thanks for the info i shall run it to 200 epochs and share my mAP, anyother architecture u tried besides mobilenet
from yolact.
@Wilbur529 cool any plans on sharing the reference code
from yolact.
I am reading both of their paper, and i haven't found a suitable pytorch implementation of them~ Once I succeed in my experiment, i will show the modified parts.
Looking forward for your update~there is a reference that may be helpful
from yolact.
@Wilbur529 can you share the reference code for the efficient net so that i can test it on different data , what is the fps and accuracy u achieved
from yolact.
@Wilbur529 wonderful experimentation and anlaysis , would you be able to share the yolact-efficient code , i would be able to perform certain more experiments , since effcient-b0 model size is an interesting one
THnaks in advance
from yolact.
@Wilbur529 @dbolya i have trained the model with efficient b0 and b4 here is my following observations .
- Efficient b0 is of model size of 42MB its mAP is similar to Resnet 18 which is of size 72MB but the performance of b0 for multiframe=2 is 20 fps but for resent18 with multiframe=2 is 22 fps for a resolution of 1280x720, which is strange since the model size and params was less but we getting less fps compared to resnet18
- Similary b4 is of model size of 96MB if of 16fps for multiframe=2 but Resnet 50 which is of size 125MB is of 18fps for multiframe=2
I am testing on 2080TI gpu system . any suggestions on this behaviour
from yolact.
@abhigoku10 I may suggest you only measure the network inference time:)
from yolact.
@abhigoku10 @Wilbur529 Yeah if you can, use the included --benchmark mode. Also, that might just be your implementation being slow, but having fewer parameters doesn't necessarily mean you're going to be faster if evaluation is more complex.
from yolact.
@dbolya i shall check up my implementation once again , i agree its not directly proportional that fewer params higher accuracy but is it not higher fps
from yolact.
@Wilbur529 @abhigoku10 can you help me with the efficient backbone? i trained efficientnet-b3 but i only have 30.05 box and 28.13 mask (using pretrained weights). I think my problem is in the config file, can you share that with me pls?
pd: i'm using resnet50 config copy and [3,4,5] selected layers.
Thx :)
from yolact.
@dbolya Any word regarding integrating a new backbone into yolact?
from yolact.
@deepseek I didn't intend to add any new backbones to the master branch, but I guess if enough people want mobile net or efficient net, I could try. It should be relatively simple to add new backbones yourself if you look at how they're defined in backbone.py and how their configs are set up.
from yolact.
@dbolya i have an efficient net cod but i wanted to review it is it possbile to review it before merging into the branch @Jackerz312 and @deepseek once is get it reviewed i can share it
from yolact.
@Wilbur529 ,
Can you please share your EfficientNet backbone?
How can i use it in me network?
from yolact.
What the FPS did you get?
On which GPU?
from yolact.
@IMFinethankyou , can you share you'r config and backbone file?
I don't know how to do it...
from yolact.
from yolact.
@IMFinethankyou , i tried this one, but YOLACT need more functions/fields like:
add_layer()
layers
from yolact.
@IMFinethankyou , i tried this one, but YOLACT need more functions/fields like:
add_layer()
layers
@sdimantsd , add_layer() is used in the backbone construction process, not neccessary in EfficientNet, and the #layers# only used in the load_weights() function for compatability of different version of backbone weights, also not necessary for EfficientNet
from yolact.
OK, i try it.
What do you mean by "anchors setting"? how do i need to change it?
What was the accuracy/map on you'r network?
What was the FPS? on what GPU?
Can you tell me what was the FPS on one of the "default" YOLACT backbone?
Thanx!!
from yolact.
@IMFinethankyou i performed the same thing but i had got same accuracy for B4 , as @sdimantsd was asking did u re-generate the anchors ?? i used the mobile net anchors . Can you share what changes you have made in the config file
from yolact.
@IMFinethankyou, I don't know what im doing wrong, but i steel can't manage EfficientNet to work as backbone...
I used it as backbone the following config:
efficient_net_config = yolact_base_config.copy({
'name': 'efficient_net',
# 'fpn': None,
# Dataset stuff
'dataset': ws_datasetV1,
'num_classes': len(ws_datasetV1.class_names) + 1,
# Training params
'lr_steps': (280000, 600000, 700000, 750000, 850000, 1000000, 1100000, 1200000, 1300000, 1350000, 1400000, 1450000),
'max_iter': 800000000,
'masks_to_train': 300,
'max_size': 700,
'backbone': efficient_net_backbone.copy({
'selected_layers': list(range(2, 5)),
'pred_scales': yolact_base_config.backbone.pred_scales,
'pred_aspect_ratios': yolact_base_config.backbone.pred_aspect_ratios,
'use_pixel_scales': True,
'preapply_sqrt': False,
'use_square_anchors': True, # This is for backward compatability with a bug
}),
})
(i copyed from another backbone, im not shur if it's good)
When i try to train i get the following error:
AttributeError: 'EfficientNetBackbone' object has no attribute 'channels'
in yolact.py line 426:
src_channels = self.backbone.channels
If i set backvone.channels to []
It crash in line 430:
self.fpn = FPN([src_channels[i] for i in self.selected_layers])
Can you please help me?
from yolact.
@IMFinethankyou, I don't know what im doing wrong, but i steel can't manage EfficientNet to work as backbone...
I used it as backbone the following config:
efficient_net_config = yolact_base_config.copy({
'name': 'efficient_net','fpn': None,
Dataset stuff
'dataset': ws_datasetV1,
'num_classes': len(ws_datasetV1.class_names) + 1,# Training params 'lr_steps': (280000, 600000, 700000, 750000, 850000, 1000000, 1100000, 1200000, 1300000, 1350000, 1400000, 1450000), 'max_iter': 800000000, 'masks_to_train': 300, 'max_size': 700, 'backbone': efficient_net_backbone.copy({ 'selected_layers': list(range(2, 5)), 'pred_scales': yolact_base_config.backbone.pred_scales, 'pred_aspect_ratios': yolact_base_config.backbone.pred_aspect_ratios, 'use_pixel_scales': True, 'preapply_sqrt': False, 'use_square_anchors': True, # This is for backward compatability with a bug }),
})
(i copyed from another backbone, im not shur if it's good)
When i try to train i get the following error:
AttributeError: 'EfficientNetBackbone' object has no attribute 'channels'
in yolact.py line 426:
src_channels = self.backbone.channelsIf i set backvone.channels to []
It crash in line 430:
self.fpn = FPN([src_channels[i] for i in self.selected_layers])Can you please help me?
backbone.channels is a list, contains the channels of the output feature map
for instance: you backbone output a tuple of 5 feature maps:
torch.Size([1, 64, 67, 67])
torch.Size([1, 128, 33, 33])
torch.Size([1, 176, 16, 16])
torch.Size([1, 304, 8, 8])
torch.Size([1, 512, 4, 4])
then the channels should be [64, 128, 176, 304, 512]
backbone.channels is used latter in the FPN module
from yolact.
@IMFinethankyou i performed the same thing but i had got same accuracy for B4 , as @sdimantsd was asking did u re-generate the anchors ?? i used the mobile net anchors . Can you share what changes you have made in the config file
I borrowed the config from the base-config, no special changes. the only change i made is set USE_SQUARE_ANCHORS to False.
from yolact.
@IMFinethankyou, thanks, i check it now.
How can i know the backbone output?
from yolact.
also, init_backbone() is not exist :-(
from yolact.
I don't know how you did it, Also backbone_modules is missing...
from yolact.
@IMFinethankyou Can you help me plz?
from yolact.
@IMFinethankyou Can you help me plz?
@sdimantsd, you should modify the efficientnet according to the backbone.py, so it can fit into the YOLACT. my connection to github is rediculously slow.
from yolact.
@IMFinethankyou , can you share you'r config and backbone file? thks
from yolact.
from yolact.
@dbolya Hello thanks for your amazing work. When I change the backbone to MobileNet by the following the above code which is provided by @Wilbur529, I got this error:
RuntimeError: copy_if failed to synchronize: device-side assert triggered
And sometimes I got Nan and inf in the loss However I am working with the latest version with has this problem was solved by #222 (Note: I got it only when changing the backbone to mobilenet)
@Wilbur529 @abhigoku10 Thank you for providing us by a reference code for MobilnetV2 backbone but unfortunately I doesn't work for me as I mentioned above so did you face the above problems and could help?
Are there any changes than the above provided code?
Thanks in advance.
from yolact.
I solved the above issue by lowering the learning rate to 1e-4 instead of 1e-3.
I mentioned the solution to help others who may face the same issue.
But I faced another problem now, the mAP on the validation dataset (I used a custom dataset) as shown in the below figure
Note that: The training is working fine when I used the base backbone, the problem appears only when I changed the backbone to mobilenet.
from yolact.
Hello,
@Jackerz312 @Wilbur529 @abhigoku10
could you provide us with your implementation to Efficient-net backbone as It seems I couldn't port it in right way I followed this implementation https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/model.py
I have a couple of questions @dbolya which may help me in porting it in the right manner
1- How should I set this values? will they be affected by changing the backbone?
'selected_layers': [1, 2, 3],
'pred_scales': [[24], [48], [96], [192], [384]],
'pred_aspect_ratios': [[[1.414]]] * 5,
2- Regarding the output I set it like that self.select_idxs = [1, 2, 4, 6]
3- should I take care of any other configuration while porting a new backbone?
For your information currently the loss is stack at certain high value and the mAP is zero
Thanks in advance.
from yolact.
@IMFinethankyou @Jackerz312 @Wilbur529 @abhigoku10
I use the efficientnet as the backbone show this errors :
Traceback (most recent call last):
File "train.py", line 532, in
train()
File "train.py", line 322, in train
losses = net(datum)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
raise output
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
output = module(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "train.py", line 147, in forward
preds = self.net(images)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/datasets/yolact-master/yolact.py", line 600, in forward
proto_out = self.proto_net(proto_x)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 338, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size 256 16 3 3, expected input[2, 40, 69, 69] to have 16 channels, but got 40 channels instead
the config as follow:
yolact_EfficientNet_mydataset_config = yolact_base_config.copy({
'name': 'yolact_EfficientNet',
'dataset': my_custom_dataset,
'num_classes': len(my_custom_dataset.class_names) + 1,
'fpn': None,
'max_iter': 120000,
'lr_steps': (60000, 100000),
#'mask_proto_src': 0,
'backbone': EfficientNet_backbone.copy({
'selected_layers': list(range(1, 4)),
'pred_scales': [[24], [48], [96], [192], [384]],
'pred_aspect_ratios': [[[1.414]]] * 5,
'use_pixel_scales': True,
'preapply_sqrt': False,
'use_square_anchors': False, # This is for backward compatability with a bug
}),
})
can you help me,please?
Can you please share your EfficientNet backbone code?
thks in advance
from yolact.
I solved the above issue by lowering the learning rate to 1e-4 instead of 1e-3.
I mentioned the solution to help others who may face the same issue.But I faced another problem now, the mAP on the validation dataset (I used a custom dataset) as shown in the below figure
Note that: The training is working fine when I used the base backbone, the problem appears only when I changed the backbone to mobilenet.
The gap between box mAP and mask mAP is not abnormal. You may find the answer in the author'
s new papar YOLACT++.
from yolact.
@IMFinethankyou @Jackerz312 @Wilbur529 @abhigoku10
I use the efficientnet as the backbone show this errors :
Traceback (most recent call last):
File "train.py", line 532, in
train()
File "train.py", line 322, in train
losses = net(datum)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 152, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 162, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 83, in parallel_apply
raise output
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 59, in _worker
output = module(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "train.py", line 147, in forward
preds = self.net(images)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/home/datasets/yolact-master/yolact.py", line 600, in forward
proto_out = self.proto_net(proto_x)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in call
result = self.forward(*input, **kwargs)
File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 338, in forward
self.padding, self.dilation, self.groups)
RuntimeError: Given groups=1, weight of size 256 16 3 3, expected input[2, 40, 69, 69] to have 16 channels, but got 40 channels insteadthe config as follow:
yolact_EfficientNet_mydataset_config = yolact_base_config.copy({
'name': 'yolact_EfficientNet','dataset': my_custom_dataset, 'num_classes': len(my_custom_dataset.class_names) + 1, 'fpn': None, 'max_iter': 120000, 'lr_steps': (60000, 100000), #'mask_proto_src': 0, 'backbone': EfficientNet_backbone.copy({ 'selected_layers': list(range(1, 4)), 'pred_scales': [[24], [48], [96], [192], [384]], 'pred_aspect_ratios': [[[1.414]]] * 5, 'use_pixel_scales': True, 'preapply_sqrt': False, 'use_square_anchors': False, # This is for backward compatability with a bug }),
})
can you help me,please?
Can you please share your EfficientNet backbone code?
thks in advance
It seems like you did not change the channels num properly. You can check the first few layers in the protonet.
from yolact.
Hey @Wilbur529, Have you tried EfficientNet as a backbone? if yes, what performance and FPS have you achieved compare to ResNet50? Are you planning to share your EfficientNet backbone?
Also, I have a question about your MobileNet backbone that what is the intuition behind of selecting these layers select_idxs = [1, 2, 4, 6]
not other layers?
Thanks!
from yolact.
from yolact.
@abhigoku10
i was able to implement efficient net as backbone,but after training to 1000 steps the loss gradient cannot be reduced.i use efficient-b0 b3 b4, b4 of loss gradient connot be reduced,b0 and b3 of mAP is very low.
please help !
Thanks in advance!
from yolact.
@linshaodan even i had the same behaviour i had trained for 6lakh iterations , but my mAP was low compared to resnet50 but fps was high
from yolact.
@abhigoku10
Yes, I got the same results as you said.
thank you.
from yolact.
Can you tell me what was the FPS on one of the "default" YOLACT backbone?
OK, i try it.
What do you mean by "anchors setting"? how do i need to change it?
What was the accuracy/map on you'r network?
What was the FPS? on what GPU?
Can you tell me what was the FPS on one of the "default" YOLACT backbone?
Thanx!!26.8 mAP for B0, 29.6mAP for B3.
GTX 1080Ti, base YOLACT 37 FPS
EfficientNet-B0 41FPS
@IMFinethankyou could you tell me the mAP for val set and test-dev?
from yolact.
@IMFinethankyou Hi,
I modify EfficientDet as EfficientNet backbone,
but I met this:
during training with efficientnet-b0 backbone.
And the loss doesn't reduce after iteration 1000.
The config as shown below:
efficientnet_backbone = resnet101_backbone.copy({
'name': 'EfficientNet',
'path': 'xxx.pth',
'type': EfficientNetBackbone,
'transform': efficientnet_transform,
'selected_layers': list(range(1, 4)),
})
'backbone': efficientnet_backbone.copy({
'selected_layers': list(range(1, 4)),
'use_pixel_scales': True,
'preapply_sqrt': False,
'use_square_anchors': False,
'pred_aspect_ratios': [[[1.414]]] * 5,
'pred_scales': [[24], [48], [96], [192], [384]],
}),
I don't know what goes wrong.
Can you share your config and backbone file or give some suggestions?
thks.
from yolact.
I made some study and changed the backbone to EfficientNet, you can have my source code at SpaceView/Yolact_EfficientNet, Please go there and try.
The source code uses EfficientNet-b0, but you can change it to any sub-model easily.
I can say that the training speed is really much slower even than the Resnet50, and I searched that this is well noted by lots of done work. Currently I don't have enough time to fine-tune this model, not to say finish the training.
I made my above comments 3 days ago, NOW I update my comments as below,
--> EfficientNet is difficult to train, not a good choice for yolact, as far as I can say. On CPU the overall speed is almost the same as Resnet50. the main reason is that the FPN, proto_net, predictions layers, etc. are the same, the backbone doesn't take such a dominant role as we have expected.
--> VGG16 is also not a good choice, it is much slower even than Resnet101, both in training and evaluation.
from yolact.
from yolact.
haha, i found the problem...Just after i commented... When i do test on my own dataset, i removed some other anchors on purpose... and i forgot to recover them.
did you remove anchors? where to recover them?
from yolact.
Related Issues (20)
- Hyperparameter tuning
- AP输出
- Loss curve
- Just wonder how this project make the results sync while using thread pool
- How to change the network
- How to draw a specific shape on the screen while recognizing a specific object?
- epoch
- How to pick the best epoch? HOT 1
- KeyError 2156740059
- small-, medium-, large-mAP definition?
- Question on DCN implementation
- mAP and AR: definition of small medium large ? HOT 2
- Handling negative images or images with no object of interest
- After entering the verification command, do not display the verified image
- After entering the verification command, do not display the verified image
- I get Illegal instruction (core dumped)
- "mtrand.pyx", line 936, in numpy.random.mtrand.RandomState.choice HOT 2
- How to get the coordinates and polygon information of evaluation results
- Post processing process issues
- There is a bug when I want to do test with yolact. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yolact.