feiyuhuahuo / yolact_minimal Goto Github PK

in configs the lr is 0.00005*(train_bs/8),could you tell me the default valve 8 means what.

is it right i set the value 2 if i have two gpus

Hi @feiyuhuahuo,

Thanks for your Amazing work!.

I have a Dataset consisting of 8Million images and I have a separate annotation file for each image which is in JSON format.
The sample JSON is available at this link: https://drive.google.com/file/d/12lASRiHGm5fhbHP5YNCaQM-SwfvnGIcR/view?usp=sharing. This is for only one image, similarly, we have it for all the images.

My Query here is: do we have an annotation format having one annotation file per image that is supported by the repo? If yes, I will then write a simple script for the format conversion.

But the issue with using the Coco format(single format file for all the images) is that: The single JSON file for 8Million images is becoming very big and it eats up the memory.

So, what other alternative way I can look for with the present huge Dataset and the annotations for training the model?

looking forward to your reply.

Thanks and Regards

I changed the config.py but get error on pre-weight
That is error:
Traceback (most recent call last):
File "train.py", line 51, in
assert re.findall(r'res.+[a-z]+', args.resume)[0] == cfg_name, 'Resume weight is not compatible with current cfg.'
IndexError: list index out of range
Traceback (most recent call last):
File "train.py", line 51, in
assert re.findall(r'res.+[a-z]+', args.resume)[0] == cfg_name, 'Resume weight is not compatible with current cfg.'
IndexError: list index out of range
Traceback (most recent call last):
File "/home/zhsy-zgh/anaconda3/envs/yolact/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/zhsy-zgh/anaconda3/envs/yolact/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/zhsy-zgh/anaconda3/envs/yolact/lib/python3.6/site-packages/torch/distributed/launch.py", line 263, in
main()
File "/home/zhsy-zgh/anaconda3/envs/yolact/lib/python3.6/site-packages/torch/distributed/launch.py", line 259, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/home/zhsy-zgh/anaconda3/envs/yolact/bin/python', '-u', 'train.py', '--local_rank=2', '--resume=model/swin_tiny.pth']' returned non-zero exit status 1.

Hello everyone,

I want to detect cars in aerial images from a low altitude(drones) with YOLACT. So far I've gathered data that has yet to be labeled.
The thing is, the original images vary in aspect ratios - 5472x3648 (3:2) and 3840x2160(16:9) respectively.

When resizing the images to the input size of YOLACT (550x550) the cars looked either squished or very elongated. I'm concerned that YOLACT won't be able to learn the masks of the cars properly as they look squished or elongated depending on their pose.
Therefore, my questions:
1.) Are aspect ratios besides 1:1 supported? Or even multiple aspect ratios like in my case?
2.) If not, should I just let them resize to 550x550 and hope for the best?
3.) OR, Should I crop them to 1:1 and lose information or use padding with black bars?

Besides that, I'm uncertain about cut off cars. Should I label them or only those who are fully present in the frame?

A lot of questions I know, but I would be very happy about your help

Regards,
Ruwen

Hi, I want to use the detect.py to cut out some images. But when I was running:
python detect.py --weight=weights/best_29.3_res50_coco_400001.pth --image=images
I got:

Model loaded with weights/best_29.3_res50_coco_400001.pth.

Traceback (most recent call last):
  File "detect.py", line 57, in <module>
    progress_bar = ProgressBar(40, ds)
  File "Yolact_minimal/utils/common_utils.py", line 23, in __init__
    self.update_str()
  File "Yolact_minimal/utils/common_utils.py", line 26, in update_str
    num_bars = int(self.length * (self.cur_val / self.max_val))
ZeroDivisionError: division by zero

I check the line 56 in detect.py, it seems that the length of data_loader is zero.
I'm new to pytorch. Can you help me? Thanks!

I am plotting the bounding boxes after the augmentation Its not matching with ground truth boxes .
The red color box is the Ground truth box
white color box is getting from after augmentation

Hi,
I have a very small dataset. I would like to train only the final layer after freezing all other layers. How can I do that? Any help is highly appreciated.

I converted res50.pth (best_29.3_res50_coco_400001.pth) to onnx using export2onnx.py, The predictions with onnx are not good.

Hi @feiyuhuahuo,

I just wanted to know where actually the Validation is happening on the Validation Dataset?
By the way, Do we have to give the Validation Dataset separately?

Thanks

I'm getting the above error.

Hi, I would like to detect only one class. Not able to understand where should I make the change? Thank you

Hi!
When I run detect by using my weight that Trained by my own dataset, I meet this wrong message:

Traceback (most recent call last):
File "/home/zgh/Yolact_minimal-master/detect.py", line 124, in
frame_trans = val_aug(frame_origin, cfg)
File "/home/zgh/Yolact_minimal-master/utils/augmentations.py", line 221, in val_aug
img = multi_scale_resize(img, resize_range=val_size, during_training=False)
File "/home/zgh/Yolact_minimal-master/utils/augmentations.py", line 188, in multi_scale_resize
return cv2.resize(img, (resize_range, resize_range))
TypeError: an integer is required (got type res101_custom)

More: I detected some images and no problem. Thank you for your great project!

Hi @feiyuhuahuo thanks for your work. Have you tried to convert models to use it on cell phone?
I met some trouble when converting. could you pls share if you have experience?

Hi @feiyuhuahuo,

I am converting yoloact model (trained on my custom dataset) to ONNX, the conversion is completing successfully, but in the ONNX model one of the node is not getting connected to the graph.
I have attached the picture of the graph below.

Having said that, the pytorch model Inference is working fine, but not sure why one of the node is not getting connected to the graph in the onnx model.

Can you please assist me to solve this error? Would be a great help from yourside.

Also, I tried with torch version 1.6, 1.7 but still the same.

I am attaching the screenshot of the graph below:

Thanks,
Darshan

I try to train on my own dataset with only one class, but I'm not sure how to modify CUSTOM_LABEL_MAP to adapt to my dataset. Could you help me?

Hello, when I train my own data set (512×512), the graphics card occupancy rate is 0% (but the graphics card memory occupies about 10%), and there is no intermediate process, such as loss function display, why is this?

Traceback (most recent call last):
File "detect.py", line 73, in
ids_p, class_p, box_p, coef_p, proto_p = nms(class_p, box_p, coef_p, proto_p, net.anchors, cfg)
File "/root/paddlejob/workspace/env_run/video_tag/code/yolact/utils/output_utils.py", line 153, in nms
box_thre = torch.clip(box_thre, min=0., max=1.)
AttributeError: module 'torch' has no attribute 'clip'

@feiyuhuahuo thanks for you wonderful work and sharing the code based , i want to add a multiclass label classification for person which gives the gender and age , i want to add this to the yolact architecture as a head any suggestions
Thansk in adavance

你好，我训练自己数据集的时候配置参数
if self.mode == 'train':
self.train_imgs = 'my_dataset/'
self.train_ann = 'my_dataset/custom_ann.json'
但是运行train.py的时候报这个错，
File "/app/Yolact_minimal/utils/coco.py", line 78, in getitem
assert osp.exists(img_path), f'Image path does not exist: {img_path}'
Image path does not exist: my_dataset/my_dataset\1.jpeg
请问是哪里设置错了吗

Hi,

Hope you are all well !

I forked your repository, https://github.com/x0rzkov/Yolact_minimal/, and made some additions like adding a dockerfile for cpu and gpu, but one of my new feature is not working.

I wanted to create a web service for testing results from postman, and I have an error like below:

Traceback (most recent call last):
  File "./server.py", line 131, in process
    img_numpy = draw_img(results, img_origin, args)
  File "/yolact/utils/output_utils.py", line 257, in draw_img
    class_ids, classes, boxes = [x[:args.visual_top_k].cpu().numpy() for x in results[:3]]
  File "/yolact/utils/output_utils.py", line 257, in <listcomp>
    class_ids, classes, boxes = [x[:args.visual_top_k].cpu().numpy() for x in results[:3]]
RuntimeError: Can't call numpy() on Variable that requires grad. Use var.detach().numpy() instead.

To run the websever in local

pip3 install requests werkzeug flask Image
python3 server.py

To run the webserver with docker

docker network create web
docker-compose up yolact

Any idea what is the problem ?
Do you want me to make a pull request for the docker files ?

Cheers,
X

Besides, results/images and results/videos will be created, too.

Hi,

Hope you are all well again :-) !

I wanted to know if we can implement an option to crop the results into separate pictures. I am talking about both; the bounding box and the instance segmentation.

Thanks in advance for your insights and inputs on the topic.

Cheers,
X

Hi, thanks for the past.
I want to do another task that segment multi classes in a picture and save a picture including all of them.
For example, when you can segment a person and a handbag in a picture, I want you to save the pictures of the person and a handbag.
Thank you.

Traceback (most recent call last):
File "train.py", line 205, in
table, box_map, mask_map = compute_val_map(net.module)
File "train.py", line 59, in compute_val_map
table, box_row, mask_row = evaluate(yolact_net, val_dataset, during_training=True)
File "/xw/Yolact_minimal/eval.py", line 308, in evaluate
nms_outs = NMS(net_outs, traditional_nms)
File "/xw/Yolact_minimal/utils/output_utils.py", line 116, in NMS
box_decode = decode(box_p, anchors) # [19248, 4]
File "/xw/Yolact_minimal/utils/box_utils.py", line 173, in decode
boxes = torch.cat((priors[:, :2] + box_p[:, :2] * variances[0] * priors[:, 2:],
RuntimeError: The size of tensor a (19248) must match the size of tensor b (1212624) at non-singleton dimension 0

Do you know how to draw the loss curve of YOLACT?

Hi @feiyuhuahuo,

Thanks for your great work.

I am training the Yolact_minimal on the custom Dataset. During the training I am facing the issue as: IndexError: list index out of range

I am attaching the clear error trace below.

Traceback (most recent call last):
  File "train.py", line 102, in <module>
    for images, targets, masks in data_loader:
  File "/home/acer-pred/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/acer-pred/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/home/acer-pred/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/home/acer-pred/.local/lib/python3.6/site-packages/torch/_utils.py", line 425, in reraise
    raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/acer-pred/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/acer-pred/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 48, in fetch
    return self.collate_fn(data)
  File "/home/acer-pred/Yolact_minimal/utils/coco.py", line 124, in train_collate
    valid_batch.append(valid_batch[i])
IndexError: list index out of range

Can you please help me to resolve the issue? Would help me a lot

Thanks and Regards

Hi,
Do you think it is possible to improve the performance of crop function ?

It slows down the model and introduces lag.
Another issue I found was, the bounding box of one subject is cutting out other subject's mask as well.

It seems something wrong with the lr setting when using multiple gpus.
As the reduction method of the loss function is sum. If the lr also multiplies by the number of gpus, the loss would immediately explode.

This problem occurs when I train the model in 2 gpus and batchsize=8 , epoch = 0, iteration = 910. It seems like there is a wrong with 'match' function.

each_box_max, each_box_index = overlaps.max(1)    # size [num_objects], the max IoU for each gt box
RuntimeError: cannot perform reduction function max on tensor with no elements because the operation does not have an identity

Hi @feiyuhuahuo,

Thanks for your great work.

I am training Yoloact_minimal with Custom Annotation format and modified Dataloader. But whenever the training gets started, then I am facing issue as _pickle.PicklingError: Can't pickle <class 'numpy.core._exceptions.UFuncTypeError'>: it's not the same object as numpy.core._exceptions.UFuncTypeError. I do not have clue on why the issue is coming and what is causing.

I am attaching the detailed trace below:

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
    obj = _ForkingPickler.dumps(obj)
  File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
    cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <class 'numpy.core._exceptions.UFuncTypeError'>: it's not the same object as numpy.core._exceptions.UFuncTypeError

Can you please assist me in solving the issue. Would be a great help from your side.

Looking forward to your reply.

Thanks.

Hi, thank you for your project.But when I run "python train.py --config=res101_custom_config", it occurs:

Traceback (most recent call last):
File "", line 888, in _find_spec
AttributeError: 'PyxImporter' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "train.py", line 20, in
from eval import evaluate
File "/Users/liukaijun/lkj项目/小样本检测/5_Yolact的最小复现/eval.py", line 18, in
from utils.output_utils import after_nms, NMS
File "/Users/liukaijun/lkj项目/小样本检测/5_Yolact的最小复现/utils/output_utils.py", line 13, in
from utils.cython_nms import nms as cnms
File "", line 971, in _find_and_load
File "", line 951, in _find_and_load_unlocked
File "", line 890, in _find_spec
File "", line 864, in _find_spec_legacy
File "/Users/liukaijun/anaconda3/lib/python3.6/site-packages/pyximport/pyximport.py", line 253, in find_module
fp, pathname, (ext,mode,ty) = imp.find_module(fullname,package_path)
File "/Users/liukaijun/anaconda3/lib/python3.6/imp.py", line 271, in find_module
"not {}".format(type(path)))
RuntimeError: 'path' must be None or a list, not <class '_frozen_importlib_external._NamespacePath'>

Sorry for troubling you, and do you know how to solve with it?

I annotated my dataset with lamebe, which is to generate a JSON file for each picture, but in the code, "train_images": "path_to train_images", "train_info": "path_to train_annotation", which should be a JSON file generated by all pictures. Please ask how to do this?Can you help me?

Hello, I noticed that the original project has an MIT license - could you please add one to this project as well?

Hi, I have one more question.
I wanted to segment the pictures of a person with cutout,but I got an error below.
NUM_DETECTED : 1 Traceback (most recent call last): File "detect.py", line 82, in <module> img_numpy = draw_img2(ids_p, class_p, boxes_p, masks_p, img_origin, cfg, img_name=img_name,count=i) File "C:\Users\###\Documents\Yolact_minimal\utils\output_utils.py", line 392, in draw_img2 cv2.imwrite(f'results/images/img{str(count+1).zfill(5)}_{str(i)}.png', img_matting) cv2.error: OpenCV(4.5.2) C:\Users\runneradmin\AppData\Local\Temp\pip-req-build-1bq9o88m\opencv\modules\imgcodecs\src\loadsave.cpp:721: error: (-215:Assertion failed) !_img.empty() in function 'cv::imwrite'
It seems like img_matting is empty even though num_detected shows 1.
Why does this happen?
Thanks.

when I export the swin_tiny model(download from github) to onnx , meet the error below:

Traceback (most recent call last):
  File "export2onnx.py", line 30, in <module>
    opset_version=args.opset, enable_onnx_checker=True)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/__init__.py", line 230, in export
    custom_opsets, enable_onnx_checker, use_external_data_format)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py", line 91, in export
    use_external_data_format=use_external_data_format)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py", line 639, in _export
    dynamic_axes=dynamic_axes)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py", line 421, in _model_to_graph
    dynamic_axes=dynamic_axes, input_names=input_names)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py", line 203, in _optimize_graph
    graph = torch._C._jit_pass_onnx(graph, operator_export_type)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/__init__.py", line 263, in _run_symbolic_function
    return utils._run_symbolic_function(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py", line 930, in _run_symbolic_function
    symbolic_fn = _find_symbolic_in_registry(domain, op_name, opset_version, operator_export_type)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/utils.py", line 888, in _find_symbolic_in_registry
    return sym_registry.get_registered_op(op_name, domain, opset_version)
  File "/usr/local/lib/python3.6/dist-packages/torch/onnx/symbolic_registry.py", line 111, in get_registered_op
    raise RuntimeError(msg)
RuntimeError: Exporting the operator roll to ONNX opset version 12 is not supported. Please open a bug to request ONNX export support for the missing operator.

Could you please give me some advices? Thank you!!!

I am getting no annotation in this image error.??

Hi, I'm new here.
I found this model really fantastic.
Now, I am trying to use this to cut out the background of image.
And I got the error above.
How can I sort it out?

你好，感谢你的开源，我运行代码的时候，出现这个问题，
from cython_nms import nms as cnms
ModuleNotFoundError: No module named 'cython_nms'，请问是版本问题吗 有什么解决方案吗 谢谢

When I add background class in train.json with Segmentation is [], the error occured as below:

Traceback (most recent call last):
  File "train_pizza.py", line 85, in <module>
    for images, targets, masks in train_data_loader:    # 开始训练
  File "/home/byronnar/anaconda3/envs/dev/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 363, in __next__
    data = self._next_data()
  File "/home/byronnar/anaconda3/envs/dev/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 403, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/home/byronnar/anaconda3/envs/dev/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/byronnar/anaconda3/envs/dev/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/mnt/c/Users/MLT/Desktop/projects/instance_segmentation/yolact_onnx_parser_trt/utils/coco.py", line 90, in __getitem__
    if bbox[0] < 0 or bbox[1] < 0 or bbox[2] < 4 or bbox[3] < 4:
IndexError: list index out of range

How should I modify? Thank you!

I was going through your code and I couldn't understand why are you dividing the mask_loss by area which is in normalized form (code).

The code is:
mask_loss = mask_loss.sum(dim=(0, 1)) / pos_get_csize[:, 2] / pos_get_csize[:, 3]

I guess you are trying to find the area of each prior box and divide it to the mask_loss. But the coordinates are in normalized from (if I'm not wrong).

any help appreciated !

Hi @feiyuhuahuo,

Thanks for your great work.

I am training the Yolact_minimal on the custom Dataset. During the training I am facing the issue as: IndexError: list index out of range

I am attaching the clear error trace below.

 File "train.py", line 102, in <module>
    for images, targets, masks in data_loader:
  File "/anaconda/envs/py38_default/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/anaconda/envs/py38_default/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
    return self._process_data(data)
  File "/anaconda/envs/py38_default/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
    data.reraise()
  File "/anaconda/envs/py38_default/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise
    raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 3.
Original Traceback (most recent call last):
  File "/anaconda/envs/py38_default/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
    data = fetcher.fetch(index)
  File "/anaconda/envs/py38_default/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/anaconda/envs/py38_default/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/media/sladmin/6bb2b407-f222-4e9d-bdcb-a8062f6be6ee/darshan/Yolact_minimal/utils/coco.py", line 334, in __getitem__
    rles = maskUtils.frPyObjects(segm, height, width)
  File "pycocotools/_mask.pyx", line 292, in pycocotools._mask.frPyObjects
IndexError: list index out of range

Can you please help me to resolve the issue? Would help me a lot

Thanks and Regards

Hi, I was getting better results about a month back. The weights are different as well, Can you share the links of old weights?
Thank you

when I use this command(python3 detect.py --trained_model=yolact_base_54_800000.pth --video ./../../video/test2_2s.mkv) to detect a video, blow errors will be occured. Pleased tell me how to slove it.

pytorch1.8.1,python=3.8,bs=8,opencv_python4.5.1.48

Traceback (most recent call last):
  File "train.py", line 167, in <module>
    table, box_row, mask_row = evaluate(net.module, cfg, step)
  File "/Aistation010806/Yolactv1/Yolact_minimal-dev/eval.py", line 52, in evaluate
    ids_p, class_p, boxes_p, masks_p = after_nms(ids_p, class_p, box_p, coef_p, proto_p, img_h, img_w)
  File "/Aistation010806/Yolactv1/Yolact_minimal-dev/utils/output_utils.py", line 261, in after_nms
    masks = F.interpolate(masks.unsqueeze(0), (ori_size, ori_size), mode='bilinear', align_corners=False).squeeze(0)
  File "/opt/conda/envs/dual_learning/lib/python3.8/site-packages/torch/nn/functional.py", line 3554, in interpolate
    return torch._C._nn.upsample_bilinear2d(input, output_size, align_corners, scale_factors)
RuntimeError: Non-empty 4D data tensor expected but got a tensor with sizes [1, 0, 160, 160]

Could you tell me how can I solve this mistake. ; )
thx ; )

Hello again.
Thanks for the past a lot.
Now, I want to know the ratio of how much mask includes in bbox.
I think I can calculate it for here below.
masks_semantic = mask_p * (ids_p[:, None, None] + 1) # expand ids_p' shape for broadcasting # The color of the overlap area is different because of the '%' operation. masks_semantic = masks_semantic.astype('int').sum(axis=0) % (cfg.num_classes - 1) color_masks = COLORS[masks_semantic].astype('uint8') img_fused = cv2.addWeighted(color_masks, 0.2, img_origin, 0.8, gamma=0)

Thank you.

Hi.
I appreciate your work!
I was trying to train my custom dataset, but The following error has occurred.

Traceback (most recent call last):
File "train.py", line 44, in
cfg = get_config(args, mode='train')
File "C:\Users\user\models\yolact\config.py", line 218, in get_config
dist.init_process_group(backend='nccl', init_method='env://')
File "C:\Users\user\anaconda3\envs\yolact\lib\site-packages\torch\distributed\distributed_c10d.py", line 421, in init_process_group
init_method, rank, world_size, timeout=timeout
File "C:\Users\user\anaconda3\envs\yolact\lib\site-packages\torch\distributed\rendezvous.py", line 82, in rendezvous
raise RuntimeError("No rendezvous handler for {}://".format(result.scheme))
RuntimeError: No rendezvous handler for env://
Traceback (most recent call last):
File "C:\Users\user\anaconda3\envs\yolact\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "C:\Users\user\anaconda3\envs\yolact\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\user\anaconda3\envs\yolact\lib\site-packages\torch\distributed\launch.py", line 260, in
main()
File "C:\Users\user\anaconda3\envs\yolact\lib\site-packages\torch\distributed\launch.py", line 256, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['C:\Users\user\anaconda3\envs\yolact\python.exe', '-u', 'train.py', '--local_rank=0', '--cfg=res101_custom']' returned non-zero exit status 1.

OS : Windows10
pytorch : 1.7.0
CUDA : 11.0
The inference is working fine.
How do I solve this problem?