Giter VIP home page Giter VIP logo

circlenet's Introduction

CircleNet: Anchor-free Detection with Circle Representation

The official implementation of CircleNet, MICCAI 2020, IEEE TMI 2021

Journal Paper

Circle Representation for Medical Object Detection,
Ethan H. Nguyen, Haichun Yang, Ruining Deng, Yuzhe Lu, Zheyu Zhu, Joseph T. Roland, Le Lu, Bennett A. Landman, Agnes B. Fogo, Yuankai Huo,
IEEE Transactions on Medical Imaging (10.1109/TMI.2021.3122835); arXiv (arXiv:2110.12093)

Conference Paper

CircleNet: Anchor-free Detection with Circle Representation,
Haichun Yang, Ruining Deng, Yuzhe Lu, Zheyu Zhu, Ye Chen, Joseph T. Roland, Le Lu, Bennett A. Landman, Agnes B. Fogo, Yuankai Huo
MICCAI 2020; arXiv (arXiv:2006.02474)

Contact: [email protected]. Feel free to reach out with any questions or discussion!

Abstract

Box representation has been extensively used for object detection in computer vision. Such representation is efficacious but not necessarily optimized for biomedical objects (e.g., glomeruli), which play an essential role in renal pathology. We propose a simple circle representation for medical object detection and introduce CircleNet, an anchor-free detection framework. Compared with the conventional bounding box representation, the proposed bounding circle representation innovates in three-fold:

(1) it is optimized for ball-shaped biomedical objects;

(2) The circle representation reduced the degree of freedom compared with box representation;

(3) It is naturally more rotation invariant. When detecting glomeruli and nuclei on pathological images, the proposed circle representation achieved superior detection performance and be more rotation-invariant, compared with the bounding box.

Highlights

  • Simple: One-sentence summary: Instead of the conventional bounding box, we propose using a bounding circle to detect ball-shaped biomedical objects.

  • State-of-the-art: On two datasets (glomeruli and nuclei), our CircleNet method outperforms baseline methods by over 10%.

  • Fast: Only requires a single network forward pass.

Installation

Please refer to INSTALL.md for installation instructions. [Update2023]Here is the new INSTALL2023.md for installation in year 2023.

CircleNet - Image Demo

CircleNet can easily be run on a single image or a folder of images.

First, download the models (By default, circledet_monuseg_hg from Model Zoo and put them in CircleNet_Root/models

We provide an example image from MoNuSeg 2018 in docs.

For nuclei detection, run

python demo.py circledet --arch hourglass --demo ../docs/demo.png --load_model ../models/circledet_monuseg_hg.pth

If set up correctly, the output should look like

To use CircleNet in your own project, you can

import sys
CIRCLENET_PATH = /path/to/CircleNet/src/lib/
sys.path.insert(0, CIRCLENET_PATH)

from detectors.detector_factory import detector_factory
from opts import opts

MODEL_PATH = /path/to/model
TASK = 'circledet'
opt = opts().init('{} --load_model {}'.format(TASK, MODEL_PATH).split(' '))
detector = detector_factory[opt.task](opt)

img = image/or/path/to/your/image(s)/
ret = detector.run(img)['results']

ret will be a Python list where each item describes a circle detection: [x, y, radius, confidence, category]

CircleNet - Whole Slide Image Demo

CircleNet can also be run on Whole Slide Images in *.scn file format.

Please download the following two files:

  1. Human Kidney WSI (case 03-1.scn)

  2. Human Kidney Model (model_10.pth)

To run it on a testing scan, please go to src folder and run

sudo apt install python-openslide
python run_detection_for_scn.py circledet --arch dla_34 --demo "/media/huoy1/48EAE4F7EAE4E264/Projects/from_haichun/batch_1_data/scn/Case 03-1.scn" --load_model /media/huoy1/48EAE4F7EAE4E264/Projects/detection/CircleNet/exp/circledet/kidpath_dla_batch4/model_10.pth --filter_boarder --demo_dir "/media/huoy1/48EAE4F7EAE4E264/Projects/detection/test_demo"

The demo_dir is output dir, which you set anywhere in your computer.

After running the code, you will see a Case 03-1.xml file. Put the xml and scn files into the same folder, and open the scn file using ImageScope software (only avilable in Windows OS). You should see something like the following image, with green detection results.

A Google Colab version of above can be found here, here are some bugs you might see here.

Benchmark Evaluation and Training

After installation, follow the instructions in DATA.md to setup the datasets. Then check GETTING_STARTED.md to reproduce the results in the paper. We provide scripts for all the experiments in the experiments folder.

Develop

If you are interested in training CircleNet in a new dataset, use CircleNet in a new task, or use a new network architecture for CircleNet please refer to DEVELOP.md. Also feel free to send us emails for discussions or suggestions.

License

CircleNet itself is released under the MIT License (refer to the LICENSE file for details). Parts of code and documentation are borrowed from CenterNet. We thank them for their elegant implementation.

Citation

If you find this project useful for your research, please use the following BibTeX entry.

@article{nguyen2021circle,
  title={Circle Representation for Medical Object Detection},
  author={Nguyen, Ethan H and Yang, Haichun and Deng, Ruining and Lu, Yuzhe and Zhu, Zheyu and Roland, Joseph T and Lu, Le and Landman, Bennett A and Fogo, Agnes B and Huo, Yuankai},
  journal={IEEE Transactions on Medical Imaging},
  year={2021},
  publisher={IEEE}
}

circlenet's People

Contributors

ethanhnguyen avatar hrlblab avatar jychen229 avatar lavywei666 avatar luyuzhe111 avatar

Stargazers

单旭铭 avatar  avatar  avatar Cristiano Arêdes avatar Chen Feng avatar Derkai avatar  avatar Bohdan Khytryk avatar VanCheng avatar fangyeqi avatar Serdar Yıldız avatar  avatar Ziwei Cui avatar  avatar xinbenlv avatar  avatar Srishti_M avatar Jumng Xiong avatar  avatar  avatar Vũ Hồng Quang avatar  avatar  avatar  avatar MTamam avatar admirestator avatar  avatar Pieter Blok avatar  avatar Yifan Wu avatar 木灬唯紫 avatar dundun avatar Inès  avatar 姚传洪 avatar  avatar  avatar Luckie Yunchao Hao avatar  avatar Physu avatar a5chin avatar Aristarkh Tikhonov avatar  avatar Henry Lee avatar Xion Lee avatar chenyunlong avatar Tianyue Xu avatar andrealoddo avatar Chen Yulin avatar Maxim Lyuzin avatar  avatar Yongcong Lin avatar  avatar jwmneu avatar QianChen avatar Yuchen Li avatar Niharika Vadlamudi avatar Tabris avatar Robin Heckenauer avatar Ellery Queen avatar  avatar DHW avatar RedShadow avatar  avatar Tik avatar pzSuen avatar  avatar QuanNM avatar PeterZ avatar  avatar  avatar Zifan Chen avatar

Watchers

 avatar  avatar

circlenet's Issues

About the dataset

How could i get the data of circle_center and circle_radius in a new dataset? Use the labelImg, it can only get the bbox data.

关于数据分类问题

该项目是否没有分类器,Heat Map最后的Conv(256,C),这里的C是细胞类别还是检测细胞列表?
我现在有一个四类细胞的免疫组化图像,是否无法使用?

Mismatch CUDA and PyTorch version

Hi,

I have follow steps you have mention in REANME file but causing issue -

The detected CUDA version (10.1) mismatches the version that was used to compile
PyTorch (11.3). Please make sure to use the same CUDA versions.

Also, _ext folder is not there in DCNv2.

Please respond back.

windows

Is it available on Windows?

Changing the input size

Hi, I was trying to train CircleNet on my custom dataset with the image resolution of 128. I managed to train the NN with this following command:
python main.py circledet --exp_id Apple2 --arch hourglass --batch_size 4 --master_batch 4 --lr 2.5e-4 --dataset apple --load_model /home/lmga-titanx/CircleNet/src/lib/models/ctdet_coco_hg.pth --input_res 128 --input_h 128 --input_w 128 --num_epochs 200

However, when I tried to run detection with my trained model using the following script:

TASK = 'circledet'
opt = opts().init('{} --dataset {} --load_model {} --arch {} --input_w {} --input_h {} --K {}'.format(TASK, 'coco', MODEL_PATH, 'hourglass', 128,128, 1).split(' '))

detector = detector_factory[opt.task](opt)
ret = detector.run(img)['results']

It returns an output that seemingly corresponds to the resolution of 512:

{1: array([[356.9697   , 344.58255  , 263.82367  ,   0.7991894,   0.       ]],
      dtype=float32)}

Can you please let me know what I may have done incorrectly? Thank you!

run demo.py encounter bug

Hello, how can I solve this bug?:)

command: python demo.py circledet --arch hourglass --demo ../docs/demo.png --load_model ../models/circledet_monuseg_hg.pth

debug:
Traceback (most recent call last):
File "E:/Pycharm_Project/Circle_detection/CircleNet-master/src/demo.py", line 11, in
from lib.detectors.detector_factory import detector_factory
File "E:\Pycharm_Project\Circle_detection\CircleNet-master\src\lib\detectors\detector_factory.py", line 5, in
from .exdet import ExdetDetector
File "E:\Pycharm_Project\Circle_detection\CircleNet-master\src\lib\detectors\exdet.py", line 15, in
from external.nms import soft_nms
ModuleNotFoundError: No module named 'external.nms'

environment: windows10+python3.8+torch1.10+cython0.29.24

关于result.json的问题

image
这个是进行评估之后输出的result.json文件,我想知道里面的score参数是什么意思,是概率或者置信度吗?
我是否需要设定一个阈值,只有分数大于该阈值才被认为是有效的,这个阈值应当设置为多少?
希望作者可以回答我的问题,万分感谢!!

circle loss

Hello, thank you for your code sharing. I encountered the following problems when training my dataset: first, the training loss seems to be the same as that of centernet, with no circle loss. Second, Why is cIOU not reflected in loss.
Hope to get your answer, thank you.

dataset

Hello, your work has helped me a lot!!!
May I ask if the glomerular detection dataset in this paper can be provided privately.
Thanks u~

转COCO数据集格式请教

The nuclei dateset with annotation file can be found here: https://github.com/hrlblab/CircleNet/blob/master/docs/DATA.md

1 在COCO格式的注释里,annotations中的segmentation字段多出了如下图所示的内容,其中width和height在image字段中已经出现了,如果直接用circlenet直接训练的话,这个必须加吗?
image

2 extreme_points代表什么意思?
3 MoNuSeg_extreme_train2021.json和MoNuSeg_train2021.json有什么区别?

Validation loss increases after a certain epoch.

Hi, I realized that my validation loss is increasing over time after a certain epoch while the training loss is converging. It seems to me that the network is overfitting. I have tried disabling random cropping, and introduced augmentations to mitigate this issue, however the problem seems to remain. I'm fairly new to neural network training, any suggestion is much appreciated!

Screenshot from 2022-11-17 11-39-49
Screenshot from 2022-11-17 11-40-02

Here is my opt file:

==> cudnn version: 7104
==> Cmd:
['main.py', 'circledet', '--exp_id', 'Apple5', '--arch', 'hourglass', '--batch_size', '4', '--master_batch', '4', '--lr', '2.5e-4', '--dataset', 'apple2', '--load_model', '/home/lmga-titanx/CircleNet/src/lib/models/ctdet_coco_hg.pth', '--num_epochs', '1000', '--input_w', '128', '--input_h', '128', '--K', '1', '--not_rand_crop', '--rotate', '1']
==> Opt:
  K: 1
  aggr_weight: 0.0
  agnostic_ex: False
  arch: hourglass
  aug_ddd: 0.5
  aug_rot: 0
  batch_size: 4
  cat_spec_wh: False
  center_thresh: 0.1
  chunk_sizes: [4]
  data_dir: /home/lmga-titanx/CircleNet/src/lib/../../data
  dataset: apple2
  debug: 0
  debug_dir: /home/lmga-titanx/CircleNet/src/lib/../../exp/circledet/Apple5/debug
  debugger_theme: white
  demo: 
  demo_dir: 
  dense_hp: False
  dense_wh: False
  dep_weight: 1
  dim_weight: 1
  down_ratio: 4
  eval_oracle_dep: False
  eval_oracle_hm: False
  eval_oracle_hmhp: False
  eval_oracle_hp_offset: False
  eval_oracle_kps: False
  eval_oracle_offset: False
  eval_oracle_wh: False
  exp_dir: /home/lmga-titanx/CircleNet/src/lib/../../exp/circledet
  exp_id: Apple5
  ez_guassian_radius: False
  filter_boarder: False
  fix_res: True
  flip: 0.5
  flip_test: False
  gpus: [0]
  gpus_str: 0
  head_conv: 64
  heads: {'hm': 1, 'cl': 1, 'reg': 2}
  hide_data_time: False
  hm_hp: True
  hm_hp_weight: 1
  hm_weight: 1
  hp_weight: 1
  input_h: 128
  input_res: 128
  input_w: 128
  keep_res: False
  kitti_split: 3dop
  load_model: /home/lmga-titanx/CircleNet/src/lib/models/ctdet_coco_hg.pth
  lr: 0.00025
  lr_step: [90, 120]
  lv: 2
  mask_focal_loss: False
  master_batch_size: 4
  mean: [[[0.40789655 0.44719303 0.47026116]]]
  metric: loss
  mse_loss: False
  nms: False
  no_color_aug: False
  norm_wh: False
  not_cuda_benchmark: False
  not_hm_hp: False
  not_prefetch_test: False
  not_rand_crop: True
  not_reg_bbox: False
  not_reg_hp_offset: False
  not_reg_offset: False
  num_classes: 1
  num_epochs: 1000
  num_iters: -1
  num_stacks: 2
  num_workers: 4
  off_weight: 1
  ontestdata: False
  output_h: 32
  output_res: 32
  output_w: 32
  pad: 127
  peak_thresh: 0.2
  print_iter: 0
  rect_mask: False
  reg_bbox: True
  reg_hp_offset: True
  reg_loss: l1
  reg_offset: True
  resume: False
  root_dir: /home/lmga-titanx/CircleNet/src/lib/../..
  rot_weight: 1
  rotate: 1.0
  rotate_reproduce: 0
  save_all: False
  save_dir: /home/lmga-titanx/CircleNet/src/lib/../../exp/circledet/Apple5
  scale: 0.4
  scores_thresh: 0.1
  seed: 317
  shift: 0.1
  std: [[[0.2886383  0.27408165 0.27809834]]]
  task: circledet
  test: False
  test_scales: [1.0]
  test_val_name: 
  trainval: False
  val_intervals: 1
  vis_thresh: 0.2
  wh_weight: 0.1```


Running demo.ipynb

Hi, I have followed all your steps on demo.ipynb, but when I run the final command, it show this kind of error. How do I solve it. And is there new version for CUDA 10.1 10.2 or 9.1 to run this code.
777a014e550553dd6a27cc6ed9cdb99

Hi,I want to use other dataset instead of MoNuSeg2021,but the circle_center and circle_radius fields are not in the json file. How do I get them

for example

Our dataset no circle_center and circle_radius fields in json file {"id": 1326, "image_id": 12, "category_id": 1, "iscrowd": 0, "area": 693, "bbox": [303.0, 270.0, 38.0, 30.0], "segmentation": [[337.0, 299.5, 315.0, 297.5, 305.0, 293.5, 302.5, 284.0, 307.5, 271.0, 314.0, 270.5, 324.0, 282.5, 331.0, 284.5, 333.0, 279.5, 340.5, 293.0, 337.0, 299.5]], "width": 500, "height": 500}

The monuseg2021 dataset you provided have circle_center and circle_radius fields {"id": 65786, "image_id": 309, "category_id": 1, "iscrowd": 0, "area": 191, "bbox": [393.0, 305.0, 19.0, 14.0], "segmentation": [[410.0, 318.5, 395.0, 314.5, 392.5, 309.0, 398.0, 304.5, 405.0, 305.5, 411.5, 312.0, 410.0, 318.5]], "width": 512, "height": 512, "circle_center": [401.583984375, 313.134765625], "circle_radius": 9.980734386753602, "extreme_points": [[398, 304], [392, 309], [410, 318], [411, 312]]}

No such file or directory when running demo

I am running into an error when I run this command in the circle_net_demo.ipynb file

!python run_detection_for_scn.py circledet --arch dla_34 --demo "/content/drive/My Drive/CircleNet/example_case/Case 03-1.scn" --load_model /content/drive/My\ Drive/CircleNet/trained_model/model_10.pth --filter_boarder --demo_dir "/content/drive/My Drive/CircleNet/demo_dir/test_demo"

Here is the traceback

Traceback (most recent call last):
  File "run_detection_for_scn.py", line 470, in <module>
    demo(opt)
  File "run_detection_for_scn.py", line 461, in demo
    run_one_scn(demo_scn, demo_dir, opt)
  File "run_detection_for_scn.py", line 241, in run_one_scn
    detector = Detector(opt)
  File "/content/drive/MyDrive/CircleNet/src/lib/detectors/circledet.py", line 23, in __init__
    super(CircledetDetector, self).__init__(opt)
  File "/content/drive/MyDrive/CircleNet/src/lib/detectors/base_detector.py", line 25, in __init__
    self.model = load_model(self.model, opt.load_model)
  File "/content/drive/MyDrive/CircleNet/src/lib/models/model.py", line 34, in load_model
    checkpoint = torch.load(model_path, map_location=lambda storage, loc: storage)
  File "/usr/local/lib/python3.7/site-packages/torch/serialization.py", line 356, in load
    f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/content/drive/My Drive/CircleNet/trained_model/model_10.pth'

Dataset Annotation Format

If the dataset is inconvenient to publish, I hope to give pointers on the annotation of the dataset. It is best to provide an annotation file, thank you!!!

自己项目的数据集制作

怎么制作自己的数据集在这个网络模型上部署呀?用什么标注软件,标注什么类型?有人跑过可以详细说一下吗?我大概是要对一个类圆的果实进行圆形的分割,请问该怎么制作数据集?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.