abe404 / rootpainter3d Goto Github PK
View Code? Open in Web Editor NEWRootPainter3D: Interactive-machine-learning enables rapid and accurate contouring for radiotherapy
License: GNU General Public License v3.0
RootPainter3D: Interactive-machine-learning enables rapid and accurate contouring for radiotherapy
License: GNU General Public License v3.0
Right now the contrast settings assume that the images contain integer HU values, but some datasets for example the the hepatic vessels data from http://medicaldecathlon.com/ includes floating point values.
Looking at the data it seems it's safe to cast it to int. I will do that for now until we encounter a dataset that actually needs floating point values.
This is functionality that needs copying across from the 2D version.
The CI build process does not yet work.
the plan is that I will get it working for ubuntu 20.04 (keep altering the main.spec file until it builds) and then Andre will get it working with ubuntu 18
Trainer fails because of dtype mismatch in call to cnn function.
Log output below. Input data is uint16.
found instruction segment_-8639686351424129568
execute_instruction segment
segmented input shape (235, 235, 235)
segment image, input shape = (235, 235, 235)
Exception parsing instruction Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same Traceback (most recent call last):
File "/trainer/trainer.py", line 154, in execute_instruction
getattr(self, name)(config)
File "/trainer/trainer.py", line 509, in segment
overwrite=overwrite)
File "/trainer/trainer.py", line 592, in segment_file
out_d, classes, bounded)
File "/trainer/model_utils.py", line 147, in ensemble_segment_3d
pred_maps = segment_3d(cnn, image, batch_size, in_patch_shape, out_patch_shape)
File "/trainer/model_utils.py", line 244, in segment_3d
outputs = cnn(tiles_for_gpu).detach().cpu()
File "/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 166, in forward
return self.module(*inputs[0], **kwargs[0])
File "/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/trainer/unet3d.py", line 129, in forward
out1 = self.conv_in(x)
File "/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/python3.6/site-packages/torch/nn/modules/container.py", line 141, in forward
input = module(input)
File "/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/python3.6/site-packages/torch/nn/modules/conv.py", line 590, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/python3.6/site-packages/torch/nn/modules/conv.py", line 586, in _conv_forward
input, weight, bias, self.stride, self.padding, self.dilation, self.groups
RuntimeError: Input type (torch.cuda.DoubleTensor) and weight type (torch.cuda.FloatTensor) should be the same
Segmentations are generated during the interactive training process i.e for each image the user annotates.
In some cases - for example semi-automatic image segmentation, it can be useful to work with these segmentations.
Right now the segmentations are available on disk and their corresponding annotations are also available on disk but it is not trivial for users to assign their corrections i.e the annotation file to the corresponding segmentation file in such a way that they get a corrected segmentation, which is what they would likely want in a semi-automatic image segmentation workflow.
There should be an option in the extras menu in the client software to allow the user to perform this function.
The form should include segmentation folder directory field, annotation folder field , output folder field and submit button. The user should be given feedback whilst the corrections (annotations) are being assigned and the corrected segmentations are being generated.
RootPainter3D/trainer/model_utils.py
Line 237 in b83ae7f
As discussed here: #8 (comment)
We should make a CI for building a binary installer for macOS.
Input images with floats in [0,1] are displayed as all black. I suspect this related to how contrast is applied, since the contrast limits for these images are set to [0,0] and cannot be changed.
There is not currently a clear way for two users to work on the same project. A problem is that two users could open the same file and then over-write each others annotations when saving.
A lock file system could solve this.
A possibily limitation / issue that may arrise is that if the application crashed for some reason or was quit unexpectedly, their may not be a clean way to delete the lock file. I will consider this issue later but some possible solutions include:
I get this warning when starting the trainer:
RootPainter3D/trainer/env/lib/python3.10/site-packages/paramiko/transport.py:236: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish,
The patch size is currently found automatically by testing various input sizes and seeing which ones cause a memory error.
This seems to be time consuming and slows the development cycle.
Suggested steps:
See #9 (comment)
Andre wants to test the software with .nii files. I think that it's probably easy to add support for this considering we already support compressed nifty (nii.gz).
RootPainter server eats up all the CPU resources, which may lead to problems running the server and client on the same machine. It also makes it hard to run other things in parallel.
"Redo" and "Save annotation" both have Ctrl + Shift + Z as shortcut.
Using the shortcut does nothing and gives the following output
QAction::event: Ambiguous shortcut overload: Ctrl+Shift+Z
Maybe make "Save annotation" Ctrl + Shift + S?
Right now running the client on Mac with an M1/M2 chip requires using Rosetta. It would be better not to have to do this, but the client must be updated to PyQt6.
To run with Rosetta
https://dev.to/courier/tips-and-tricks-to-setup-your-apple-m1-for-development-547g
I know from the 2D version that this upgrade can be a hassle, so I will try to address it as an isolated issue to avoid blocking other tasks.
The decision of whether to add images to the validation or training set is made when an annotation is saved. This however happens for each class and so images may be in the training set for one class and validation set for another class. The model could therefore potentially overfit to the validation set.
A simple fix could be to check if the file is already in the training or validation set for another class before choosing the set.
Just tried to install the generated binary release from the latest artefact and it crashes.
I believe we should thoroughly test if the software can be run as a binary.
Ideally, the user should not need to run anything from the terminal, that somewhat defeats the purpose of having a user friendly tool, if the end user is forced to open a terminal to use the tool.
Might be that we have to run something from the terminal, for instance to setup training on the server, but ideally we should not need to.
Perhaps we could schedule a meeting discussing this further and getting RootPainter3D running on my machine?
When all this is running it would also be ideal to make a simple tutorial video demonstrating how to get started and using the software :) We could make this as an Issue (or feature request rather) when that is relevant.
The "Keyboard shortcuts" menu item does not display anything. It prints the following error
Traceback (most recent call last):
File "main.py", line 80, in
init_root_painter()
File "main.py", line 66, in init_root_painter
exit_code = app_context.app.exec_()
File "/RootPainter3D/painter/src/main/python/root_painter.py", line 422, in show_shortcut_window
self.shortcut_window = ShortcutWindow()
File "/RootPainter3D/painter/src/main/python/about.py", line 798, in init
self.initUI()
File "/RootPainter3D/painter/src/main/python/about.py", line 859, in initUI
table.horizontalHeader().sectionSizeFromContents(True)
RuntimeError: no access to protected functions or signals for objects not created from Python
Linux Debian 10
Python 3.7.3
PyQt 5.14.2
Can be solved by removing
Right now the system crashes if an image is smaller than the network patch size. The network patch size should just be made smaller automatically (or we could automatically pad the image, but I think this makes less sense).
There are two views, the main and "sagittal". Would be nice if the third view could also be seen. Maybe by toggling what is shown in the sagittal window.
I am annotating some structures that are much easier to differentiate in the third view.
As discussed in PR #7, we thought it was a good idea to setup an workflow for building an installer of the software.
However, this requires us to restructure the code such that the program itself can be installed and run as an executable. The core code could likely be merged into the executable, but there is likely some other stuff that should live outside the executable. Not sure.
I can make an initial attempt and submit a PR when I have something somewhat working.
Training kills the server on some of my projects, because images often have only background annotated. It might be worth considering if this should be treated as severely as a load failure.
See:
RootPainter3D/painter/src/im_utils.py
Line 56 in 2d3155d
Consider instead working with the data in the original format.
I had created a project using the hepatic vessels data from http://medicaldecathlon.com/
The dataset had files in it such as ._hepaticvessel_333.nii.gz
As far as I know, these are metadata files for OSX. I think it is safe to assume if a file starts with . then it should not be read by RootPainter3D.
Alter the create project functionality to ignore such files as I do not want them to end up in the created project file.
Network input (and corresponding output) patch size does not currently adapt to the available GPU memory.
It's possible for users to modify the patch size to suit their GPU memory requirements but it's annoying and error prone to do so.
Users shouldn't shouldn’t have to think about this. Patch size adaptation to available GPU memory should happen automatically
A problem has been reported with segment folder not working in the dev_mc branch.
Looks like the following is causing the issue:
segment_folder.py", line 87, in segment_folder
assert seg_classes[0][0] == 'Background'
AssertionError
I will work on first reproducing this issue and then working on a solution (it's probably quite simple to fix).
I believe I have tracked the error down to the line 336 in im_utils.py.
"all_annot_fnames = set(cur_annot_fnames + prev_annot_fnames)"
Reading line 338 makes me assume that the variable "all_annot_fnames" is supposed to match index-wise with all_dirs, as this is what cur_annot_fnames does. However, cur_annot_fnames has repeated elements, because it contains filenames across multiple directories corresponding to each of the multiple classes. Moreover the set datastructure does not preserve order, so any correspondence between cur_annot_fnames and all_dirs is lost. The end result is that sometimes filenames are assumed to be in directories they are not actually present in and it therefore triggers the retry error. If my interpretation is correct, I also assume that this means a much smaller validation set is actually used than what is present, as filenames are unique across classes.
In the 3D multiclass version, when changing the current class to be annotated a start training instruction is sent. I think it would be better if a start training instruction is only be sent when explicitly asked for.
See #9 (comment)
I should make a guide with rules that is visible from the main readme.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.