Code and data for the "annotation" component of the IPCAI 2020 paper: "Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration." https://arxiv.org/abs/1911.07042 or https://doi.org/10.1007/s11548-020-02162-7
License: Other
Hi Robert, I am curious how the locations for the 3D vol-landmarks the 2D gt-landmarks are generated. Are these values in the CTs / fluoroscopy images that we can identify by pixel intensity?
Hi Robert, sorry to bother again but I am having doubts about my understanding of some of the parameters.
My rough understanding is that the extrinsic matrix, in general, gives the position/orientation of the camera capturing an image. That is why it would make sense in my head to have one such matrix for each projection.
In your dataset there seems to be a global extrinsic matrix to shift to camera space, and 3 different matrices to shift into volume space (whether pelvis or either femur). Could you please give a brief explanation as to how you obtained these matrices, what the global one represents, and where the shift takes us (is it the center of the volume? one of the corners?). Trying to generalise to some other data has proven a little difficult for me..
Thank you.
I am trying to recapitulate the preprocessing steps you mention in the README:
The preprocessing steps consist of cropping 50 pixels from each border to remove collimator effects, log transforming the projection data, rotation to ensure patient is "up," and downsampling.
I can't get the log transform step quite right. Here's what I've tried so far:
import h5py
import matplotlib.pyplot as plt
import torch
from torchvision.transforms.functional import center_crop, resize
fullres = h5py.File("ipcai_2020_full_res_data.h5", "r")
downsam = h5py.File("data/ipcai_2020_ds_8x.h5", "r")
# Full-res image
idx = 20
img = fullres[f"17-1882/projections/{idx:03d}/image/pixels"][:]
# Processing the full-res image
img_ = torch.from_numpy(img)
img_ = center_crop(img_, [1536-50, 1536-50]) # Crop 50 pixels from each border to remove collimator effects
img_ = -img_.log() # Log-transform the projection data
img_ = resize(img_.unsqueeze(0).unsqueeze(0), [180, 180], antialias=True).squeeze() # Downsample
# Get the preprocessed image
dsimg = downsam["01/projs"][:]
plt.figure(constrained_layout=True, dpi=300, figsize=(10, 2.5))
plt.subplot(131)
plt.imshow(img, cmap="gray")
plt.colorbar()
plt.title("Full-res image")
plt.subplot(132)
plt.imshow(img_, cmap="gray")
plt.colorbar()
plt.title("My processed image")
plt.subplot(133)
plt.imshow(dsimg[idx], cmap="gray")
plt.colorbar()
plt.title("Your preprocessed image")
plt.show()which produces
I'm guessing you've implemented the log transform differently?
Thank you for taking the time to organize and document this dataset so thoroughly! It's a joy to work with.
Hi,
Thanks for providing such a great code！
I am passing the following command
python -u train.py ipcai_2020_ds_8x.h5 --train-pats 2,3,4,5,6 --num-classes 7 --init-lr 0.1 --momentum 0.9 --unet-batch-norm --unet- no-max-pool --unet-img-dim 192 --unet-num-lvls 6 --batch-size 5 --max-num-epochs 500 --unet-init-feats-exp 5 --wgt-decay 0.0001 --data-aug --unet-padding --nesterov --checkpoint-net yy_check_net.pt --checkpoint-freq 100 --use-lands --best-net yy_best_net.pt --lr-sched plateau --train -valid-split 0.85 --train-loss-txt yy_train_loss.txt --valid-loss-txt yy_valid_loss.txt --heat-coeff 0.5
When running the training code, it throws an error
num. lands read from file: 14
initializing training dataset/dataloader
split dataset into train/validation
randomly splitting all complete tensors into training/validation...
Length of training dataset: 217
Length of validation dataset: 38
creating network
moving network to device...
creating loss function
Dice + Heatmap Loss...
creating SGD optimizer and LR scheduler
Start Training...
Epoch: 000
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py:1041: UserWarning: Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead
"Argument resample is deprecated and will be removed since v0.10.0. Please, use interpolation instead"
Traceback (most recent call last):
File "train.py", line 399, in <module>
for (i, data) in enumerate(train_dl, 0):
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
data = self._next_data()
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
TypeError: Caught TypeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/***/Downloads/DeepFluoroLabeling-IPCAI2020-2020.03.17/train_test_code/dataset.py", line 230, in __getitem__
A_inv = TF._get_inverse_affine_matrix(center_of_rot, rot_ang, (trans_x, trans_y), scale_factor, shear)
File "/home/***/anaconda3/envs/re/lib/python3.7/site-packages/torchvision/transforms/functional.py", line 903, in _get_inverse_affine_matrix
sx, sy = [math.radians(s) for s in shear]
TypeError: 'float' object is not iterableCan you help me？
Thanks in advance,
Crack
Hi Robert, thank you for this code and especially dataset, it's incredibly thorough and very useful!
However I have one question, how do you generate the "intrinsic" matrix found in the "proj-params" section of the dataset? From what I understand, you can obtain the extrinsic matrix using information provided with most CT files, however I have not found how to extract or simulate the intrinsic matrix using a different set of data. Is that directly related to the specific C-arm or capture device? Would one be able to get this information just from a DICOM series? I understand this may not be your expertise but if you had an insight on that it'd be greatly appreciated.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.