sunwj / car Goto Github PK
View Code? Open in Web Editor NEWContent adaptive resampler for image downscaling
License: GNU General Public License v3.0
Content adaptive resampler for image downscaling
License: GNU General Public License v3.0
Hello,
I am trying to build and run the CAR-pytorch project on my Arch Linux machine, but I am encountering compilation errors during the installation of the adaptive_gridsampler.
Here are the steps I have followed:
adaptive_gridsampler
directory and attempted to run python3 setup.py build_ext --inplace
.Initially, I encountered an error related to an incompatible version of PyTorch. The project requires PyTorch 1.3.1, but it is not available through pip or conda for Python 3.8. I tried installing PyTorch 1.4.0, but the error persisted. I then created a conda environment with Python 3.7 and installed PyTorch 1.3.1, but the compilation error still occurred.
Here is the error message I received:
/home/myuser/.conda/envs/car-env/lib/python3.7/site-packages/torch/include/pybind11/cast.h: In function ‘typename pybind11::detail::type_caster<typename pybind11::detail::intrinsic_type<T>::type>::cast_op_type<T> pybind11::detail::cast_op(make_caster<T>&)’:
/home/myuser/.conda/envs/car-env/lib/python3.7/site-packages/torch/include/pybind11/cast.h:914:120: error: expected template-name before ‘<’ token
914 | return caster.operator typename make_caster<T>::template cast_op_type<T>();
| ^
/home/myuser/.conda/envs/car-env/lib/python3.7/site-packages/torch/include/pybind11/cast.h:914:120: error: expected identifier before ‘<’ token
/home/myuser/.conda/envs/car-env/lib/python3.7/site-packages/torch/include/pybind11/cast.h:914:123: error: expected primary-expression before ‘>’ token
914 | return caster.operator typename make_caster<T>::template cast_op_type<T>();
| ^
/home/myuser/.conda/envs/car-env/lib/python3.7/site-packages/torch/include/pybind11/cast.h:914:126: error: expected primary-expression before ‘)’ token
914 | return caster.operator typename make_caster<T>::template cast_op_type<T>();
| ^
error: command '/opt/cuda/bin/nvcc' failed with exit code 1
I would appreciate any guidance or assistance in resolving this issue. If there are any updates or additional steps required for compatibility with the latest versions of the dependencies, please let me know.
Thank you in advance for your help.
Can you please let me know how to get/run the training code?
Hey! Thank you for sharing the code.
However, when I was trying to compile the code for resampler, I faced the following error. Can you please look into it? Thanks!
(ocr) siddhant.b@gnode55:~/CAR/adaptive_gridsampler$ CUDAHOSTCXX=/usr/bin/gcc-5 python3 setup.py build_ext --inplace
running build_ext
building 'adaptive_gridsampler_cuda' extension
creating build
creating build/temp.linux-x86_64-3.7
gcc -pthread -B /home/siddhant.b/miniconda3/envs/ocr/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/siddha
nt.b/miniconda3/envs/ocr/lib/python3.7/site-packages/torch/include -I/home/siddhant.b/miniconda3/envs/ocr/lib/python3.7/site-packages/torch/include/torch/csrc/api/include
-I/home/siddhant.b/miniconda3/envs/ocr/lib/python3.7/site-packages/torch/include/TH -I/home/siddhant.b/miniconda3/envs/ocr/lib/python3.7/site-packages/torch/include/THC
-I/usr/local/cuda-9.0/include -I/home/siddhant.b/miniconda3/envs/ocr/include/python3.7m -c adaptive_gridsampler_cuda.cpp -o build/temp.linux-x86_64-3.7/adaptive_gridsampl
er_cuda.o -std=c++11 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=adaptive_gridsampler_cuda -D_GLIBCXX_USE_CXX11_ABI=0
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/usr/local/cuda-9.0/bin/nvcc -I/home/siddhant.b/miniconda3/envs/ocr/lib/python3.7/site-packages/torch/include -I/home/siddhant.b/miniconda3/envs/ocr/lib/python3.7/site-pa
ckages/torch/include/torch/csrc/api/include -I/home/siddhant.b/miniconda3/envs/ocr/lib/python3.7/site-packages/torch/include/TH -I/home/siddhant.b/miniconda3/envs/ocr/lib
/python3.7/site-packages/torch/include/THC -I/usr/local/cuda-9.0/include -I/home/siddhant.b/miniconda3/envs/ocr/include/python3.7m -c adaptive_gridsampler_kernel.cu -o bu
ild/temp.linux-x86_64-3.7/adaptive_gridsampler_kernel.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr
--compiler-options '-fPIC' -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_70,code=comp
ute_70 -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=adaptive_gridsampler_cuda -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
adaptive_gridsampler_kernel.cu(10): error: explicit type is missing ("int" assumed)
adaptive_gridsampler_kernel.cu(10): error: qualified name is not allowed
adaptive_gridsampler_kernel.cu(10): error: expected a ")"
adaptive_gridsampler_kernel.cu(17): warning: parsing restarts here after previous syntax error
adaptive_gridsampler_kernel.cu(72): error: class "at::Tensor" has no member "packed_accessor32"
adaptive_gridsampler_kernel.cu(72): error: type name is not allowed
adaptive_gridsampler_kernel.cu(72): error: argument list for class template "at::RestrictPtrTraits" is missing
adaptive_gridsampler_kernel.cu(72): error: expected an expression
adaptive_gridsampler_kernel.cu(72): error: class "at::Tensor" has no member "packed_accessor32"
adaptive_gridsampler_kernel.cu(72): error: type name is not allowed
adaptive_gridsampler_kernel.cu(72): error: argument list for class template "at::RestrictPtrTraits" is missing
adaptive_gridsampler_kernel.cu(72): error: expected an expression
adaptive_gridsampler_kernel.cu(73): error: class "at::Tensor" has no member "packed_accessor32"
adaptive_gridsampler_kernel.cu(73): error: type name is not allowed
adaptive_gridsampler_kernel.cu(73): error: argument list for class template "at::RestrictPtrTraits" is missing
adaptive_gridsampler_kernel.cu(73): error: expected an expression
adaptive_gridsampler_kernel.cu(72): error: class "at::Tensor" has no member "packed_accessor32"
adaptive_gridsampler_kernel.cu(72): error: type name is not allowed
adaptive_gridsampler_kernel.cu(72): error: argument list for class template "at::RestrictPtrTraits" is missing
adaptive_gridsampler_kernel.cu(72): error: expected an expression
adaptive_gridsampler_kernel.cu(73): error: class "at::Tensor" has no member "packed_accessor32"
adaptive_gridsampler_kernel.cu(73): error: type name is not allowed
adaptive_gridsampler_kernel.cu(73): error: argument list for class template "at::RestrictPtrTraits" is missing
adaptive_gridsampler_kernel.cu(73): error: expected an expression
adaptive_gridsampler_kernel.cu(73): error: class "at::Tensor" has no member "packed_accessor32"
adaptive_gridsampler_kernel.cu(73): error: type name is not allowed
adaptive_gridsampler_kernel.cu(73): error: argument list for class template "at::RestrictPtrTraits" is missing
adaptive_gridsampler_kernel.cu(73): error: expected an expression
adaptive_gridsampler_kernel.cu(74): error: class "at::Tensor" has no member "packed_accessor32"
adaptive_gridsampler_kernel.cu(74): error: type name is not allowed
adaptive_gridsampler_kernel.cu(74): error: argument list for class template "at::RestrictPtrTraits" is missing
adaptive_gridsampler_kernel.cu(74): error: expected an expression
adaptive_gridsampler_kernel.cu(71): error: no instance of function template "kernel_adaptive_gridsampler_update_output" matches the argument list
argument types are: (<error-type>, int, <error-type>, <error-type>, int, <error-type>, <error-type>, int, <error-type>, <error-type>, int, <error-type>, const int, const int, <error-type>, int, <error-type>, int64_t)
24 errors detected in the compilation of "/tmp/tmpxft_00004ba5_00000000-8_adaptive_gridsampler_kernel.compute_70.cpp1.ii".
error: command '/usr/local/cuda-9.0/bin/nvcc' failed with exit status 1
(ocr) siddhant.b@gnode55:~/CAR/adaptive_gridsampler$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Sep__1_21:08:03_CDT_2017
Cuda compilation tools, release 9.0, V9.0.176
(ocr) siddhant.b@gnode55:~/CAR/adaptive_gridsampler$ gcc --version
gcc (Ubuntu 5.5.0-12ubuntu1~16.04) 5.5.0 20171010
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
(ocr) siddhant.b@gnode55:~/CAR/adaptive_gridsampler$ g++ --version
g++ (Ubuntu 5.5.0-12ubuntu1~16.04) 5.5.0 20171010
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
I even tried suggestions from here, but it did not help.
Thank you for posting your code. I noticed the run.py command only works if you already have the original, high resolution image. Is it possible to infer with a smaller source image and produce a high res output?
I managed to get the provided models running, however when I run it the downscaled pictures are just black and therefore so are the upscaled pictures. The pictures have the correct amount of pixels and the ratio is correct. Does anyone know what might cause this issue?
Thank you for your work. However, I was wondering if I could run/compile this without a GPU? Could you please mention the changes to be made in the code in order for this to work?
I'm trying to make use of the trained model in Colab, but i get the following error during evaluation :
CUDA error at adaptive_gridsampler_kernel.cu:76 code=209(cudaErrorNoKernelImageForDevice) "cudaGetLastError()"
Colab Specs:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
physical_device_desc: "device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7"
Please let me know the exact specs required to run the trained model
Hi,
Thank you for posting your code, downscaling and upscaling result images are really neat!
However, the gridsampler CUDA extension lacks a backward method. Could you please release it?
Dear author:
I want to train this model on different datasets and compare the results. Unfortunately, there is no code for training. Can you please add training codes, too?
thanks a lot
Hi,
I am having trouble compiling the adaptive_gridsampler. The error message is very long and goes past the limit saved by my terminal window, but seems to repeat itself. Here is the error:
class ExpandingArray {
^~~~~~~~~~~~~~
In file included from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/core/ScalarType.h:6:0,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/core/Scalar.h:9,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/core/TensorBody.h:7,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/Tensor.h:11,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/Context.h:4,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/ATen.h:5,
from adaptive_gridsampler_cuda.cpp:1:
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::ExpandingArray<2>]’:
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/options/pooling.h:428:8: required from ‘void torch::nn::Cloneable<Derived>::clone_(torch::nn::Module&, const c10::optional<c10::Device>&) [with Derived = torch::nn::FractionalMaxPool2dImpl]’
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:48:17: required from here
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::ExpandingArray<2>’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
In file included from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/options/conv.h:6:0,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/conv.h:3,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional.h:4,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:4,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/extension.h:4,
from adaptive_gridsampler_cuda.cpp:2:
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/expanding_array.h:23:7: note: in call to ‘torch::ExpandingArray<2>& torch::ExpandingArray<2>::operator=(const torch::ExpandingArray<2>&)’
class ExpandingArray {
^~~~~~~~~~~~~~
In file included from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/core/ScalarType.h:6:0,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/core/Scalar.h:9,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/core/TensorBody.h:7,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/Tensor.h:11,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/Context.h:4,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/ATen/ATen.h:5,
from adaptive_gridsampler_cuda.cpp:1:
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/util/Optional.h: In instantiation of ‘c10::optional<T>& c10::optional<T>::operator=(c10::optional<T>&&) [with T = torch::ExpandingArray<2, double>]’:
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/options/pooling.h:428:8: required from ‘void torch::nn::Cloneable<Derived>::clone_(torch::nn::Module&, const c10::optional<c10::Device>&) [with Derived = torch::nn::FractionalMaxPool2dImpl]’
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/optim/sgd.h:48:17: required from here
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/c10/util/Optional.h:396:23: error: passing ‘const torch::ExpandingArray<2, double>’ as ‘this’ argument discards qualifiers [-fpermissive]
contained_val() = std::move(*rhs);
~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~
In file included from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/options/conv.h:6:0,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/conv.h:3,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional.h:4,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/nn.h:4,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/all.h:7,
from /home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/extension.h:4,
from adaptive_gridsampler_cuda.cpp:2:
/home/jamespb/Projects/CAR/car_venv/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/expanding_array.h:23:7: note: in call to ‘torch::ExpandingArray<2, double>& torch::ExpandingArray<2, double>::operator=(const torch::ExpandingArray<2, double>&)’
class ExpandingArray {
^~~~~~~~~~~~~~
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Additional info:
OS: Ubuntu 18.04
gcc: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
python packages:
Package Version
--------------- -----------------
cycler 0.10.0
decorator 4.4.2
future 0.18.2
imagecodecs 2020.2.18
imageio 2.8.0
kiwisolver 1.2.0
matplotlib 3.2.1
networkx 2.4
numpy 1.18.4
Pillow 7.1.2
pip 20.1.1
pkg-resources 0.0.0
pyparsing 2.4.7
python-dateutil 2.8.1
PyWavelets 1.1.1
scikit-image 0.17.2
scipy 1.4.1
setuptools 39.0.1
six 1.14.0
tifffile 2020.5.11
torch 1.5.0
torch-nightly 1.2.0.dev20190805
tqdm 4.46.0
wheel 0.34.2
Thanks, any help is much appreciated
Hi, nice to meet you. I am reading your paper and in your paper you mentioned that you will release the code here. However, I cannot find the training code. Where can I find your training code?
I can not download....Thanks soooo much.
I get the subject error and then the out image is just black. What can be the issue here?
Thank you for your research and the code.
Is it possible to share the number of parameters and MACs of this model?
Plz
Can I get your train code or backward code of GridSampler? Thx
Hi @sunwj, thanks for open sourcing the code!
I tried to run it and have successfully setup the dependencies. However, when running run.py, it gives this error message
CUDA error at adaptive_gridsampler_kernel.cu:76 code=48(cudaErrorNoKernelImageForDevice) "cudaGetLastError()"
And the output is a black image and with (Mean PSNR: 6.53; Mean SSIM: 0.1514). Wondering if there's a CPU version just for model inference? That would be less likely to encounter this kind of environmental error!
Hi @sunwj . Thank you for providing this repository. I was very happy to try it on high-resolution images ( resolution > = 16 Megapixels) and noticed the GPU was running out of memory. I went back and tried it with the benchmark datasets and noticed that images with 0.6 megapixels (1024 x 640 pixels) would consume 5 GB VRAM. What could we do to reduce the VRAM requirements? Is this relationship linear with size of the image?
Hi, first really thank you for your work!
Now I'm reproducing your work.
I have two question about clipping value and soft round function.
When I set learning rate to 10e-4, graident values are diverged. So I use gradient clipping and set the value to 0.01.
I wonder if it is reasonable value.
What value did you set to tuning parameter alpha?
thank you in advance!
hi
i downloaded zip files you mentioned is pretrained weights.
as i check, there is only 2X and 4X downsampling.
please upload file for Super-resolution Too
thanks
can someone post a workable version of pytorch, cuda, c++ that can compile the code?
For the inference operation, the following operation was carried out, but an error appears.
git clone https://github.com/sunwj/CAR.git
cd CAR
cd adaptive_gridsampler
python3 setup.py build_ext --inplace
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.