haojhuang / equivariant-transporter-net Goto Github PK

Hi~ How to do batch training?

When I run:
python train.py --equ --lite --angle_lite --task=block-insertion --n_demos 1 --interval 200 --n_step 1000 --init
I met a problem below, How can I solve this ?

(etn) randy@randy-Precision-7920-Tower:/media/randy/299D817A2D97AD94/FTY/Equivariant-Transporter-Net-main$ python train.py --equ --lite --angle_lite --task=block-insertion --n_demos 1 --interval 200 --n_step 1000 --init
2022-03-25 12:56:58.160504: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
pybullet build time: Dec 1 2021 18:33:04
Traceback (most recent call last):
File "train.py", line 12, in
from networks.non_equi_transporter import TransporterAgent as non_equi_agent
File "/media/randy/299D817A2D97AD94/FTY/Equivariant-Transporter-Net-main/networks/non_equi_transporter.py", line 12, in
from non_equi_transport import Transport
File "/media/randy/299D817A2D97AD94/FTY/Equivariant-Transporter-Net-main/networks/non_equi_transport.py", line 11, in
import kornia as K
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/kornia/init.py", line 10, in
from . import (
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/kornia/augmentation/init.py", line 54, in
from kornia.augmentation.container import AugmentationSequential, ImageSequential, PatchSequential, VideoSequential
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/kornia/augmentation/container/init.py", line 1, in
from kornia.augmentation.container.augment import AugmentationSequential
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/kornia/augmentation/container/augment.py", line 20, in
from kornia.geometry.boxes import Boxes
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/kornia/geometry/boxes.py", line 464, in
@torch.jit.script
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/init.py", line 818, in script
ast = get_jit_class_def(obj, name)
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 143, in get_jit_class_def
self_name=self_name) for method in methods]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 143, in
self_name=self_name) for method in methods]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 158, in get_jit_def
return build_def(ctx, py_ast.body[0], type_line, self_name)
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 198, in build_def
build_stmts(ctx, body))
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 122, in build_stmts
stmts = [build_stmt(ctx, s) for s in stmts]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 122, in
stmts = [build_stmt(ctx, s) for s in stmts]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 174, in call
return method(ctx, node)
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 332, in build_If
build_stmts(ctx, stmt.body),
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 122, in build_stmts
stmts = [build_stmt(ctx, s) for s in stmts]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 122, in
stmts = [build_stmt(ctx, s) for s in stmts]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 174, in call
return method(ctx, node)
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 288, in build_Raise
expr = build_expr(ctx, stmt.exc)
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 174, in call
return method(ctx, node)
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 405, in build_Call
args = [build_expr(ctx, py_arg) for py_arg in expr.args]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 405, in
args = [build_expr(ctx, py_arg) for py_arg in expr.args]
File "/home/randy/anaconda3/envs/etn/lib/python3.7/site-packages/torch/jit/frontend.py", line 173, in call
raise UnsupportedNodeError(ctx, node)
torch.jit.frontend.UnsupportedNodeError: JoinedStr aren't supported
def init(
self, boxes: torch.Tensor, raise_if_not_floating_point: bool = True,
mode: str = "xyzxyz_plus"
) -> None:
if not isinstance(boxes, torch.Tensor):
raise TypeError(f"Input boxes is not a Tensor. Got: {type(boxes)}.")
~ <--- HERE

if not boxes.is_floating_point():
    if raise_if_not_floating_point:
        raise ValueError(f"Coordinates must be in floating point. Got {boxes.dtype}.")

    boxes = boxes.float()

if len(boxes.shape) == 0:
    # Use reshape, so we don't end up creating a new tensor that does not depend on

TensorFlow2.3.0 corresponds to cuda 10.1, but pytorch1.8.1 corresponds to cuda 10.2. Does this result in testing using CPU instead of gpu?

conda install pytorch==1.8.1 torchvision==0.9.1 torchaudio==0.8.1 cudatoolkit=10.2 -c pytorch

pybullet build time: Mar 26 2022 03:00:15
2022-04-06 12:59:08.046104: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2022-04-06 12:59:08.053577: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-04-06 12:59:08.054386: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: Quadro RTX 8000 computeCapability: 7.5
coreClock: 1.77GHz coreCount: 72 deviceMemorySize: 47.46GiB deviceMemoryBandwidth: 625.94GiB/s
2022-04-06 12:59:08.054535: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:982] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-04-06 12:59:08.054721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 1 with properties:
pciBusID: 0000:02:00.0 name: Quadro RTX 8000 computeCapability: 7.5
coreClock: 1.77GHz coreCount: 72 deviceMemorySize: 47.46GiB deviceMemoryBandwidth: 625.94GiB/s
2022-04-06 12:59:08.054852: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/fty98/.conda/envs/ravens/lib/python3.7/site-packages/cv2/../../lib64:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin::/usr/local/cuda/lib64
2022-04-06 12:59:08.054927: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcublas.so.10
2022-04-06 12:59:08.054970: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcufft.so.10
2022-04-06 12:59:08.054999: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcurand.so.10
2022-04-06 12:59:08.055033: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusolver.so.10
2022-04-06 12:59:08.055058: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcusparse.so.10
2022-04-06 12:59:08.055176: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/fty98/.conda/envs/ravens/lib/python3.7/site-packages/cv2/../../lib64:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin:/home/fty98/.mujoco/mujoco200/bin::/usr/local/cuda/lib64
2022-04-06 12:59:08.055197: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1753] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
No GPUs detected. Running with CPU.

Hi! I tested the trained checkpoints file, and the following appeared in the terminal. It seems that there is no error. May I ask if this is testing? Where is the .pkl file?

(etn) fty98@debian-rtx8000:~/lab/fty/Equivariant-Transporter-Net$ python gripper_test.py --equ --lite --angle_lite --task=block-insertion --n_demos 1 --n_steps 10002022-04-01 12:15:20.677732: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/usr/local/cuda/lib64
2022-04-01 12:15:20.677804: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
pybullet build time: Mar 26 2022 03:00:15
text argument:./raven/assets
equi agent
/home/fty98/.conda/envs/etn/lib/python3.7/site-packages/e2cnn/nn/modules/r2_conv/basisexpansion_singleblock.py:80: UserWarning: indexing with dtype torch.uint8 is now deprecated, please use a dtype torch.bool instead. (Triggered internally at /opt/conda/conda-bld/pytorch_1616554786529/work/aten/src/ATen/native/IndexingUtils.h:30.)
full_mask[mask] = norms.to(torch.uint8)
Loading pre-trained model at 1000 iterations.
./checkpoints_equ/block-insertion-1-0/transport-ckpt2-1000.pt
int args: [

Hi,
first off I want to thank you for publishing an actually working codebase and well documented project.
I am using Eq-Transporter Nets in my Master Thesis and when looking through the code in the Attention and Transport modules I noticed a potential bug when crops are taken from the input images.

e.g. Line 121 from the attention module: crop = input_tensor[:,:,p[0]:(p[0] + self.crop_size),p[1]:(p[1] + self.crop_size)]
or Line 89 from the transport module: crop = crop[:, :, p[0]:(p[0] + self.crop_size_1), p[1]:(p[1] + self.crop_size_1)]

Wouldn't this put the upper left corner of the crop window at the given location instead of centering the crop window at the location?
I would expect the crop to be made like this:
crop = input_tensor[:,:,(p[0]-crop_size/2):(p[0]+crop_size/2),(p[1]-crop_size/2):(p[1]+crop_size/2)]
So that it is centered at the pick location.

Is this an actual bug or am I just missing something here?