Getting all black images from test reconstruction about codef HOT 6 CLOSED

Hi, which video are you using? It is possible to get black during training when the black region is very large. You can set the parameter sigmoid_offset to non-zero.

from codef.

Hi,

I am working with the provided white_smoke data.

Here is my train script:

GPUS=4

NAME=white_smoke
EXP_NAME=base

ROOT_DIRECTORY="all_sequences/$NAME/$NAME"
MODEL_SAVE_PATH="ckpts/all_sequences/$NAME"
LOG_SAVE_PATH="logs/all_sequences/$NAME"

python train.py --root_dir $ROOT_DIRECTORY
--model_save_path $MODEL_SAVE_PATH
--log_save_path $LOG_SAVE_PATH
--gpus $GPUS
--encode_w --annealed
--config configs/${NAME}/${EXP_NAME}.yaml
--exp_name ${EXP_NAME} \

Here is my test script:

GPUS=4

NAME=white_smoke
EXP_NAME=base

ROOT_DIRECTORY="all_sequences/$NAME/$NAME"
LOG_SAVE_PATH="logs/test_all_sequences/$NAME"

#WEIGHT_PATH=ckpts/all_sequences/$NAME/${EXP_NAME}/${NAME}.ckpt
WEIGHT_PATH=ckpts/all_sequences/$NAME/${EXP_NAME}/step=8000.ckpt

python train.py --test --encode_w
--root_dir $ROOT_DIRECTORY
--log_save_path $LOG_SAVE_PATH
--weight_path $WEIGHT_PATH
--gpus $GPUS
--config configs/${NAME}/${EXP_NAME}.yaml
--exp_name ${EXP_NAME}
--save_deform False

One thing I noticed is that after the training script finishes with 10K steps, there is no checkpoint with ${NAME}.ckpt, so I use the step=8000.ckpt instead.

from codef.

It is indeed unusual to get completely black results with this sequence and I cannot reproduce it. I would suggest a couple of steps to potentially resolve this issue:

1. Try training the model a few more times. Typically, the PSNR tends to be quite high after only a few iterations. If the output is entirely black, the PSNR would generally not be as high you can early stop it.

2. If multiple training attempts still lead to the same issue, consider adjusting the sigmoid_offset to 0.1. This parameter can influence the output, and tweaking it might help in avoiding entirely black results.

from codef.

cl

from codef.

Hi,

I ran test_multi.sh using the pre-trained beauty_0 ckpts and am still getting all black images -- do you have an idea on why this may be happening? I'm using the same versions for the listed requirements.

from codef.

I dug through the code and made sure the input into the decoder at this line is non-zero, but found that the output is zero. Is there a specific version of tcnn you are using?

https://github.com/qiuyu96/CoDeF/blob/main/models/implicit_model.py#L312

from codef.