danbigioi / diffusionvideoediting Goto Github PK

Hi Dan, hope everything is going well :)

I couldn't find implementations for some of the evaluation metrics. Specifically, LSE-C, LSE-D, SSIM, PSNR, CPBD.

Could you please guide where I can locate these scripts from the projects, or which GitHub repo I should visit to borrow these codes?

Thanks for your time and patience :)

Li

Hi team, may I ask what's the approximate training time with the default configuration? Thank you !:)

Since the training pipeline of diffuse head is unclear, can I use the code to train talking head generation model just like diffuse heads!

Good job， could you upload your data preprocessing scripts

Hi there,

I was trying to train the model and I came across this error. My server is 4*V100 GPU same as the setting reported in your manuscript. Can you help me check the reason?

Thank you!

(DVE) jicheng@lambda-server:~/DiffusionVideoEditing$ python run.py -c config/audio_talking_heads.json -p train
export CUDA_VISIBLE_DEVICES=0,1,2,3
using GPU 0 for training
using GPU 3 for training
using GPU 2 for training
using GPU 1 for training
/home/jicheng/DiffusionVideoEditing/run.py:28: UserWarning: You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True
warnings.warn('You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True')
<core.logger.InfoLogger object at 0x7f34f7f70760>
/home/jicheng/DiffusionVideoEditing/run.py:28: UserWarning: You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True
warnings.warn('You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True')
<core.logger.InfoLogger object at 0x7f8152b19760>
/home/jicheng/DiffusionVideoEditing/run.py:28: UserWarning: You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True
warnings.warn('You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True')
<core.logger.InfoLogger object at 0x7f5059173760>
/home/jicheng/DiffusionVideoEditing/run.py:28: UserWarning: You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True
warnings.warn('You have chosen to use cudnn for accleration. torch.backends.cudnn.enabled=True')
<core.logger.InfoLogger object at 0x7f9b0f919760>
done1
done1
done1
done1
done2
done2
done2
done2
hello there
hello there
hello there
hello there
hello there
hello there
hello there
hello there
done3
0%| | 0/2690 [00:00<?, ?it/s]done3
done3
0%| | 0/2690 [00:00<?, ?it/s]done3
0%| | 0/2690 [00:00<?, ?it/s]INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.888828 10138 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.898816 14040 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.899549 9704 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.905575 4768 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.906440 8967 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.906997 7471 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.911995 5160 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.911994 13659 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
F20230801 21:50:32.908715 11734 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
WARNING: Logging before InitGoogleLogging() is written to STDERR
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.996906 3905 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
F20230801 21:50:32.996906 1846 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
F20230801 21:50:32.996907 3535 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.997499 49046 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.997561 12889 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:32.998524 984 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
*** Check failure stack trace: ***
F20230801 21:50:32.997519 6317 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.002343 13270 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.010723 5785 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.002553 2338 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.006657 12117 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.012253 6315 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.012764 6943 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.008280 6702 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.013471 9800 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.009902 4003 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.021328 9404 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.022063 13274 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.026723 8247 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.028404 13272 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.050551 10959 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.017261 13944 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
0%| | 0/2690 [01:29<?, ?it/s]
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.432536 1418 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.434023 14760 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
0%| | 0/2690 [01:29<?, ?it/s]
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.448812 15144 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.452442 10525 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.462917 2736 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.467746 12501 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.469575 3078 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.471261 11737 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.472213 609 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.473325 9018 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.486632 8246 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.486693 12118 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.499100 9015 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.504900 1417 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.505055 14336 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.513185 1847 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
WARNING: Logging before InitGoogleLogging() is written to STDERR
F20230801 21:50:33.523406 5929 threadpool_pthread_impl.cc:51] Check failed: res == 0 (11 vs. 0) pthread_create failed
*** Check failure stack trace: ***
0%| | 0/2690 [01:30<?, ?it/s]
0%| | 0/2690 [01:30<?, ?it/s]
Traceback (most recent call last):
File "run.py", line 96, in
mp.spawn(main_worker, nprocs=ngpus_per_node, args=(ngpus_per_node, opt))
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 240, in spawn
return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 198, in start_processes
while not context.join():
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 160, in join
raise ProcessRaisedException(msg, error_index, failed_process.pid)
torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 3 terminated with the following error:
Traceback (most recent call last):
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1011, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/queue.py", line 179, in get
self.not_empty.wait(remaining)
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/threading.py", line 306, in wait
gotit = waiter.acquire(True, timeout)
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
_error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 8967) is killed by signal: Aborted.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/jicheng/DiffusionVideoEditing/run.py", line 63, in main_worker
model.train()
File "/home/jicheng/DiffusionVideoEditing/core/base_model.py", line 45, in train
train_log = self.train_step()
File "/home/jicheng/DiffusionVideoEditing/models/model.py", line 126, in train_step
for train_data in tqdm.tqdm(self.phase_loader):
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter
for obj in iterable:
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 530, in next
data = self._next_data()
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1207, in _next_data
idx, data = self._get_data()
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1163, in _get_data
success, data = self._try_get_data()
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1024, in _try_get_data
raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 8967) exited unexpectedly

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/jicheng/anaconda3/envs/DVE/lib/python3.8/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap
fn(i, *args)
File "/home/jicheng/DiffusionVideoEditing/run.py", line 69, in main_worker
phase_writer.close()
File "/home/jicheng/DiffusionVideoEditing/core/logger.py", line 118, in close
self.writer.close()
AttributeError: 'NoneType' object has no attribute 'close'

Thanks for your awesome and creative work!
I read your paper and try to reproduction the performance. but via diffusion method, it will be convenient and helpful if you could provide the pretrained model, Thanks.

May I ask how long it will take and what equipment will be used to train the model once?

I am running the inference script but getting somewhat unprocessed images like the one below. Only the first image in Generated_Frames is decent the rest all are pixelated. There are no errors thrown so I don't know exactly what is going wrong. Any help would be appreciated!

Is it possible to upload the wav files of the audio features? I tried to revert it by using inverse melspectrogram method from Librosa package but it failed.