ml4its / timevqvae Goto Github PK

/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/configuration_validator.py:108: PossibleUserWarning: You defined a validation_step but have no val_dataloader. Skipping val loop.
rank_zero_warn(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

| Name | Type | Params

0 | encoder_l | VQVAEEncoder | 265 K
1 | decoder_l | VQVAEDecoder | 265 K
2 | vq_model_l | VectorQuantize | 0
3 | encoder_h | VQVAEEncoder | 167 K
4 | decoder_h | VQVAEDecoder | 166 K
5 | vq_model_h | VectorQuantize | 0

864 K Trainable params
0 Non-trainable params
864 K Total params
3.458 Total estimated model params size (MB)
/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py:1600: PossibleUserWarning: The number of training batches (1) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
rank_zero_warn(
Epoch 0: 0% 0/1 [00:00<?, ?it/s] /usr/local/lib/python3.10/dist-packages/torch/functional.py:641: UserWarning: stft with return_complex=False is deprecated. In a future pytorch release, stft will return complex tensors for all inputs, and return_complex=False will raise an error.
Note: you can still call torch.view_as_real on the complex output to recover the old return format. (Triggered internally at ../aten/src/ATen/native/SpectralOps.cpp:862.)
return _VF.stft(input, n_fft, hop_length, win_length, window, # type: ignore[attr-defined]
Traceback (most recent call last):
File "/content/TimeVQVAE/stage1.py", line 84, in
train_stage1(config, train_data_loader)
File "/content/TimeVQVAE/stage1.py", line 50, in train_stage1
trainer.fit(train_exp,
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 608, in fit
call._call_and_handle_interrupt(
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/call.py", line 38, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 650, in _fit_impl
self._run(model, ckpt_path=self.ckpt_path)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 1103, in _run
results = self._run_stage()
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 1182, in _run_stage
self._run_train()
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 1205, in _run_train
self.fit_loop.run()
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/fit_loop.py", line 267, in advance
self._outputs = self.epoch_loop.run(self._data_fetcher)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 213, in advance
batch_output = self.batch_loop.run(kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/batch/training_batch_loop.py", line 88, in advance
outputs = self.optimizer_loop.run(optimizers, kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 202, in advance
result = self._run_optimization(kwargs, self._optimizers[self.optim_progress.optimizer_position])
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 249, in _run_optimization
self._optimizer_step(optimizer, opt_idx, kwargs.get("batch_idx", 0), closure)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 370, in _optimizer_step
self.trainer._call_lightning_module_hook(
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 1347, in _call_lightning_module_hook
output = fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/core/module.py", line 1744, in optimizer_step
optimizer.step(closure=optimizer_closure)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/core/optimizer.py", line 169, in step
step_output = self._strategy.optimizer_step(self._optimizer, self._optimizer_idx, closure, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/strategies/strategy.py", line 234, in optimizer_step
return self.precision_plugin.optimizer_step(
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 119, in optimizer_step
return optimizer.step(closure=closure, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/optim/lr_scheduler.py", line 69, in wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/optim/optimizer.py", line 280, in wrapper
out = func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/optim/optimizer.py", line 33, in _use_grad
ret = func(self, *args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/optim/adamw.py", line 148, in step
loss = closure()
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/plugins/precision/precision_plugin.py", line 105, in _wrap_closure
closure_result = closure()
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 149, in call
self._result = self.closure(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 135, in closure
step_output = self._step_fn()
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/loops/optimization/optimizer_loop.py", line 419, in _training_step
training_step_output = self.trainer._call_strategy_hook("training_step", *kwargs.values())
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/trainer/trainer.py", line 1485, in _call_strategy_hook
output = fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/pytorch_lightning/strategies/strategy.py", line 378, in training_step
return self.model.training_step(*args, **kwargs)
File "/content/TimeVQVAE/experiments/exp_vq_vae.py", line 137, in training_step
recons_loss, vq_losses, perplexities = self.forward(x)
File "/content/TimeVQVAE/experiments/exp_vq_vae.py", line 67, in forward
x_l = timefreq_to_time(u_l, self.n_fft, C) # (B, C, L)
File "/content/TimeVQVAE/utils/init.py", line 226, in timefreq_to_time
x = torch.istft(x, n_fft, normalized=False, return_complex=False)
RuntimeError: istft requires a complex-valued input tensor matching the output from stft with return_complex=True.

hi，I'm very interesting in your job,but i don't know how to use this model in my dataset,can you tell me some detail.Very thanks for your answer.

Hello,

I am deeply intrigued by your research and have been diligently monitoring its progress. I would like to express my utmost gratitude for generously sharing your exceptional research findings. I eagerly anticipate the forthcoming advancements in your work.

Upon running the provided stage1.py script in my Python environment, I encountered the aforementioned error during the execution of the istft function. It appears that I may be using a distinct version of PyTorch. Could you kindly suggest the recommended approach for utilizing it?

Thank you kindly.

Hello,

First thank you very much for the simplicity of reproducing your work, the notebook is very straightforward, and I really enjoyed reading your work.

While reproducing the generation for all the datasets using the notebook and the pre-trained models you provide, I noticed that for 33 datasets the generations have 1 time stamp less than the original dataset. The generations are correct though I plotted them and they have the right form but when i print the shape they have one time stamp less. Here is the list of the datasets:

['ArrowHead',
'Car',
'CinCECGTorso',
'DiatomSizeReduction',
'EthanolLevel',
'FaceAll',
'FacesUCR',
'Fish',
'FreezerRegularTrain',
'FreezerSmallTrain',
'Fungi',
'GesturePebbleZ1',
'GesturePebbleZ2',
'Ham',
'HandOutlines',
'InsectEPGRegularTrain',
'InsectEPGSmallTrain',
'Lightning2',
'Lightning7',
'MedicalImages',
'OSULeaf',
'PickupGestureWiimoteZ',
'ShakeGestureWiimoteZ',
'SmoothSubspace',
'SonyAIBORobotSurface2',
'Strawberry',
'ToeSegmentation1',
'ToeSegmentation2',
'Trace',
'UWaveGestureLibraryAll',
'UWaveGestureLibraryX',
'UWaveGestureLibraryY',
'UWaveGestureLibraryZ']

Can you let me know if:

1- This is my wrong doing or do you have the same thing when producing them yourselves.
2- If you do notice the same thing, do you know the reason your model does that ?

Thank you for your response in advance.