Is there training and decoding recipe for the ASR task about lingvo HOT 5 CLOSED

You have to wait for a lot of more steps to know if it's working.

We have ASR decoding as an InferenceGraph. During training, we run the decoder. If you set it up correctly, this should land in tensorboard.

from lingvo.

I0314 11:15:01.696768 139831054587648 trainer.py:371] Steps/second: 0.065135, Examples/second: 8.545684
I0314 11:15:09.285109 139831558129408 trainer.py:521] step: 21 grad_norm/all:97.655243 grad_scale_all:1 log_pplx:8.0629063 loss:127.90414 num_samples_in_batch:256 var_norm/all:500.48935
I0314 11:15:11.704150 139831054587648 trainer.py:371] Steps/second: 0.066233, Examples/second: 8.528275
I0314 11:15:21.711497 139831054587648 trainer.py:371] Steps/second: 0.064206, Examples/second: 8.267336
I0314 11:15:25.010360 139831558129408 trainer.py:521] step: 22 grad_norm/all:1374.1166 grad_scale_all:1 log_pplx:8.244091 loss:331.20636 num_samples_in_batch:80 var_norm/all:500.49078
I0314 11:15:31.745743 139831054587648 trainer.py:371] Steps/second: 0.065262, Examples/second: 9.540066
I0314 11:15:33.404177 139831558129408 trainer.py:521] step: 23 grad_norm/all:47.343914 grad_scale_all:1 log_pplx:7.901073 loss:57.3908 num_samples_in_batch:512 var_norm/all:500.49213
I0314 11:15:41.733783 139831054587648 trainer.py:371] Steps/second: 0.066265, Examples/second: 9.726511
I0314 11:15:44.285320 139831558129408 trainer.py:521] step: 24 grad_norm/all:761.16956 grad_scale_all:1 log_pplx:8.65769 loss:216.17171 num_samples_in_batch:160 var_norm/all:500.49359
I0314 11:15:51.742242 139831054587648 trainer.py:371] Steps/second: 0.067208, Examples/second: 9.677933
I0314 11:15:59.431760 139831558129408 trainer.py:521] step: 25 grad_norm/all:1201.0476 grad_scale_all:1 log_pplx:8.2714634 loss:331.16873 num_samples_in_batch:80 var_norm/all:500.49509
I0314 11:16:01.753633 139831054587648 trainer.py:371] Steps/second: 0.068099, Examples/second: 9.631927
I0314 11:16:11.763622 139831054587648 trainer.py:371] Steps/second: 0.066291, Examples/second: 9.376266
I0314 11:16:15.647465 139831558129408 trainer.py:521] step: 26 grad_norm/all:1433.9872 grad_scale_all:1 log_pplx:8.3032694 loss:337.83926 num_samples_in_batch:80 var_norm/all:500.49652
I0314 11:16:21.770076 139831054587648 trainer.py:371] Steps/second: 0.067161, Examples/second: 9.795187
I0314 11:16:24.920377 139831558129408 trainer.py:521] step: 27 grad_norm/all:101.16747 grad_scale_all:1 log_pplx:8.4395685 loss:129.95616 num_samples_in_batch:256 var_norm/all:500.49808
I0314 11:16:31.783205 139831054587648 trainer.py:371] Steps/second: 0.067986, Examples/second: 9.951100
I0314 11:16:36.002479 139831558129408 trainer.py:521] step: 28 grad_norm/all:225.24529 grad_scale_all:1 log_pplx:8.3123541 loss:213.52359 num_samples_in_batch:160 var_norm/all:500.49957

I am following the jupyter notebook and it is giving like this......
tell me if I am going wrong

from lingvo.

That looks like it is running fine. Did you have some problems with it?

from lingvo.

I am trying to use trained model in Lingvo to decode my own test set. But I did not find a recipe to do this. Do you mind give me a hint on it?

from lingvo.

@zhoudoufu were you able to decode. I am stuck at the same problem. Let me know if you have a solution?

from lingvo.