Comments (4)
krandiash
commented on July 26, 2024
1
Hi thanks for the question!
You can resume training by passing in a path to the checkpoint using the trainer.resume_from_checkpoint flag. So for example,
CUDA_VISIBLE_DEVICES=0 python -m train experiment=s4-lra-pathx loader.batch_size=16 trainer.accumulate_grad_batches=2 trainer.resume_from_checkpoint=/path/to/checkpointThis will start another wandb run that picks up where the last one left off. You'll want to point to the last.ckpt file that gets saved to the logging folder (your training logs should point you to where that is).
from s4.
krandiash
commented on July 26, 2024
1
That's right: you should see a new (partial) run on the dashboard.
from s4.
ag1988
commented on July 26, 2024
Thank you so much for the prompt response Karan! I'll follow your
instructions and let you know if I have any further questions!
Regards,
Ankit
…On Thu, Feb 24, 2022 at 4:04 PM Karan Goel ***@***.***> wrote:
Hi thanks for the question!
You can resume training by passing in a path to the checkpoint using the
trainer.resume_from_checkpoint flag. So for example,
CUDA_VISIBLE_DEVICES=0 python -m train experiment=s4-lra-pathx loader.batch_size=16 trainer.accumulate_grad_batches=2 trainer.resume_from_checkpoint=/path/to/checkpoint
This will start another wandb run that picks up where the last one left
off. You'll want to point to the last.ckpt file that gets saved to the
logging folder (your training logs should point you to where that is).
—
Reply to this email directly, view it on GitHub
<#7 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AID3IGG2IAOZPOV4DI7SYNDU42MPRANCNFSM5PH7YPWQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
from s4.
ag1988
commented on July 26, 2024
Btw just so that I get it right, you mentioned that wandb will pick up from where the last run left off. That means that it wont copy the logs of the previous run and will appear as a partial run on the dashboard, right?
from s4.
Related Issues (20)
- Paper, Table 1, Convolution number of parameters HOT 2
- About `krylov()` HOT 1
- Missing or misplaced "old" config folder? HOT 4
- "pretrained_model" is not defined before being called in train.py HOT 2
- Question on HMDB51 Dataset (S4ND Video Experiment)
- Unable to generate the weather using generate.py with time Series training checkpoint
- Large difference of inference result between forward and step
- AttributeError: 'SSMKernelDPLR' object has no attribute 'kernel' HOT 1
- Training on 12bits audio instead of 8bit? (Question, what do I need to change?)
- S4 Listops have nan loss HOT 2
- Quantization for S4/ Hippo
- The dynamics of the latent state of the model
- segmentation fault when running python -m train pipeline=mnist model=s4 HOT 1
- how to use the S4Block .step()
- KeyError in train.py self.dataset = SequenceDataset.registry[self.hparams.dataset._name_]
- Why is Sashimi's effect in speech signal enhancement (denoisy) so bad？
- Passing a video to S4ND
- CUDA error: no kernel image is available for execution on the device HOT 3
- Different results on the same dataset with the same Settings in the paper s4d
- Help with Forex Dataset Configuration for s4 Model
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from s4.