Questions regarding pre-training about climax HOT 5 CLOSED

Hi, thanks for showing your interest in ClimaX.

1. It took 2-3 days.
   You're right about the scheduler. We used the same number of finetuning epochs (50) for all tasks, but since different tasks have different numbers of data points, the finetuning steps should be different. You can follow this schedule: linear warmup for 5 epochs and cosine decay for 45 epochs.
2. What do you mean by temporal resolution? We use only 1 step in the input, which means it has a shape of V x H x W
3. We ran some experiments in the early stage of the project, and the takeaway is ViT overfits heavily without any pretraining. You need to use very heavy dropout rates and/or weight decays to mitigate this.

from climax.

Thanks for the response!

Regarding the temporal resolution, I am interested in the frequency of input assimilations or initial states included in the training set (sampled from the hourly ERA5 data). For instance, FourCastNet and GraphCast use assimilations at 00, 06, 12, and 18H UTC, resulting in four training samples per day. On the other hand, Pangu-Weather employs hourly initial states.

Could you provided details on how often ERA5 data is sampled for ClimaX's training set?

from climax.

Hi @itsnamgyu! Thanks for your question. One key difference in inference setup for ClimaX vs other methods you mentioned is that we condition on $\Delta t$ in the future we want to make predictions for and can therefore make a prediction at any hour in a single forward pass vs doing rollouts in other methods.

Specifically

from climax.

@itsnamgyu I assume your question has been answered. Feel free to open it again if you have more questions.

from climax.

Oh, yes, I was asking about this information We used the available hourly ERA5 data here. Thanks for the help :)

from climax.