Select a model to train and run on TPUs about xla HOT 10 OPEN

/assigntome

from xla.

@duncantech I am thinking of training the latest Gemma model, indeed, with Pytorch XLA. Is it okay then?

from xla.

I think the geema model should work out of box. Take a look at https://github.com/google/gemma_pytorch#try-it-out-with-pytorchxla. Feel free to give it a try and see if we can improve anything.

from xla.

I think the geema model should work out of box. Take a look at https://github.com/google/gemma_pytorch#try-it-out-with-pytorchxla. Feel free to give it a try and see if we can improve anything.

Ok. I will look into the gemma part.

For a different model I am trying with, a few things I need to know: do I need to use any free cloud tpu provider, for example, Kaggle or Colab tpu, or is it necessary to do it with the v5 in Google Cloud?

from xla.

That part I think @duncantech can answer.

from xla.

You can work with a free TPU provider if you'd like to get things started.

We should also be able to give a small amount of v5es to try with too.

from xla.

@sitamgithub-MSIT we haven't heard an update in a bit and just wondering if you're still working on the issue?

from xla.

@sitamgithub-MSIT we haven't heard an update in a bit and just wondering if you're still working on the issue?

Yes I am working to it. I am checking this example in the hugging face for Gemma. I am thinking about reproducing the same for CodeGemma, though.

from xla.