microsoft / tensorflow-rematerialization Goto Github PK

Thank you for your nice work on XLA that could significantly reduce memory usage!

Upon my understanding, the rematerialization is a trade-off between memory usage and speed in terms of xla (where the memory read would mostly faster than recalculation). I wonder if you could provide some information about the speed loss (if any) of each remat algorithm corresponds to the amount of memory saved. Thank you :).