phelps-matthew / feathermap Goto Github PK

When I clone the repository, it crosses over 500MB. Are there large files in the git history?

Hi, I'm trying to get feather map working with SIREN, which uses uniformly initialized Linear layers (with varying scales).

Does FeatherMap support only fan in at the moment?

If I wanted to use Quantization Aware Training (QAT) in conjunction with structured hashing, should I quantize before or after FeatherMap?

i.e, (before, intuitively seems correct to me):

# model has appropriate torch.quantization.DeQuantStub()
model = ResNet50()

model.train()
model.qconfig = torch.quantization.get_default_qat_qconfig('fbgemm')
model_fp32_prepared = torch.quantization.prepare_qat(model_fp32)

# feather map stuff
model_fp32_prepared = FeatherNet(model_fp32_prepared, compress=0.10)

# train loop

# eval stuff
model_fp32_prepared.eval()
model_int8 = torch.quantization.convert(model_fp32_prepared)
model_int8.deploy()

To be specific, I'm interested in the low-rank decomposition being INT8.