how to convert sparse model to dense model? about spark HOT 1 CLOSED

Actually we don't need a sparse-to-dense conversion (there is no "sparse"-type weights). You can use the pretrained weight directly like the following code (just like an ImageNet-supervised pretrained resnet50):

import torch, timm
res50 = timm.create_model('resnet50')
state = torch.load('resnet50_1kpretrained_timm_style.pth', 'cpu')
missing_keys, unexpected_keys = res50.load_state_dict(state, strict=False)
assert missing_keys == ['fc.weight', 'fc.bias']
assert len(unexpected_keys) == 0

The reason is we use torch builtin operators (nn.Conv2d, nn.LayerNorm, etc.) to simulate those sparse operators (see /pretrain/encoder.py). So it keeps the same weight formats and names.

from spark.