Load pre-trained Imagenet weights on Mvit

[williamberrios]

Hi all,

I was wondering if someone could load the pre-trained weights of Mvit from Imagenet? I haven't found any documentation in order to load it from .pyth.

Thank you in advance!!

[hanoonaR]

Hi @williamberrios, I could load the pre-trained weights with pytorch.
This is how I did it:
model.load_state_dict(checkpoint['model_state'], strict=False)

[williamberrios]

Thank you @hanoonaR, however when I use that I see the following warning.
_IncompatibleKeys(missing_keys=['blocks.0.attn.q.weight', 'blocks.0.attn.q.bias', 'blocks.0.attn.k.weight', 'blocks.0.attn.k.bias', 'blocks.0.attn.v.weight', 'blocks.0.attn.v.bias', 'blocks.1.attn.q.weight', 'blocks.1.attn.q.bias', 'blocks.1.attn.k.weight', 'blocks.1.attn.k.bias', 'blocks.1.attn.v.weight', 'blocks.1.attn.v.bias', 'blocks.2.attn.q.weight', 'blocks.2.attn.q.bias', 'blocks.2.attn.k.weight', 'blocks.2.attn.k.bias', 'blocks.2.attn.v.weight', 'blocks.2.attn.v.bias', 'blocks.3.attn.q.weight', 'blocks.3.attn.q.bias', 'blocks.3.attn.k.weight', 'blocks.3.attn.k.bias', 'blocks.3.attn.v.weight', 'blocks.3.attn.v.bias', 'blocks.4.attn.q.weight', 'blocks.4.attn.q.bias', 'blocks.4.attn.k.weight', 'blocks.4.attn.k.bias', 'blocks.4.attn.v.weight', 'blocks.4.attn.v.bias', 'blocks.5.attn.q.weight', 'blocks.5.attn.q.bias', 'blocks.5.attn.k.weight', 'blocks.5.attn.k.bias', 'blocks.5.attn.v.weight', 'blocks.5.attn.v.bias', 'blocks.6.attn.q.weight', 'blocks.6.attn.q.bias', 'blocks.6.attn.k.weight', 'blocks.6.attn.k.bias', 'blocks.6.attn.v.weight', 'blocks.6.attn.v.bias', 'blocks.7.attn.q.weight', 'blocks.7.attn.q.bias', 'blocks.7.attn.k.weight', 'blocks.7.attn.k.bias', 'blocks.7.attn.v.weight', 'blocks.7.attn.v.bias', 'blocks.8.attn.q.weight', 'blocks.8.attn.q.bias', 'blocks.8.attn.k.weight', 'blocks.8.attn.k.bias', 'blocks.8.attn.v.weight', 'blocks.8.attn.v.bias', 'blocks.9.attn.q.weight', 'blocks.9.attn.q.bias', 'blocks.9.attn.k.weight', 'blocks.9.attn.k.bias', 'blocks.9.attn.v.weight', 'blocks.9.attn.v.bias', 'blocks.10.attn.q.weight', 'blocks.10.attn.q.bias', 'blocks.10.attn.k.weight', 'blocks.10.attn.k.bias', 'blocks.10.attn.v.weight', 'blocks.10.attn.v.bias', 'blocks.11.attn.q.weight', 'blocks.11.attn.q.bias', 'blocks.11.attn.k.weight', 'blocks.11.attn.k.bias', 'blocks.11.attn.v.weight', 'blocks.11.attn.v.bias', 'blocks.12.attn.q.weight', 'blocks.12.attn.q.bias', 'blocks.12.attn.k.weight', 'blocks.12.attn.k.bias', 'blocks.12.attn.v.weight', 'blocks.12.attn.v.bias', 'blocks.13.attn.q.weight', 'blocks.13.attn.q.bias', 'blocks.13.attn.k.weight', 'blocks.13.attn.k.bias', 'blocks.13.attn.v.weight', 'blocks.13.attn.v.bias', 'blocks.14.attn.q.weight', 'blocks.14.attn.q.bias', 'blocks.14.attn.k.weight', 'blocks.14.attn.k.bias', 'blocks.14.attn.v.weight', 'blocks.14.attn.v.bias', 'blocks.15.attn.q.weight', 'blocks.15.attn.q.bias', 'blocks.15.attn.k.weight', 'blocks.15.attn.k.bias', 'blocks.15.attn.v.weight', 'blocks.15.attn.v.bias'], unexpected_keys=['blocks.0.attn.qkv.weight', 'blocks.0.attn.qkv.bias', 'blocks.1.attn.qkv.weight', 'blocks.1.attn.qkv.bias', 'blocks.2.attn.qkv.weight', 'blocks.2.attn.qkv.bias', 'blocks.3.attn.qkv.weight', 'blocks.3.attn.qkv.bias', 'blocks.4.attn.qkv.weight', 'blocks.4.attn.qkv.bias', 'blocks.5.attn.qkv.weight', 'blocks.5.attn.qkv.bias', 'blocks.6.attn.qkv.weight', 'blocks.6.attn.qkv.bias', 'blocks.7.attn.qkv.weight', 'blocks.7.attn.qkv.bias', 'blocks.8.attn.qkv.weight', 'blocks.8.attn.qkv.bias', 'blocks.9.attn.qkv.weight', 'blocks.9.attn.qkv.bias', 'blocks.10.attn.qkv.weight', 'blocks.10.attn.qkv.bias', 'blocks.11.attn.qkv.weight', 'blocks.11.attn.qkv.bias', 'blocks.12.attn.qkv.weight', 'blocks.12.attn.qkv.bias', 'blocks.13.attn.qkv.weight', 'blocks.13.attn.qkv.bias', 'blocks.14.attn.qkv.weight', 'blocks.14.attn.qkv.bias', 'blocks.15.attn.qkv.weight', 'blocks.15.attn.qkv.bias'])

I think that is not ok :'(

[Anirudh257]

@haooooooqi @feichtenhofer Even I am facing this issue. It seems that the checkpoint provided here is trained on another version of the SlowFast code. It will be great if you can update the checkpoint.

[klauscc]

I got the same issue when loading the pretrained weights of K400 and K600. Do you know which commit these checkpoints are trained on?

[Anirudh257]

@klauscc please check my pull-request: #516