Hi good job, could you tell me how to fit your code for input size, such as 128128， 192192， 256*256 ？ thanks.

Please, do you have training classification code in ImageNet?

I have tried to load the model.ckpt through model.load_weights() and also followed the tutorial in using eval_driver but was not able to load them correctly. It showed AssertionError: Some objects had attributes which were not restored.

ModuleNotFoundError: No module named 'maxvit.models'; 'maxvit' is not a package

I have tried with preprocessing method:

img_ = eval_driver.get_preprocess_fn()(tf.io.read_file(path))

But get constantly errors like:

{function_node __wrapped__ExtractJpegShape_device_/job:localhost/replica:0/task:0/device:CPU:0}} Invalid JPEG data, size 442707 [Op:ExtractJpegShape]
Not a JPEG file: starts with 0x89 0x50
{{function_node __wrapped__ExtractJpegShape_device_/job:localhost/replica:0/task:0/device:CPU:0}} Invalid JPEG data, size 328248 [Op:ExtractJpegShape]
{{function_node __wrapped__ExtractJpegShape_device_/job:localhost/replica:0/task:0/device:CPU:0}} Invalid JPEG data, size 156137 [Op:ExtractJpegShape]
Not a JPEG file: starts with 0x89 0x50
Not a JPEG file: starts with 0x89 0x50
{{function_node __wrapped__ExtractJpegShape_device_/job:localhost/replica:0/task:0/device:CPU:0}} Invalid JPEG data, size 258371 [Op:ExtractJpegShape

• In maxvit.py class Attention #L216-L238 implementation, we have self.relative_bias = self.add_weight(...) added as weight first, then rearranged self.reindexed_bias = attn_utils.reindex_2d_einsum_lookup(...) in build, not in call. I'm not sure about this, but in my understanding, the second rearranged self.reindexed_bias is a copy of the first initialized weights, and thus the original weights will not get updated during training. I think the self.reindexed_bias = attn_utils.reindex_2d_einsum_lookup(...) operation should be moved into def call block, like into maxvit.py#L258.
• This Colab maxvit_train_test.ipynb is a basic training test of whether relative_bias is changed during training. Anyway, maybe it's I'm not using it correctly. please help confirm.
• BTW, this is my keras implementation of MaxViT Github keras_cv_attention_models/maxvit. Currently It's only MaxViT_T weights ported, as the ported relative_bias seems not right...

为什么计算局部注意力时，需要把特征图变换成 (H/P × W/P, P², C) 这个形状，即将P²放在倒数第二个维度？

而计算全局注意力时，则需要把特征图变换成 (G², H/G × W/G, C) 这个形状，然后再交换 【倒数第二个维度】 和 【倒数第三个维度】 的顺序，即变成 (H/G × W/G, G², C)，既然这种形式和局部形式相同，为什么不直接进行相同的变换呢，而是再去额外的交换维度？

how did you use max vit in gan and object detection and segmentation? Is code available? And how to fine tune this model on different data set, is it documented?

@vztu I wonder if you can please share the exact training details to reproduce the MaxViT-T performance (83.62%). By this, I wonder about the total batch size, number of GPU/TPUs, nodes, learning rate and other important considerations.

Hi, thanks your god job. Could you pls tell me how to use your ckpt file in the pytorch ? thx.

HI Sir, I am new here, thanks for your great work. I am not very familiar with the syntax of TensorFlow. Is there a model implemented by pytorch?

Thank you for your excellent work. Could you please provide a script for converting the tensorflow pretrained weights to the pythoch format?

I got the following error when trying to train and save a model:

NotImplementedError:
Layer MaxViT has arguments ['config']
in `__init__` and therefore must override `get_config()`.

Example:

class CustomLayer(keras.layers.Layer):
    def __init__(self, arg1, arg2):
        super().__init__()
        self.arg1 = arg1
        self.arg2 = arg2

    def get_config(self):
        config = super().get_config()
        config.update({
            "arg1": self.arg1,
            "arg2": self.arg2,
        })
        return config

I have tried install MaxVit on Windows, but get:

C:\Projects*\venv\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
C:\Projects*\venv\lib\site-packages\setuptools\command\easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
zip_safe flag not set; analyzing archive contents...
error: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\projects\**\venv\lib\site-packages\maxvit-1.0.0-py3.10.egg'

Thanks for the wonderful paper - it was a pleasure to read!

Could you kindly elaborate a bit more on the COCO training details? In particular, I was wondering about the following three points:

• What augmentations were used? Just large or small scale jittering and maybe flipping (left-right)?
• What learning rate schedule was used?
• Was the same learning rate for used for the different layers?