The adamae from wgcban

adamae's Introduction

Welcome to My GitHub Profile!

My research interests lie at the intersection of Computer Vision and Deep Learning, with a specific focus on:

Self-Supervised Learning (SSL)
Generative AI
Remote Sensing
Low-Level Vision

🌐 Homepage: www.wgcban.com

Featured Projects

Self-Supervised Representation Learning

Guarding Barlow Twins Against Overfitting with Mixed Samples GitHub 🚨
AdaMAE: Adaptive Masking for Efficient Spatiotemporal Learning with Masked Autoencoders @ CVPR'23 GitHub

Generative AI

Unite and Conquer: Cross Dataset Multimodal Synthesis using Diffusion Models @ CVPR'23 GitHub

Remote Sensing

Change Detection

ChangeFormer: A Transformer-Based Siamese Network for Change Detection @ IGARSS'22 GitHub
DDPM-CD: Remote Sensing Change Detection using Denoising Diffusion Probabilistic Models GitHub
Revisiting Consistency Regularization for Semi-supervised Change Detection in Remote Sensing Images GitHub
Metric-CD: Deep Metric Learning for Unsupervised Remote Sensing Change Detection GitHub

Image Super-Resolution / Pansharpening

HyperTransformer: A Textural and Spectral Feature Fusion Transformer for Pansharpening @ CVPR'22 GitHub
DIP-HyperKite: Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction @ TGRS'22 GitHub

Segmentation

SPIN Road Mapper: Extracting Roads from Aerial Images via Spatial and Interaction Space Graph Reasoning for Autonomous Driving @ ICRA'22 GitHub

SAR Despeckling

SAR-Transformer: Transformer-based SAR Image Despeckling @ IGARSS'22 GitHub
SAR-Overcomplete: SAR Despeckling Using Overcomplete Convolutional Networks @ IGARSS'22 GitHub
SAR-DDPM: SAR Despeckling using a Denoising Diffusion Probabilistic Model @ Geoscience and Remote Sensing Letters GitHub

Low-level Vision

Diffuse-Denoise-Count: Accurate Crowd-Counting with Diffusion Models GitHub

adamae's People

Contributors

Stargazers

Watchers

adamae's Issues

Sampling from Probability Distributions

Hi @wgcban

Thank you for your paper and code for AdaMAE.

In this line a Multinomial distribution is used for sampling the indices for the visible tokens given the probability p_x. Could you please explain if this operation would be differentiable during back-propagation?

From what I understand REINFORCE is applied in this part (from Line 71 to Line 80). Is there any connection between sampling from a Categorical distribution in this part and the one from Multinomial distribution above? I am a bit confused. Could you please clarify?

How to obtain the reconstructed image for inference and masked

Hello, I very much agree with your work. I would like to know how to obtain the schematic diagram of the reconstructed image and the mask image during the experiment. Because I'm just getting started. I really appreciate it if you can help me with this question.

Cannot reproduce SSV2 finetuning results

Hi, thank you for this great work. I'm trying to reproduce the results for SSV2 pretraining -> SSV2 fine-tuning using the checkpoint provided in the repo. I tried to follow this script https://github.com/wgcban/adamae/blob/main/scripts/kinetics/videomae_vit_base_patch16_224_learnablemasking_ratio_0.95_epoch_800/example-finetune.sh (I think the scripts for kinetics and ssv2 were put in the wrong folder), with the only difference being I'm using 32 gpus with batch size=16 for each GPU instead of 64 gpus with batch size=8 for each GPU. However, I only got 68.7 top 1 accuracy. I noticed in the paper that ssv2 fine-tuning base learning rate is 1.5e-4 instead of 5e-4 that is shown in the script. Could you confirm the configurations for fine-tuning ssv2? Thanks in advance.

Pre-trained models' epoch values don't match paper

Hi, thanks for sharing the great work. When I tried loading the provided pre-trained models and printing the values corresponding to their 'epoch' keys, I observed values of 1199 for SSv2 and 1899 for K400. Could you please confirm that the provided models are the ones corresponding to 800 epochs of pretraining as stated in the paper?

Recommend Projects

wgcban / adamae Goto Github PK