mit-han-lab / dlg Goto Github PK

[NeurIPS 2019] Deep Leakage From Gradients

License: MIT License

Python 100.00%

dlg's Introduction

Deep Leakage From Gradients [arXiv] [Webside]

@inproceedings{zhu19deep,
  title={Deep Leakage from Gradients},
  author={Zhu, Ligeng and Liu, Zhijian and Han, Song},
  booktitle={Advances in Neural Information Processing Systems},
  year={2019}
}

Gradients exchaging is popular used in modern multi-node learning systems. People used to believe numerical gradients are safe to share. But we show that it is actually possible to obtain the training data from shared gradients and the leakage is pixel-wise accurate for images and token-wise matching for texts.

Overview

The core algorithm is to match the gradients between dummy data and real data.

It can be implemented in less than 20 lines with PyTorch!

def deep_leakage_from_gradients(model, origin_grad): 
  dummy_data = torch.randn(origin_data.size())
  dummy_label =  torch.randn(dummy_label.size())
  optimizer = torch.optim.LBFGS([dummy_data, dummy_label] )

  for iters in range(300):
    def closure():
      optimizer.zero_grad()
      dummy_pred = model(dummy_data) 
      dummy_loss = criterion(dummy_pred, F.softmax(dummy_label, dim=-1)) 
      dummy_grad = grad(dummy_loss, model.parameters(), create_graph=True)

      grad_diff = sum(((dummy_grad - origin_grad) ** 2).sum() \
        for dummy_g, origin_g in zip(dummy_grad, origin_grad))
      
      grad_diff.backward()
      return grad_diff
    
    optimizer.step(closure)
    
  return  dummy_data, dummy_label

Prerequisites

To run the code, following libraies are required

Python >= 3.6
PyTorch >= 1.0
torchvision >= 0.4

Code

Note: We provide for quick reproduction.

# Single image on CIFAR
python main.py --index 25

# Deep Leakage on your own Image
python main.py --image yours.jpg

Deep Leakage on Batched Images

Deep Leakage on Language Model

License

This repository is released under the MIT license. See LICENSE for additional details.

dlg's People

Contributors

Stargazers

Watchers

Forkers

niklausliu patrickzh advboxzoo yycen hustzxd lukeeeeee jeandut lumingzheng baowangmath zmhe lemonviv hattyhu yyuzhongpv kpansxxa junyizhu-ai zhenv5 linghm changqing1234 quanthsu smduan ilcyb zinuzian milkigit haoyangyang jgshu carprefer ashly1111111 nchen909 xianlimei kmapark vaibhav0195 a8252525 13227018679 som-don fanqihang maodong2056 cseuk6 lowbee2019 nvnhcmus cloud9bumsu cvarun16 stevenzhang9577 dy1ant hjh233 lyken17 chicharitozzz qixiaoye luckmonkeys tomer1203 giyoungjeon denliness ikace guttappa1238 eladgsofer litingfeng00 conan-saber hy-love-sleep zzrhh xiaoni-61 chrisnielsen-forked princess-aryitubay raywoo1 favorwen kelvinkan xdroc lhysgithub elijahahianyo niladridas pengyuzhang97 lingandzero qiu105 18z zhaitongqing233 cangxiongchen zhangzp9970 nastul leekichang mrshubw yanzhaowu peacepeacehan yld957 andycheng2 wangclin amit-eee kancheng hiroi-sora zzzidku gillhuang-xtler umityigitbsrn comradespark ngoctuansqtt c6ai huizhouhnu msinamsina yaldakasravi onestepandtwosteps 123jhc gaosimon runshapan shuklayash10

dlg's Issues

does dummy_label effective?

In your algorithm, you use "dummy_label", but "gt_label" in main.py and the code at hanlab. Can you provide your comprehensive implementation in your paper?

why I got random images?

It doesn't matter how many times I run this code, the result I get is a bunch of random images.

DLG on Bert

Can you provide the code of DLG on Bert？

DLG is available for non-twice-differentiable function.

Hi.
I try to use dlg to recover the data from non-twice-differentiable function, the algorithm successfully recover the data. Here are the following code:

import torch
import torch.nn as nn
import torch.optim as optim
torch.manual_seed(12345)
class predictor(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(predictor, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu1 = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, output_size)
    def forward(self, x): 
        x = self.fc1(x)
        x = self.relu1(x)
        x = self.fc2(x)
        return x
if __name__ == '__main__': 
    ipt = torch.randn((1, 14)).requires_grad_(True)
    lbl = torch.randn((1, 1)).requires_grad_(True)
    model = predictor(input_size=14, hidden_size=32, output_size=1)
    criterion = nn.MSELoss()
    opt = model(ipt)
    loss = criterion(opt, lbl)
    print(loss)
    dy_dx = torch.autograd.grad(loss, model.parameters())
    original_dy_dx = list((_.detach().clone() for _ in dy_dx))
    print(dy_dx)
    cal_loss = dy_dx[-1].detach().clone()[0]
    cal_loss.requires_grad_(True)
    print(cal_loss)
    dummy_data = torch.randn(ipt.size()).requires_grad_(True)
    dummy_label = torch.randn(lbl.size()).requires_grad_(True)
    optimizer = optim.LBFGS([dummy_data, dummy_label], lr=0.1)
    for iters in range(1500):
        def closure():
            optimizer.zero_grad()
            dummy_pred = model(dummy_data)
            dummy_loss = criterion(dummy_pred, dummy_label)
            dummy_dy_dx = torch.autograd.grad(dummy_loss, model.parameters(), create_graph=True) 
            grad_diff = 0
            for i in range(len(dummy_dy_dx)):
                grad_diff += ((dummy_dy_dx[i] - original_dy_dx[i]) ** 2).sum()
            grad_diff.backward()
            return grad_diff
        optimizer.step(closure)
        if iters % 10 == 0:
            current_loss = closure()
            print(current_loss)
            print(iters, "%.4f" % current_loss.item())
    print(ipt)
    print(dummy_data)
    print(lbl)
    print(dummy_label)

The result as follow:

The data was almostly recovered.
The model code is:

class predictor(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(predictor, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu1 = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, output_size)
    def forward(self, x): 
        x = self.fc1(x)
        x = self.relu1(x)
        x = self.fc2(x)
        return x

I test the model with 2 ReLU layers:

class predictor(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(predictor, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.relu1 = nn.ReLU()
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.relu2 = nn.ReLU()
        self.fc3 = nn.Linear(hidden_size, output_size)
    def forward(self, x): 
        x = self.fc1(x)
        x = self.relu1(x)
        x = self.fc2(x)
        x = self.relu2(x)
        x = self.fc3(x)
        return x

Here is the result:

The result was only slightly worse.
The paper replaces the ReLU function with sigmoid function and gets a good result. So, I try to use sigmoid function to improve the result. Here is the code:

class predictor(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(predictor, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.sigmoid1 = nn.Sigmoid()
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.sigmoid2 = nn.Sigmoid()
        self.fc3 = nn.Linear(hidden_size, output_size)
    def forward(self, x): 
        x = self.fc1(x)
        x = self.sigmoid1(x)
        x = self.fc2(x)
        x = self.sigmoid2(x)
        x = self.fc3(x)
        return x

and here is the result:

The result got worse.
So I don't think the non-twice-differentiable function lead to a worse result. When the DLG algorithm is optimizing, it's not optimizing weights, it's optimizing dummy_data and dummy_lable. So the second order derivative is d(dL/dW)/ddummy_data and d(dL/dW)/ddummy_label, not d(dL/dW)/dW.
Looking forward to your reply. :-)

Code has a few bugs

Hey, l find that this study is very interesting and helpful.

So I am trying to run your code but find some bugs.

In main.py:

Line 48,

from model.vision import LeNet

Should be:

from models.vision import LeNet
Line 51,

net.apply(weights_init)

the name 'weights_init' is not defined.

Looking forward to your reply.

I can not recover the image from the gradients.

Why can't I reproduce the effect of the paper? According to your code, the image I get is just some disordered pixel blocks, even if I increase the number of iterations.

grad_diff.backward()

On line 93 in main.py, when doing the back propagation, did you freeze the model parameters so that it would only update the dummy inputs?