Giter VIP home page Giter VIP logo

lipschitz-sll-networks's People

Contributors

araujoalexandre avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar

Forkers

hukkai

lipschitz-sll-networks's Issues

Checkpoint Loading Error Missing key(s) in state_dict: "model.model.1....

I get this error while trying to load_state_dict(checkpoint)

RuntimeError: Error(s) in loading state_dict for NormalizedModel:
	Missing key(s) in state_dict: "model.model.1.kernel", "model.model.1.bias", "model.model.1.q", "model.model.2.kernel", "model.model.2.bias", "model.model.2.q", "model.model.3.kernel", "model.model.3.bias", "model.model.3.q", "model.model.4.kernel", "model.model.4.bias", "model.model.4.q", "model.model.5.kernel", "model.model.5.bias", "model.model.5.q", "model.model.6.kernel", "model.model.6.bias", "model.model.6.q", "model.model.7.kernel", "model.model.7.bias", "model.model.7.q", "model.model.8.kernel", "model.model.8.bias", "model.model.8.q", "model.model.9.kernel", "model.model.9.bias", "model.model.9.q", "model.model.10.kernel", "model.model.10.bias", "model.model.10.q", "model.model.11.kernel", "model.model.11.bias", "model.model.11.q", "model.model.12.kernel", "model.model.12.bias", "model.model.12.q", "model.model.13.kernel", "model.model.13.bias", "model.model.13.q", "model.model.14.kernel", "model.model.14.bias", "model.model.14.q", "model.model.15.kernel", "model.model.15.bias", "model.model.15.q", "model.model.16.kernel", "model.model.16.bias", "model.model.16.q", "model.model.17.kernel", "model.model.17.bias", "model.model.17.q", "model.model.18.kernel", "model.model.18.bias", "model.model.18.q", "model.model.19.kernel", "model.model.19.bias", "model.model.19.q", "model.model.20.kernel", "model.model.20.bias", "model.model.20.q", "model.model.23.weight", "model.model.23.bias", "model.model.23.q", "model.model.24.weight", "model.model.24.bias", "model.model.24.q", "model.model.25.weight", "model.model.25.bias", "model.model.25.q", "model.model.26.weight", "model.model.26.bias", "model.model.26.q", "model.model.27.weight", "model.model.27.bias", "model.model.27.q", "model.model.28.weight", "model.model.28.bias", "model.model.28.q", "model.model.29.weight", "model.model.29.bias", "model.model.29.q", "model.model.30.weight", "model.model.30.bias". 
[...]

Any idea how to fix this?

SLL with --last_layer 'lln' is not 1-Lipschitz

Dear Alexandre,

I am trying to train the SLL models on the cifar100 dataset with the option --last_layer 'lln'.

The learned model, which achieves similar performance in the paper, is not 1-Lipschitz. By searching the empirical lower Lipschitz bound

L=norm(model(x+dx)-model(x))/(norm(dx)+1e-6)

with PGD-like algorithm, I obtain L=1.8. I guess the possible reason might be that although Q in the output layer has unit row vectors, its singular value could be still larger than 1.

self.Q = F.normalize(self.weight, p=2, dim=1)

Best regards
Ray

--
Ruigang Wang
[email protected]

Postdoc, Australian Centre for Robotics (ACFR),
University of Sydney, Australia.

Flipped W^T W and W W^T operation

Hello,

I noticed that in the paper, you have it annotated that the matrix T should be computed as

$T_{ii} = \sum^n_{j=1}\left|W^TW\right|_{ij} \frac{q_j}{q_i}$

t = torch.abs(torch.einsum('i,ik,kj,j -> ij', q_inv, self.weight.T, self.weight, q)).sum(1)

while in your implementation you have it so that

t = torch.abs(torch.einsum('i,ik,kj,j -> ij', q_inv, self.weight, self.weight.T, q)).sum(1)

which would be equivalent to

$T_{ii} = \sum^n_{j=1}\left|WW^T\right|_{ij} \frac{q_j}{q_i}$

is this an error?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.