Giter VIP home page Giter VIP logo

Comments (3)

KaidiXu avatar KaidiXu commented on July 21, 2024 1

Hi @akshaykulkarni07, thanks for sharing the details, I run your model with the config file and actually the results are as expected.

The problem is that the number of unstable neurons in your model is quite large. Although we support running our verification of this model, the output bounds are extremely loose, (as you can see the initial bound can be -1e13, and usually they should be larger than -1,000) 😂

The solution is that you can reduce the model size by decreasing depth and width or train the model by using verified or adversarial training (IBP or CROWN-IBP) to reduce the unstable neurons.

Please let me know if you have any questions!

from alpha-beta-crown.

huanzhang12 avatar huanzhang12 commented on July 21, 2024

Could you please give us the model, configuration file, command, and log files so we can provide more help? Thanks.

from alpha-beta-crown.

akshaykulkarni07 avatar akshaykulkarni07 commented on July 21, 2024
# This is an example configuration file that contains most useful parameter settings.
model:
  name: partial(ResNet18, in_planes=64)
  path: checkpoints/GAT_CIFAR10_ResNet18.pkl
data:
  dataset: CIFAR
  mean: [0.0, 0.0, 0.0]  # Mean for normalization.
  std: [1.0, 1.0, 1.0]  # Std for normalization.
  start: 0  # First example to verify in dataset.
  end: 1000  # Last example to verify in dataset.
specification:
  norm: .inf  # Linf norm (can also be 2 or 1).
  epsilon: 0.00784313725490196  # epsilon=2./255.
attack:  # Currently attack is only implemented for Linf norm.
  pgd_steps: 100  # Increase for a stronger attack. A PGD attack will be used before verification to filter on non-robust data examples.
  pgd_restarts: 30  # Increase for a stronger attack.
solver:
  batch_size: 8  # Number of subdomains to compute in parallel in bound solver. Decrease if you run out of memory.
  crown:
    batch_size: 8
    max_crown_size: 8
  alpha-crown:
    iteration: 100   # Number of iterations for alpha-CROWN optimization. Alpha-CROWN is used to compute all intermediate layer bounds before branch and bound starts.
    lr_alpha: 0.1    # Learning rate for alpha in alpha-CROWN. The default (0.1) is typically ok.
  beta-crown:
    lr_alpha: 0.01  # Learning rate for optimizing the alpha parameters, the default (0.01) is typically ok, but you can try to tune this parameter to get better lower bound.
    lr_beta: 0.05  # Learning rate for optimizing the beta parameters, the default (0.05) is typically ok, but you can try to tune this parameter to get better lower bound.
    iteration: 20  # Number of iterations for beta-CROWN optimization. 20 is often sufficient, 50 or 100 can also be used.
bab:
  timeout: 120  # Timeout threshold for branch and bound. Increase for verifying more points.
  branching:  # Parameters for branching heuristics.
    reduceop: min  # Reduction function for the branching heuristic scores, min or max. Using max can be better on some models.
    method: kfsb  # babsr is fast but less accurate; fsb is slow but most accurate; kfsb is usually a balance.
    candidates: 3  # Number of candidates to consider in fsb and kfsb. More leads to slower but better branching. 3 is typically good enough.

This is the configuration file. The model is the standard ResNet18 and the pretrained checkpoint is from here. Then the command I run is python abcrown.py --config cifar_resnet_GAT.yaml where cifar_resnet_GAT.yaml contains the above provided config.

As for the logs, I have the terminal output saved to a text file but it's very very long (like 2.8 million lines when evaluating on 1000 samples of CIFAR10), so it would be helpful if you could point to any specific output you want to see. Generally, I'm seeing this

Result: unknown in 121.7578 seconds

after every sample gets processed.

from alpha-beta-crown.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.