Redundant testing/ouput in versions >= 0.7.0 about conftest HOT 9 CLOSED

Confirmed, I think this is an unintended side effect of #34.

Adding an additional deny causes the failures to duplicate.

Pinging @brendanjryan in case the fix is obvious. Otherwise I'll try and grab some time to look at this tomorrow as well.

You might be able to see the problem in the trace I think, run contest test --trace, for example

from conftest.

I have a fix. Ran out of time this evening I think to finish off, but wanted to post. Will finish up tomorrow and release a bug fix.

func getRules(ctx context.Context, re *regexp.Regexp, compiler *ast.Compiler) ([]string) {

     var res []string

     for _, m := range compiler.Modules {
         for _, r := range m.Rules {
             n := r.Head.Name.String()
             if re.MatchString(n) {
                 // the same rule names can be used multiple times, but
                 // we only want to run the query and report results once
                 if !stringInSlice(n, res) {
                     res = append(res, n)
                 }
             }
         }
     }

     return res
 }

 func stringInSlice(a string, list []string) bool {
     for _, b := range list {
         if b == a {
             return true
         }
     }
     return false
 }

from conftest.

Sounds like a bug. Could you include an example and some output? Will look into.

from conftest.

Hi, I'm working with 0.9.0 and get a similiar Output.
I'm testing a openshift route object using helm:

apiVersion: route.openshift.io/v1
 kind: Route
 metadata:
  labels:
    app: {{ .Chart.Name }}
  name: {{ .Chart.Name }}
spec:
  host: foobar
  tls:
    termination: Edge
  to:
    kind: Service
    name: {{ .Chart.Name }}
    weight: 100
  wildcardPolicy: None

My simple policy:

package main

deny[msg] {
  input.kind = "Route"
  name = input.metadata.name
  not input.spec.tls.termination
  msg = sprintf("%s/%s has to be secure", [input.kind,name])
}

deny[msg] {
  input.kind = "Route"
  name = input.metadata.name
  input.spec.host
  msg = sprintf("%s/%s can not contain the \"spec.host\" attribute", [input.kind,name])
}

Just the second policy should fail, the first is good, but the output is the following:

[user@rhel7 helm]$ helm template . | conftest test -
   Route/infra-test can not contain the "spec.host" attribute
   Route/infra-test can not contain the "spec.host" attribute

Once I delete the not failing policy it's working like it should:

[user@rhel7 helm]$ helm template . | conftest test -
   Route/infra-test can not contain the "spec.host" attribute

from conftest.

My experience is similar to @mKamleiter's, though I hadn't yet made the connection about failing/passing policies. I think we're on to something here...

from conftest.

ah good catch @garethr! This looks good to me!

from conftest.

Awesome, thanks! I'll stand by to test against my policies.

from conftest.

Should now be fixed in master and I'm releasing a 0.9.1 with the fix. Let me know if that doesn't resolve your issue.

from conftest.

All good for me in 0.10.0. Thanks!

from conftest.