Conftest does not support returning details of a PASS conftest output

[ssodhi-intuit]

We are using conftest evaluator for evaluating infrastructure code against some set of rules. From auditing standpoint it would be useful to not only display violations but also display which resource is passing the set of rules.

Is there any guidance on how it can be supported in conftest out of the box, if not is there guidance for library consumer on how to do above mentioned without having to explicitly write another rule for non violation?

[jalseth]

Edit: Apologies I misread the question, my previous comment wasn't relevant.

Showing the "successes" without additional rules isn't possible. Rego is a query language, so the only thing Conftest gets back from the OPA engine are the matches for the deny rules. You may also want to read #731 (comment) for related (slightly different request) previous discussion on this.

[ssodhi-intuit]

Appreciate your input. We are also looking into other open source tools, and found that regula is solving this problem by implementing a wrapper rego code that evaluates the object and populates PASS in addition to any FAILURE.
Reference:

1. Regula github
2. Wrapper code

Wondering if this is something you can support as well?

[boranx]

here are my 2 cents:
We could do that as an extra step when the query is evaluated: https://github.com/open-policy-agent/conftest/blob/master/policy/engine.go#L444
If we compare the input with resultSet, and get a diff to see what tests are not evaluated(passing- aka nomatch), then we'd be able to check their locations as well
However, this would be an overhead as it wouldn't have any effect on the outcome(query evaluation) besides pretty printing of the success scenarios

[apratinav-intuit]

@jalseth @boranx Thanks for your input.

We did some more study and wanted to check if you see any issues in supporting Allow rule in Conftest? Right now, we do see Conftest has support for Violation/Warn/Deny rules. Having support for Allow rule would give control back to policy author and write more rules that would let them show non-violating (or compliant) resources. Of course this would mean some additional changes in the code like adding support for allow rule, the way success count is populated, display allowed resource details in different conftest supported format (like json.. so on) etc.

We wanted to hear your thoughts and check that you would be open to taking that as PR contribution to this repo.

Looking forward to hearing from you.

Thanks
Akshay

[boranx]

do you think would #584 help to address the need or you think "allow" would need to be introduced anyway?
I'd like to loop @jpreese too as he might have better insights overall