Reducing Validation Error Noise from oneOf

[b-wils]

I've been running a bunch of Lotties through schema validation and debugging errors. One problem I'm encountering is that there is a lot of error noise. One of the biggest sources of noise is the usage of oneOf, particularly with shapes and layers. This is because if it fails for any reason, it will try to validate every shape/layer in the schema and produce an error for each one.

I've been exploring some possibilities for improving this-

1. Add discriminators to the schema to have validators only check schema that matches the type discriminator. This is not a part of JSON schema spec but is a part of Open API spec and has some support from popular validators. This field can be safely ignored by validators that don't have support. One notable downside of this is it can only use strings as discriminators, so we wouldn't be able to apply this to layers as their type is a number.

2. Migrate oneOf to if/then statements (WIP example). This works quite well to narrow the errors down to only the relevant layers. However, these changes are somewhat invasive, make the schema harder to read and would need to update our doc generation to handle this new format.

3. Post-process validator output to reduce errors. I did some exploring here and the validator output may not have enough information to do this. This work is not portable either and would need to be written for every language where it is needed.

None of these options are ideal but wanted to get people's thoughts and if there are any other ideas. We could also accept this issue since spec errors are likely an issue with the authoring tool and something they may be better suited to investigate the errors despite noise.

[mbasaglia]

Solved by the smart validator, basically it dynamically patches the schema to replace all oneOf instances with a custom rule