Hey @kevinmessiaen! This seems to be a duplicate of #1847
PS: I'd love to contribute to the tool! I'll be on the lookout for new issues/improvements :)

@pranavm7 Hey, it's not exactly the same. One is for numerical values and the other is for categorical ones which differs a bit.

We would be happy to have you contribute on this tool, do you have any improvement ideas in mind?

@kevinmessiaen I would like to work on this if this is still open.

@Kranium2002 Sure we appreciate that, I assigned you to the issue. Let me know if you have some questions or need some help!

I would be working on adding a numerical perturbation detector to test model robustness by tweaking numerical features and seeing how much the model's predictions shift by around 1 %. For classification models, it'll flag cases where the predicted label changes, and for regression, it'll detect when predictions differ beyond a threshold (like 5%). I'll integrate this into the existing framework so it reports any significant sensitivity issues. Plus, I'll build out tests to ensure it's flexible across model types and datasets.

PS: Do I make the thresholds of 1 and 5 % editable by the user or do I keep them fixed?
Your thoughts? @kevinmessiaen

Working on this in #2040