Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

explain() fails uninformatively if exact argument is specified, on cv.glmnet model objects #74

Open
nimirea opened this issue Dec 6, 2023 · 3 comments

Comments

@nimirea
Copy link

nimirea commented Dec 6, 2023

When trying to use fastshap with a cv.glmnet model, specifying exact argument to fastshap::explain() causes the function to fail, no matter what the supplied value of exact is. The error returned is not very helpful.

Here's the documentation on exact (emphasis mine):

Logical indicating whether to compute exact Shapley values. Currently only available for stats::lm(), xgboost::xgboost(), and lightgbm::lightgbm() objects. Default is FALSE. Note that setting exact = TRUE will return explanations for each of the stats::terms() in an stats::lm() object. Default is FALSE.

Based on this, I would expect the following behavior when calling fastshap::explain() with an explicit exact argument:

  • fastshap::explain(..., exact = FALSE) should produce the default behavior (i.e., same as calling without explicitly setting exact), since the documentation states the default value for exact is FALSE.
  • fastshap::explain(..., exact = TRUE) should:
    • produce exact Shapley values for stats::lm(), xgboost::xgboost(), and lightgbm::lightgbm() objects
    • fail with an informative error otherwise (e.g., "You have set exact = TRUE for an object type that does not support this functionality, please see ?fastshap::explain() for more details")

Instead, both cases yielded this error:

Error in `colnames<-`(`*tmp*`, value = feature_names) :
  attempt to set 'colnames' on an object with less than two dimensions

The function only works when exact is left unspecified.

Here's a reprex (kudos to @DavZim for the example this is based on):

library(fastshap)
library(AmesHousing)
library(glmnet)

data <- AmesHousing::make_ames()

y <- data$Sale_Price
X <- as.matrix(data[, c("Lot_Frontage", "Lot_Area", "Year_Sold", "Year_Built")])

glm <- cv.glmnet(X, y)
pred_wrapper_fun <- function(mdl, newdata) as.numeric(predict(mdl, newdata, s = glm$lambda.min))

# This works:
expl_glm <- fastshap::explain(
  object = glm,
  X = X,
  pred_wrapper = pred_wrapper_fun
)

# This doesn't work:
expl_glm <- fastshap::explain(
  object = glm,
  X = X,
  pred_wrapper = pred_wrapper_fun,
  exact = FALSE
)
# Error in `colnames<-`(`*tmp*`, value = feature_names) :
#  attempt to set 'colnames' on an object with less than two dimensions

# This also doesn't work:
expl_glm <- fastshap::explain(
  object = glm,
  X = X,
  pred_wrapper = pred_wrapper_fun,
  exact = TRUE
)
# Error in `colnames<-`(`*tmp*`, value = feature_names) :
#  attempt to set 'colnames' on an object with less than two dimensions

# This also doesn't work:
expl_glm <- fastshap::explain(
  object = glm,
  X = X,
  pred_wrapper = pred_wrapper_fun,
  exact = NULL
)
# Error in `colnames<-`(`*tmp*`, value = feature_names) :
#  attempt to set 'colnames' on an object with less than two dimensions

I am not sure whether this happens with other types of models for which exact = TRUE is not supported.

@nimirea nimirea changed the title explain() fails uninformatively if exact argument is specified, on glmnet model objects explain() fails uninformatively if exact argument is specified, on cv.glmnet model objects Dec 6, 2023
@bgreenwell
Copy link
Owner

bgreenwell commented Dec 20, 2023

Hi @nimirea, thanks for the detailed example. I should be able to look at the source code and push a fix shortly after the holiday!

In the meantime, which version of fastshap are you using?

@nimirea
Copy link
Author

nimirea commented Dec 20, 2023

Thanks! Version is 4.3.0.

@bgreenwell
Copy link
Owner

Hi @nimirea, is that the R version? Let me know which fastshap version you have installed when you can.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants