hyperparamter_search not running over all folds?

[sebastianrosengren]

for loop running 1:1

for (i in 1:1) {
  dtrain <- xgboost::xgb.DMatrix(train_feat[[i]]$data)
  attr(dtrain, "errors") <- train_feat[[i]]$errors

  bst <- xgboost::xgb.train(param, dtrain, nrounds)
  preds <- M4metalearning::predict_selection_ensemble(bst, test_feat[[i]]$data)
  er <- M4metalearning::summary_performance(preds,
                                            test_ds[[i]],
                                            print.summary = FALSE)

  final_error <- c(final_error, er$weighted_error)
  final_preds <- rbind(final_preds, preds)
}

Should maybe be 1.length(folds) ?

[pmontman]

Hi Sebastian, you are right, the loop could run for all the folds.
It is done that way for computational reasons. It is doing a holdout validation (so just an approximation anyways) that runs inside a bayesian optimization loop. So you can also modify the number of rounds of bayesian optimization if you want to find a better solution.

[sebastianrosengren]

Hi,

Sorry for the late appreciation post. Thanks for taking the time and clarifying this.

Much appreciated!