long vectors (argument 7) are not supported

[ndeluigi]

Dear @samFarrellDay,

I'm running my usual gap filling code on a dataset that is increasing in size every year and this time I'm currently experiencing a new error:

dataset 1
iteration 1 | ConduScmError in miceRanger(metalp_ds, m = 1, maxiter = 10, parallel = FALSE, :
Evaluation failed with error <Error in get.knnx(data, query, k, algorithm): long vectors (argument 7) are not supported in .C
. This is probably our fault - please open an issue at https://github.com/FarrellDay/miceRanger/issues with a reproduceable example.

The dataset has 1'176'481 observations and 13 variables. All variables are numeric and I tried with both NaN and NA for missing values. The error shows up when the second variables starts to be imputed. My computer and 56 cores and 256GB RAM

I did some tests and it seems it's a matter of size. I'm able to impute up to approx 800'000 observations, but with my full dataset that error shows up. I could subsample it to generate a model and apply that same model to the rest of the observations, but I would like to exploit the full information of the DS to generate the model.

Any idea how to fix it? Many thanks