table size is not a good predictor of algo performance in vegTypeMapGenerator

[CeresBarros]

I noticed that for a table with 1340184 rows, turning assertions off in vegTypeMapGenerator, makes the function use algo == 2 :

Browse[2]>   nrowCohortData
[1] 1340184

Browse[2]>   ## use new vs old algorithm based on size of x. new one (2) is faster in most cases.
Browse[2]>   ## enable assertions to view timings for each algorithm before deciding which to use.
Browse[2]>   algo <- ifelse(nrowCohortData > 3.5e6, 1, 2)
Browse[2]> algo
[1] 2

However, I ran the function a few times with assertions on to compare timings of the two algos and no. 2 (the 'new algo') was at least 5xs slower than no. 1 (the 'old algo'). So table size doesn't seem to be a good predictor...

LandR::vegTypeMapGenerator: new algo 10.1584620475769
LandR::vegTypeMapGenerator: old algo 2.12155604362488

LandR::vegTypeMapGenerator: new algo 5.77787399291992
LandR::vegTypeMapGenerator: old algo 0.236522197723389

This is a minor issue, but I wanted to raise it here because I feel we're not ready to decide on which algorithm to use and if we keep the mechanism of "let the function decide" we'll need a better predictor.

[eliotmcintire]

Maybe just always use algo 1, which is the simpler data table one. It is possible that the data.table one got faster.

[CeresBarros]

At this point, I believe the user can't control which algo to use, but that could be added easily. Is this what you meant? or actually remove algo == 2 altogether?

[achubaty]

re-upping. new algo is faster for smaller tables but mid-run for LandWeb I see the old algo is faster

[eliotmcintire]

I think then we should just remove the newer one and take the older one that is better on large tables. Fast when something is fast is good, but fast when something is slow is better.

…

On Mon, Oct 24, 2022 at 3:44 PM Alex Chubaty ***@***.***> wrote: re-upping. new algo is faster for smaller tables but mid-run for LandWeb is see the old algo is faster — Reply to this email directly, view it on GitHub <#39 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIMVW43OK5V22VK4DMZWBLWE4GMXANCNFSM4NIVBFIA> . You are receiving this because you commented.Message ID: ***@***.***>