You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After discussions with @VicTrqt, @SurgeArrester, @yqdleiyi and others, we think it would be a good idea to reproduce these results a few years on with the latest additions to MODNet. I'm raising this issue to keep me honest and actually do it...
Whilst there have not been huge developments in MODNet since our first submission, we have been actively using it in "production" and need to investigate:
how well our new, fast subset of matminer works on the original tasks
how well the GA hyperparameter opt works (I think the defaults will need to be updated for the larger tasks here)
generally how easily these tasks can be performed via the MODNet API now
Hopefully this will lead to improvements in MODNet generally.
I'm not sure if matbench is still being actively maintained, but we could even consider a new submission, depending on the results. There have been some queries raised about the leaderboard results that we should also keep in mind, e.g., materialsproject/matbench#262. I think we're in a better place to really share precisely the scripts used to run these benchmarks this time around, as before they were mostly tied with figure generation for our benchmarking paper, with matbench as an afterthought.
The text was updated successfully, but these errors were encountered:
After discussions with @VicTrqt, @SurgeArrester, @yqdleiyi and others, we think it would be a good idea to reproduce these results a few years on with the latest additions to MODNet. I'm raising this issue to keep me honest and actually do it...
Whilst there have not been huge developments in MODNet since our first submission, we have been actively using it in "production" and need to investigate:
Hopefully this will lead to improvements in MODNet generally.
I'm not sure if matbench is still being actively maintained, but we could even consider a new submission, depending on the results. There have been some queries raised about the leaderboard results that we should also keep in mind, e.g., materialsproject/matbench#262. I think we're in a better place to really share precisely the scripts used to run these benchmarks this time around, as before they were mostly tied with figure generation for our benchmarking paper, with matbench as an afterthought.
The text was updated successfully, but these errors were encountered: