Models created with different scikit-learn versions break compatibility

[mireianievas]

Can anyone cross-check if generating a clean mamba create installs scikit-learn=1.3.2. In my case it seems to be doing that (despite seeing that in the environment in the main version it is forced 1.2). It results in a non-working lstchain environment

Steps:

mamba create -c conda-forge -n lstchain-v0.10.5 python=3.11 lstchain=0.10.5

Errors I was getting using that version of sklearn through the dl1_to_dl2

/home/mireia.nievas/workspace/software/miniconda/envs/lstchain-v0.10.5/lib/python3.11/site-packages/sklearn/base.py:348: InconsistentVersionWarning: Trying to unpickle estimator DecisionTreeRegressor from version 1.2.2 when using version 1.3.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
Traceback (most recent call last):
  File "/home/mireia.nievas/workspace/software/miniconda/envs/lstchain-v0.10.5/bin/lstchain_dl1_to_dl2", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/mireia.nievas/workspace/software/miniconda/envs/lstchain-v0.10.5/lib/python3.11/site-packages/lstchain/scripts/lstchain_dl1_to_dl2.py", line 270, in main
    models_dict = {'reg_energy': joblib.load(Path(args.path_models, 'reg_energy.sav'))}
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(  ...  )

ValueError: node array from the pickle has an incompatible dtype:
- expected: {'names': ['left_child', 'right_child', 'feature', 'threshold', 'impurity', 'n_node_samples', 'weighted_n_node_samples', 'missing_go_to_left'], 'formats': ['<i8', '<i8', '<i8', '<f8', '<f8', '<i8', '<f8', 'u1'], 'offsets': [0, 8, 16, 24, 32, 40, 48, 56], 'itemsize': 64}
- got     : [('left_child', '<i8'), ('right_child', '<i8'), ('feature', '<i8'), ('threshold', '<f8'), ('impurity', '<f8'), ('n_node_samples', '<i8'), ('weighted_n_node_samples', '<f8')]

[vuillaut]

latest action for python 3.11 installs scikit-learn v1.2.2 as required:
https://github.com/cta-observatory/cta-lstchain/actions/runs/7209210412/job/19639699946

so did v0.10.5:
https://github.com/cta-observatory/cta-lstchain/actions/runs/7059201791/job/19216322629

[mireianievas]

Ok, strange.

[morcuended]

mamba create -c conda-forge -n lstchain-v0.10.5 python=3.11 lstchain=0.10.5

is actually giving me problems in both linux and macos:

Could not solve for environment specs
The following package could not be installed
└─ lstchain 0.10.5**  is uninstallable because it requires
   └─ python-blosc2  , which does not exist (perhaps a missing channel).

[maxnoe]

The package definitely exists:
https://anaconda.org/conda-forge/python-blosc2/files?version=2.3.2

[vuillaut]

docker run -it mambaorg/micromamba
micromamba create -c conda-forge -n lstchain-v0.10.5 python=3.11 lstchain=0.10.5
micromamba activate lstchain-v0.10.5
python -c 'import lstchain; print(lstchain.__version__)'
0.10.5

works fine

[mireianievas]

ok, lets close then.

[morcuended]

The package definitely exists:
https://anaconda.org/conda-forge/python-blosc2/files?version=2.3.2

Just a correction on this. It actually worked fine on linux, but not on macos (my bad!)

[morcuended]

The problem is still there.

setup.py requires: 'scikit-learn~=1.2' which means the latest available among 1.x not 1.2.x, while env yml file installs scikit-learn=1.2 (1.2.2 more precisely). The CI uses the env yml to setup the environment. Later it does pip install ., but since it finds the already install 1.2.2 it does not anything else.

Models created with scikit-learn 1.2.2 cannot be read with newer versions:

In dl1 to dl2 command:

/fefs/aswg/software/conda/envs/foa/lib/python3.9/site-packages/sklearn/base.py:376: InconsistentVersionWarning: Trying to unpickle estimator DecisionTreeRegressor from version 1.2.2 when using version 1.5.2. This might lead to breaking code or invalid results. Use at your own risk. For more info please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations

...

ValueError: node array from the pickle has an incompatible dtype:
- expected: {'names': ['left_child', 'right_child', 'feature', 'threshold', 'impurity', 'n_node_samples', 'weighted_n_node_samples', 'missing_go_to_left'], 'formats': ['<i8', '<i8', '<i8', '<f8', '<f8', '<i8', '<f8', 'u1'], 'offsets': [0, 8, 16, 24, 32, 40, 48, 56], 'itemsize': 64}
- got     : [('left_child', '<i8'), ('right_child', '<i8'), ('feature', '<i8'), ('threshold', '<f8'), ('impurity', '<f8'), ('n_node_samples', '<i8'), ('weighted_n_node_samples', '<f8')]

So I think we should fix the sklearn version in setup.py