The torchtext failures seem unrelated, since they also affect the other PR I just made which changes nothing in the build system (#828). Annoying, though, guess we'll have to wait for a resolution of this.

Oef, waiting for resolution might not work out, torchtext development has been discontinued... pytorch/text#2250

What of torchtext are we using?

Copy-pasting the actual error message for other Googlers encountering this problem:

tests/test_kwargs.py:6: in <module>
    from tests.utils import get_dummy_model_function
tests/utils.py:5: in <module>
    from torchtext.vocab import Vectors
/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/torchtext/__init__.py:18: in <module>
    from torchtext import _extension  # noqa: F401
/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/torchtext/_extension.py:64: in <module>
    _init_extension()
/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/torchtext/_extension.py:58: in _init_extension
    _load_lib("libtorchtext")
/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/torchtext/_extension.py:50: in _load_lib
    torch.ops.load_library(path)
/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/torch/_ops.py:1295: in load_library
    ctypes.CDLL(path)
/opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/ctypes/__init__.py:374: in __init__
    self._handle = _dlopen(self._name, mode)
E   OSError: /opt/hostedtoolcache/Python/3.9.19/x64/lib/python3.9/site-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKSs

Oef, waiting for resolution might not work out, torchtext development has been discontinued... pytorch/text#2250

What of torchtext are we using?

see #837

What of torchtext are we using?

Only Vectors to create a vocabulary with string to integer conversions, and get_tokenizer to instantiate a spacy tokenizer. Maybe we can find that functionality elsewhere. SpaCy may provide both functionalities. It is already a dependency in DIANNA.

Closing and reopening in an attempt to trigger rerunning the checks.

Still many checks not passing. @cwmeijer what to do with this PR?

Rebased on main.

Now we have other failures, which seem to be caused by keras 3, which has dropped access to the (previously private) keras.src module and some other stuff. However, I don't see this being used directly in DIANNA code itself, so the issues are probably in some dependencies that did not get updated and that did not specify upper limits to their keras dependency...

Wow what a can of worms. This is causing many different errors, like a pip solver error on Windows:

The conflict is caused by:
    dianna 1.6.0 depends on keras<3
    dianna[dev] 1.6.0 depends on keras<3
    tensorflow-intel 2.16.1 depends on keras>=3.0.0

On Ubuntu there is no conflict, but the tests error out, so I'm guessing there's still an incompatible set of packages getting installed.

If nobody else has ideas feel free to close this PR, it is not very urgent for me, just a nice to have.

Wow what a can of worms. This is causing many different errors, like a pip solver error on Windows:

The conflict is caused by:
    dianna 1.6.0 depends on keras<3
    dianna[dev] 1.6.0 depends on keras<3
    tensorflow-intel 2.16.1 depends on keras>=3.0.0

On Ubuntu there is no conflict, but the tests error out, so I'm guessing there's still an incompatible set of packages getting installed.

If nobody else has ideas feel free to close this PR, it is not very urgent for me, just nice to have.

@cwmeijer what do you think - shall we close it? It's pity not to use the elegant solution, but if it causes many problems...

I agree this would be nice to solve, but I expect this to take quite some time to solve. I propose to not solve this right now.