Named entity extractor private models

[mpangrazzi]

Related Issues

• fixes NamedEntityExtractor not usable with private models #8633

Proposed Changes:

• I've added token support to NamedEntityExtractor and then update HF backend to pass it down to tokenizer and model from_pretrained methods.
• I've duplicated public dslim/bert-base-NER to private deepset/bert-base-NER and added required tests.
• I've also fixed formatting of an existing error message about failed backend initialization.

How did you test it?

Local unit testing / e2e

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[coveralls]

Pull Request Test Coverage Report for Build 12429166430

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 122 unchanged lines in 8 files lost coverage.
• Overall coverage increased (+0.03%) to 90.687%

---

Files with Coverage Reduction	New Missed Lines	%
components/preprocessors/document_splitter.py	1	99.51%
components/generators/chat/hugging_face_api.py	4	96.4%
components/preprocessors/nltk_document_splitter.py	4	96.72%
utils/hf.py	7	86.29%
utils/type_serialization.py	7	89.04%
components/generators/chat/hugging_face_local.py	12	85.32%
core/pipeline/base.py	32	88.64%
components/extractors/named_entity_extractor.py	55	68.28%

Totals
Change from base Build 12392158409:	0.03%
Covered Lines:	8374
Relevant Lines:	9234

---

💛 - Coveralls

[julian-risch]

The code changes look quite good to me already. We only need to update from_dict/to_dict methods and I have a question about one test case and the comment in there, which I find confusing. The test won't and shouldn't fail if the HF_API_TOKEN is not set. So let's simply remove the comment. Otherwise that would mean we are trying to download the model and in that case the test should be marked with

    @pytest.mark.integration
    @pytest.mark.skipif(
        not os.environ.get("HF_API_TOKEN", None),
        reason="Export an env var called HF_API_TOKEN containing the Hugging Face token to run this test.",
    )

For how to update from_dict/to_dict you can have a look at other components, for example HuggingFaceLocalGenerator. We should also update the tests to have a test_to_dict_default and test_to_dict_with_parameters. The tests with ...no_default_parameters_hf should use monkeypatch.delenv("HF_API_TOKEN", raising=False)

[julian-risch]

One more question and then we're ready to merge.

[julian-risch]

Could you please briefly explain this change? Before we explicitly set task to "text-generation" in pipeline_kwargs and initialize the component with task="text2text-generation" to then check if the pipeline_kwargs override the init parameter, right?

[mpangrazzi]

reverted as discussed

[julian-risch]

LGTM! 👍