Validate Pydantic imports

[dlqqq]

Description

Adds a GH Actions job that fails if any of our packages import from pydantic directly.

Context

(cc @JasonWeill)

While reviewing @krassowski 's work, I noticed that both #398 and #465 were importing from pydantic directly. To be compatible with Pydantic v1 and v2, we need to pick an API to use and make sure the way we are using it works regardless of whether the user's environment contains Pydantic v1 or v2. Currently, we do so by importing from langchain.pydantic_v1, which exposes the v1 API in both cases. We do not want to import from pydantic directly as that will resolve to Pydantic v1 or v2, depending on which is installed in the user's environment.

[dlqqq]

Weird, I'm getting an unrelated error that I've never seen before. Seems like the build environment is missing the latest attr or attrs package? I'll keep looking into this.

Full logs: https://github.com/jupyterlab/jupyter-ai/actions/runs/7333155619/job/19968249161?pr=546

Excerpt:

@jupyter-ai/core: Building wheels for collected packages: jupyter_ai
@jupyter-ai/core:   Building wheel for jupyter_ai (pyproject.toml): started
@jupyter-ai/core:   Building wheel for jupyter_ai (pyproject.toml): finished with status 'error'
@jupyter-ai/core:   error: subprocess-exited-with-error
@jupyter-ai/core:   
@jupyter-ai/core:   × Building wheel for jupyter_ai (pyproject.toml) did not run successfully.
@jupyter-ai/core:   │ exit code: 1
@jupyter-ai/core:   ╰─> [49 lines of output]
@jupyter-ai/core:       INFO:hatch_jupyter_builder.utils:Running jupyter-builder
@jupyter-ai/core:       INFO:hatch_jupyter_builder.utils:Building with hatch_jupyter_builder.npm_builder
@jupyter-ai/core:       INFO:hatch_jupyter_builder.utils:With kwargs: {'build_cmd': 'build:prod', 'npm': ['jlpm']}
@jupyter-ai/core:       INFO:hatch_jupyter_builder.utils:Installing build dependencies with npm.  This may take a while...
@jupyter-ai/core:       INFO:hatch_jupyter_builder.utils:> /home/runner/.local/bin/jlpm install
@jupyter-ai/core:       Traceback (most recent call last):
@jupyter-ai/core:         File "/home/runner/.local/bin/jlpm", line 5, in <module>
@jupyter-ai/core:           from jupyterlab.jlpmapp import main
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/jupyterlab/__init__.py", line 8, in <module>
@jupyter-ai/core:           from .handlers.announcements import (
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/jupyterlab/handlers/announcements.py", line 14, in <module>
@jupyter-ai/core:           from jupyter_server.base.handlers import APIHandler
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/jupyter_server/base/handlers.py", line 23, in <module>
@jupyter-ai/core:           from jupyter_events import EventLogger
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/jupyter_events/__init__.py", line 3, in <module>
@jupyter-ai/core:           from .logger import EVENTS_METADATA_VERSION, EventLogger
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/jupyter_events/logger.py", line 19, in <module>
@jupyter-ai/core:           from .schema import SchemaType
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/jupyter_events/schema.py", line 9, in <module>
@jupyter-ai/core:           from referencing import Registry
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/referencing/__init__.py", line 4, in <module>
@jupyter-ai/core:           from referencing._core import Anchor, Registry, Resource, Specification
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/referencing/_core.py", line 8, in <module>
@jupyter-ai/core:           from attrs import evolve, field
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/attrs/__init__.py", line 3, in <module>
@jupyter-ai/core:           from attr import (
@jupyter-ai/core:       ImportError: cannot import name 'AttrsInstance' from 'attr' (/usr/lib/python3/dist-packages/attr/__init__.py)
@jupyter-ai/core:       Traceback (most recent call last):
@jupyter-ai/core:         File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
@jupyter-ai/core:           main()
@jupyter-ai/core:         File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line [345](https://github.com/jupyterlab/jupyter-ai/actions/runs/7332965086/job/19967849314#step:6:349), in main
@jupyter-ai/core:           json_out['return_val'] = hook(**hook_input['kwargs'])
@jupyter-ai/core:         File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
@jupyter-ai/core:           return _build_backend().build_wheel(wheel_directory, config_settings,
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/hatchling/build.py", line 58, in build_wheel
@jupyter-ai/core:           return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard'])))
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/overlay/local/lib/python3.10/dist-packages/hatchling/builders/plugin/interface.py", line 147, in build
@jupyter-ai/core:           build_hook.initialize(version, build_data)
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/normal/local/lib/python3.10/dist-packages/hatch_jupyter_builder/plugin.py", line 87, in initialize
@jupyter-ai/core:           raise e
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/normal/local/lib/python3.10/dist-packages/hatch_jupyter_builder/plugin.py", line 82, in initialize
@jupyter-ai/core:           build_func(self.target_name, version, **build_kwargs)
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/normal/local/lib/python3.10/dist-packages/hatch_jupyter_builder/utils.py", line 113, in npm_builder
@jupyter-ai/core:           run([*npm_cmd, "install"], cwd=str(abs_path))
@jupyter-ai/core:         File "/tmp/pip-build-env-lu2b97wl/normal/local/lib/python3.10/dist-packages/hatch_jupyter_builder/utils.py", line 229, in run
@jupyter-ai/core:           return subprocess.check_call(cmd, **kwargs)
@jupyter-ai/core:         File "/usr/lib/python3.10/subprocess.py", line [369](https://github.com/jupyterlab/jupyter-ai/actions/runs/7332965086/job/19967849314#step:6:373), in check_call
@jupyter-ai/core:           raise CalledProcessError(retcode, cmd)
@jupyter-ai/core:       subprocess.CalledProcessError: Command '['/home/runner/.local/bin/jlpm', 'install']' returned non-zero exit status 1.
@jupyter-ai/core:       [end of output]

[dlqqq]

I found a related issue: python-attrs/attrs#1210

For some reason, attr is resolving to the package on the system (e.g. ones installed via an OS's package manager rather than pip) instead of the one in the same environment as attrs. 🤔 💭

[dlqqq]

I had to wrap everything in a virtual environment, and now the job fails correctly, reporting the import added in #398 on main.

Full logs: https://github.com/jupyterlab/jupyter-ai/actions/runs/7333298903/job/19968577119?pr=546

[dlqqq]

Pushed the commit to fix the pydantic import.

I also had to follow this, since I renamed the other lint job: https://github.com/orgs/community/discussions/26698

[krassowski]

Happy to see more linting for conventions 🎉

On some loosely related things which may or may not deserve their own tickets:

• at some point it would be worth also cleaning up unused imports which are many in some places, and possibly using __all__ to restrict public API surface.
• another thing you may want to automate/document is the "rebase-centric" approach that seems to be in use in this repo.

[dlqqq]

@krassowski

Yup, these are all great suggestions! We're doing the best we can with the resources we have to spare. Feel free to open issues for any suggestions you have. For discussion on more minor topics that may not warrant their own issue, you can email me directly. I can then establish some kind of communication channel between you and our team. This seems like a good idea given how much you've been contributing lately.