[v3-dev] Define a directory structure for the jupyter_ai package

[dlqqq]

Problem

The Python code under packages/jupyter-ai/jupyter_ai lacks a directory structure. The location of each file was just determined by whoever added it first.

This brings about a few minor issues for contributors:

1. A disorganized directory structure makes it hard to find specific source files.
2. A disorganized directory structure makes it easy to create circular dependencies.
3. All of the unit tests are being placed under a single test/ directory, which is often far removed from the source code it is testing.

Proposed Solution

Define a directory structure under the "top level", which we use to denote packages/jupyter-ai/jupyter_ai:

• Direct children of the top level should only be directories that contain source files.
• Each direct child should contain an __init__.py file that exports everything from within that directory.
• Each direct child may contain a README.md file that explains the contents of that directory & their purpose.
• Unit tests should be placed only within direct children.

For example, if we follow this structure, config_manager.py should be broken down into 4 files under a new config_manager/ directory:

1. config_manager/config_manager.py: Implements the config manager.
2. config_manager/__init__.py: Exports the config manager.
3. config_manager/tests/: Contains unit tests for the config manager, e.g. test_config_manager.py.
4. config_manager/README.md: Explain the contents of this directory to developers.

Additional context

This can be documented in the contributor docs for v3, but I don't think that's really a priority / something that needs to be tracked. The directory structure proposed here was designed to be fairly intuitive & obvious for other contributors to follow.