feat: enhance multimodal support for images and audio in instructor

[jxnl]

Important

Enhance multimodal support in instructor by improving image and audio handling, and updating content conversion functions for better compatibility with AI models.

• Multimodal Enhancements:
  • Update Image class to support autodetection of image sources from URLs, file paths, and base64 data.
  • Add autodetect_safely() method to Image class for safe image detection.
  • Improve Audio class to handle audio from URLs and file paths, ensuring WAV format.
• Content Conversion:
  • Enhance convert_contents() function to handle Image and Audio objects based on mode.
  • Update convert_messages() to support autodetection of images and conversion of content for different modes.
• Miscellaneous:
  • Add caching to from_url() and from_path() methods in Image class.
  • Refactor url_to_base64() to cache base64 encoding of image URLs.

^{This description was created by for f9d95c8. It will automatically update as commits are pushed.}

[cloudflare-workers-and-pages]

Deploying instructor-py with    Cloudflare Pages

Latest commit:	f9d95c8
Status:	✅  Deploy successful!
Preview URL:	https://606e7ac4.instructor-py.pages.dev
Branch Preview URL:	https://doc-lint-2.instructor-py.pages.dev

View logs

[ellipsis-dev]

👍 Looks good to me! Reviewed everything up to f9d95c8 in 1 minute and 18 seconds

More details

• Looked at 2772 lines of code in 65 files
• Skipped 0 files when reviewing.
• Skipped posting 2 drafted comments based on config settings.

1. instructor/multimodal.py:343

• Draft comment:
  Consider using the | operator for type hinting instead of Union for consistency and readability.

    contents: str | dict[str, Any] | Image | Audio | list[str | dict[str, Any] | Image | Audio],

• Reason this comment was not posted:
  Confidence changes required: 10%
  The PR includes multiple instances where the Union type hint can be simplified using the | operator, which is more concise and modern. This change is consistent with the rest of the codebase and improves readability.

2. instructor/multimodal.py:213

• Draft comment:
  Use consistent type hinting for the source attribute. Consider using Union[str, Path] or str | Path consistently throughout the code.
• Reason this comment was not posted:
  Confidence changes required: 80%
  The code uses inconsistent type hinting for the source attribute in the Audio class. It uses str | Path in one place and Union[str, Path] in another. This inconsistency should be addressed for clarity and consistency.

Workflow ID: wflow_8Lx93v1vCi9sDFxO

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.