Fix Bugs in chat UI

[terhechte]

The current version of the chat ui doesn't work because of wrong datatype assumptions. With these changes, running, e.g. Pixtral via python -m mlx_vlm.chat_ui --model mlx-community/pixtral-12b-4bit works again (on my system).

Llama vision still fails with a different error (ValueError: aspect_ratio_idsmust be provided ifpixel_values is provided) but Pixtral works.

[Blaizzy]

Thanks a lot @terhechte!

You can fix the llama-vision error by changing utils.py line 972 to:

kwargs = {
        k: v
        for k, v in zip(
            [
                "image_grid_thw",
                "image_sizes",
                "aspect_ratio_ids",
                "aspect_ratio_mask",
                "cross_attention_mask",
            ],
            inputs[3:],
        )
    }

[Blaizzy]

Could you run pre-commit run --all to fix the tests?

[terhechte]

Done

[andimarafioti]

This improved it for me. Still some other bugs but this worked

[Blaizzy]

Awesome!

Could you please open an issue for the other bugs?