MLXChatExample: fix VLM image handling on iOS (PhotosPicker, EXIF, empty assistant trim)#472
Merged
Merged
Conversation
- Use custom Transferable types (PickedImage/PickedVideo) with explicit content types instead of generic Data.self for reliable PhotosPicker loading - Normalize image orientation via UIGraphicsImageRenderer before saving as JPEG, since CIImage(contentsOf:) does not apply EXIF orientation - Exclude trailing empty assistant message from model input so the chat template leaves the assistant turn open for generation
davidkoski
approved these changes
Jun 15, 2026
davidkoski
left a comment
davidkoski
left a comment
Collaborator
There was a problem hiding this comment.
Looks good, thank you!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three iOS-only fixes in `MLXChatExample` so VLM image input actually works on iPhone:
PhotosPicker instead of `fileImporter` — `fileImporter` shows the Files app on iOS, which doesn't expose the photo library. Replace with `.photosPicker` for image / video selection. Custom `Transferable` wrappers (`PickedImage` / `PickedVideo`) request the right content type explicitly so loading is reliable.
Normalize EXIF orientation before saving — `UIImage.jpegData()` writes the EXIF orientation tag but leaves pixel data in raw sensor orientation. `CIImage(contentsOf:)` (used downstream for the model input) does not apply that EXIF tag, so portrait / landscape photos arrive at the VLM rotated. Re-render through `UIGraphicsImageRenderer` before writing the JPEG so the file is upright.
Trim trailing empty assistant message before applying chat template — `ChatViewModel` appends an empty assistant message before generation starts (so the UI has a placeholder to stream into). When that vector is passed straight into the chat template, the template closes the assistant turn and the model emits an EOS immediately. Strip the trailing empty assistant entry, matching the implicit behavior of `ChatSession`.
macOS path unchanged — `fileImporter` still works there. The `#if os(iOS)` blocks are scoped to the new code only.
Test plan
Tested on iPhone (Daisuke の iPhone, iOS 26):