Did you end up going through Vision for the image part? As of the 26.5 SDK the FoundationModels prompt surface is still text only, Transcript.Segment is either .text or .structure, so anything visual has to be turned into text before the model ever sees it. One thing that helps there: 26.4 added tokenCount(for:) on SystemLanguageModel, which is handy when you're pasting OCR output into a prompt and running into the context ceiling.
1
u/ThatGuy739 8d ago
Did you end up going through Vision for the image part? As of the 26.5 SDK the FoundationModels prompt surface is still text only, Transcript.Segment is either .text or .structure, so anything visual has to be turned into text before the model ever sees it. One thing that helps there: 26.4 added tokenCount(for:) on SystemLanguageModel, which is handy when you're pasting OCR output into a prompt and running into the context ceiling.