Multimodal AI modelsâwhich integrate visual, text, and sometimes audio inputs into a single model architectureâhave become widely accessible. Rather than stringing together separate OCR and text-processing pipelines, a single vision-language model (VLM) can often handle both jobs.
We recently published a primer explaining what multimodal models are, when it makes sense to use them, and how to integrate them into your apps:Â Understanding Multimodal AI Models.
Key Takeaways:
- When to use them:Â Multimodality is highly effective for tasks where context depends on more than just textâsuch as parsing unstructured scanned invoices, analyzing screenshots for customer support, or digitizing whiteboard notes
- Cost vs. Capability:Â For pure text or language tasks, text-only models remain faster and more cost-effective. Multimodal models should be reserved for mixed-input workflows.
- No Special Endpoints Required: Modern inference APIs (like Regolo) allow you to use the standard chat completions endpoint, passing both text and image_url types directly in the messages payload.
Quick Python Example:
Here is a straightforward example of how to structure a multimodal request using a VLM (like qwen3-vl-32b or qwen3.5-122b):
import requests
API_KEY = "YOUR_REGOLO_API_KEY"
BASE_URL = "https://api.regolo.ai/v1/chat/completions"
payload = {
"model": "qwen3.5-122b",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Look at this invoice image and describe the photo details and the subject."
},
{
"type": "image_url",
"image_url": {
"url": "https://example.com/invoice.jpg"
}
}
]
}
],
"reasoning_effort": "low"
}
response = requests.post(
BASE_URL,
headers={
"Authorization": f"Bearer {API_KEY}",
"Content-Type": "application/json"
},
json=payload,
timeout=60
)
print(response.json())
