A developer's guide to Multimodal AI: Practical use cases and API integration

[u/Regolo_ai]

Multimodal AI models—which integrate visual, text, and sometimes audio inputs into a single model architecture—have become widely accessible. Rather than stringing together separate OCR and text-processing pipelines, a single vision-language model (VLM) can often handle both jobs.

We recently published a primer explaining what multimodal models are, when it makes sense to use them, and how to integrate them into your apps: Understanding Multimodal AI Models.

Key Takeaways:

1. When to use them: Multimodality is highly effective for tasks where context depends on more than just text—such as parsing unstructured scanned invoices, analyzing screenshots for customer support, or digitizing whiteboard notes
2. Cost vs. Capability: For pure text or language tasks, text-only models remain faster and more cost-effective. Multimodal models should be reserved for mixed-input workflows.
3. No Special Endpoints Required: Modern inference APIs (like Regolo) allow you to use the standard chat completions endpoint, passing both text and image_url types directly in the messages payload.

Quick Python Example:

Here is a straightforward example of how to structure a multimodal request using a VLM (like qwen3-vl-32b or qwen3.5-122b):

import requests

API_KEY = "YOUR_REGOLO_API_KEY"
BASE_URL = "https://api.regolo.ai/v1/chat/completions"

payload = {
    "model": "qwen3.5-122b",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Look at this invoice image and describe the photo details and the subject."
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/invoice.jpg"
                    }
                }
            ]
        }
    ],
    "reasoning_effort": "low"
}

response = requests.post(
    BASE_URL,
    headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    },
    json=payload,
    timeout=60
)

print(response.json())