Multimodal Input

What it means

The interface accepts multiple input modalities in a single turn: type a question, attach a photo, paste a link, or record audio, then send one combined prompt.

Why designers should care

Multimodal input needs clear attachment affordances, preview, size limits, privacy cues, and failure states when a file type or size is unsupported.

Example

A UX review tool accepts a Figma screenshot plus “check contrast on this modal”; the model references regions in the image and returns annotated findings.

Common mistakes

• • Attachments with no preview or remove control before send.
• • Treating image upload like generic file attach with no capture guidance.
• • No alt text or description path when images fail to upload on slow networks.

Related terms

• Multimodal: A multimodal model accepts or produces multiple media types, not just strings of text, often in one conversation or agent run.
• Token: Models process and bill text as tokens; a rough rule is ~¾ of a word per token in English, but code and symbols can consume more.
• Context Window: Everything sent to the model must fit inside the context window; overflow is truncated, summarized, or rejected depending on product policy.
• Prompt: Prompts set role, task, constraints, format, and context; the model’s reply is a direct response to that combined instruction stack.
• Large Language Model (LLM): An LLM reads a sequence of tokens (words and symbols) and generates the next tokens, producing paragraphs, code, JSON, or tool requests from natural-language instructions.