Building an AI agent that handles multiple input types in one clean flow sounds complicated. Here's how I did it in n8n — and what I'd change if I built it again.
A few months ago, a client came to me with a problem. Their WhatsApp support was a mess — customers sending voice notes, images of broken products, and text questions, all mixed together. Their team was spending hours just sorting through them before they could even start replying.
The solution was building one intelligent flow that could detect what type of message was coming in, handle it appropriately, and reply — all without human involvement.
The architecture — one trigger, three paths
The core idea is simple: every WhatsApp message hits one webhook trigger. From there, the flow checks what type of input it received and routes it accordingly.
Path 1 — Voice notes
Voice notes come in as audio files. The flow downloads the file, sends it to Whisper for transcription, and passes the resulting text to the AI agent as if it were a regular message.
The key insight here: once you transcribe the voice note, everything downstream is identical to a text message. You don't need a separate agent — just a transcription step at the top.
Path 2 — Images
Images go through a vision model that analyzes the content and generates a text description. That description then gets passed to the main agent with context like "the user sent an image of X."
// Simplified routing logic if (messageType === 'audio') → transcribe → agent if (messageType === 'image') → analyze → agent if (messageType === 'text') → agent directly
What I'd do differently
The main thing I underestimated was conversation memory. The first version had no memory — every message was treated as brand new. Adding simple memory context (last 5 messages) made a massive difference to how natural the agent felt.