MIA — MeetStream Infrastructure Agents

MIA is the AI agent layer built into MeetStream’s infrastructure. It lets you deploy intelligent, voice-enabled agents into live meetings across Google Meet, Zoom, and Microsoft Teams — no infrastructure to manage.

When a MeetStream bot joins a meeting, MIA receives the live audio stream, processes it in real time through your configured AI pipeline, and responds back into the meeting via voice, chat, or silent tool actions.

Applies to: Google Meet, Zoom, Microsoft Teams Support: docs.meetstream.ai · API: api.meetstream.ai

How It Works

Meeting Platform
       │
       ▼
  MeetStream Bot ──── joins as participant
       │
       │  Live audio (binary PCM, low-latency)
       ▼
   MIA Engine
       │
       ├── STT ──► LLM ──► TTS  (pipeline mode)
       │   or
       ├── Realtime provider      (realtime mode)
       │
       ├── MCP Servers ◄──► Tool calls
       │
       ▼
  MeetStream Bot
       │
       ▼
  Meeting Platform ──── speaks, sends chat, or executes actions

MIA connects to the bot over two dedicated channels:

Operating Modes

MIA supports two modes. Choose based on your latency and flexibility needs.

Realtime Mode

A single provider handles speech-to-text, reasoning, and text-to-speech in one integrated pipeline. This is the fastest path — audio goes in, audio comes out, with minimal hops.

• Latency: ~300ms end-to-end
• Best for: Conversational agents where response speed is critical
• Trade-off: Provider choice is all-or-nothing — you can’t mix STT from one vendor with LLM from another

Pipeline Mode

Each stage of the pipeline — STT, LLM, and TTS — is independently configurable. Pick the best provider for each job.

• Latency: ~600–900ms end-to-end
• Best for: Fine-tuned setups where you want specific providers per capability (e.g., Deepgram for STT, GPT-4.1 for reasoning, ElevenLabs for voice)
• Trade-off: Higher latency due to sequential processing stages

Tip: Start with Realtime mode for the fastest experience. Switch to Pipeline when you need specific provider combinations or features like wake words and adaptive interruption handling.

Response Modalities

Choose how the agent interacts during the meeting:

In all three modalities, the agent can execute MCP tools, built-in tools, and custom functions when configured. For example, an action-only agent can silently create tickets, update CRMs, or trigger workflows based on what it hears in the meeting.

Supported Providers

MIA uses BYOK (Bring Your Own Key) for providers configured through MeetStream Integrations. Agent configurations contain provider identifiers and settings, not raw provider API keys.

Realtime Providers

The public MIA configuration API currently accepts these unified realtime providers:

The API validates provider identifiers. Runtime adapters for other providers are not part of the public MIA configuration contract unless they appear in this table.

Pipeline STT Providers

Pipeline LLM Providers

Model names are passed to the selected provider and are not a guarantee that every model is enabled for every account.

Pipeline TTS Providers

Default TTS model by provider

When voice.model is omitted, the API auto-fills a default:

Required fields for voice: provider and voice_id are always required. model is optional (auto-filled). ElevenLabs voice IDs are provider-specific UUIDs — do not reuse OpenAI names like "alloy".

Key Capabilities

Wake Word Gating (Pipeline Mode)

Pipeline agents currently start with wake-word gating enabled unless wake_word.enabled=false is stored. Without an explicit block, runtime defaults are "hey assistant", "hello bot", and a 30-second listening window. Once activated, the agent listens and responds for the configured timeout, then returns to standby.

• Configure one or more wake phrases (e.g., “hey assistant”, “hello bot”)
• Adjustable activation timeout (default: 30 seconds)
• Each new user utterance during the active window extends the timeout
• Participant-count bypass: optionally auto-disable the gate for small meetings — set bypass_below_participants: 1 to make the bot always-on in 1:1 calls (you + bot). Reacts live to mid-call joins/leaves, so the gate re-enables the moment a second human joins.
• Max listening window: optionally cap how long a single listening session can stay open (max_listening_window, in seconds). Once exceeded, the user must say the wake word again — useful for high-traffic shared meetings where you want a hard ceiling on always-on listening.

See wake_word in the Agent Config API Reference for the full field list.

Adaptive Interruption Handling

MIA distinguishes between real interruptions and background noise like “uh-huh” or “okay”:

• Adaptive mode (pipeline): Uses ML-based barge-in detection to filter backchanneling — the agent keeps speaking through filler words but stops for genuine interruptions
• VAD mode: Silence-based detection — any speech above the threshold triggers an interrupt
• Configurable false interruption timeout with automatic speech resumption
• Minimum duration and word thresholds to prevent noise-triggered interrupts

Voice Activity Detection

Configurable per mode:

• Realtime: Provider-native VAD with semantic understanding (OpenAI) or silence-based detection. Adjustable eagerness, silence duration, and speech probability thresholds.
• Pipeline: Local high-performance VAD model with configurable activation threshold, silence duration, and speech padding.

Turn Detection

Multiple strategies for detecting when the user has finished speaking:

Endpointing can be fixed (constant delay) or dynamic (adapts to conversation rhythm).

MCP Tool Calling

Connect the agent to external tools and services via Model Context Protocol:

• Multiple MCP servers per agent, each with its own URL and auth headers
• Tool whitelisting — expose only the tools you want the agent to use
• Configurable per-server timeouts
• Fail-open behavior: If an MCP server is unreachable at session start, the agent retries without those tools rather than failing entirely
• Auth header values can reference a provisioned runtime variable only when the entire value is env:VAR_NAME

Custom Function Tools

Register any HTTPS endpoint you host as an agent tool — no MCP server required. Each function is described with a JSON Schema so the LLM knows the arguments, and MIA handles the HTTP call end-to-end.

• When to pick custom functions vs MCP: custom functions are a single endpoint registration — drop-in for one-off integrations. MCP is preferable when you already run a multi-tool MCP server that several agents share.
• Authentication: handled by you through a narrowly scoped credential in the function’s headers. Header values are stored with the agent configuration.
• Retries and timeout: automatic exponential backoff on 5xx / network errors (up to 5 retries, 120s ceiling).
• Response cap: long responses are truncated before being passed back to the LLM so a noisy endpoint doesn’t inflate context.
• Response variables: pick fields out of successful JSON responses via dot-notation paths. They are available to later custom-function templates and stored-prompt rebuilds triggered by update_mia.
• Speak during execution (optional): the agent emits a filler utterance (e.g. “One moment while I pull that up.”) while the HTTP call is in flight so the meeting doesn’t go silent.
• Speak after execution (optional): set speak_after_execution=false to end the turn silently after the tool returns — useful for pure side-effect calls (create-ticket, update-CRM, etc.).
• Limits: up to 20 custom functions per agent, up to 20 response variables per function.

See Custom Functions Guide for an end-to-end walkthrough.

Built-in Tools

Available without any external MCP server. Enable them by listing their names in agent.tools:

Echo Cancellation and Speaker Filtering

MIA automatically prevents the agent from hearing and responding to its own voice:

• Audio from known agent speakers is filtered out before processing
• RMS-gated echo suppression when the agent is actively speaking
• Silence gating on all incoming audio below a noise floor

Text-Only / Chat Mode

The agent processes speech but responds only via meeting chat messages — no voice output. Useful for note-taking bots, silent assistants, or scenarios where voice responses would be disruptive.

Optional trigger keywords can gate when the agent responds in chat mode.

First Message / Greeting

In pipeline mode, set model.first_message to a greeting that the agent speaks (or sends as chat) after it joins the meeting. The agent doesn’t wait for user input — it proactively introduces itself.

Speaker-Aware Responses (Pipeline Mode)

Set agent.speaker_aware_responses=true to prefix each transcribed user message with the active participant’s display name before it reaches the LLM. Use agent.speaker_aware_prompt to replace the default instruction governing how the model uses those names.

• Zoom: strongest source attribution from participant-tagged SDK audio
• Google Meet: approximate active-speaker attribution over mixed audio; per-track input is best effort
• Microsoft Teams: isolated participant tracks when available; dominant-speaker fallback otherwise
• Custom prompt limit: 2000 characters

The prefix uses the latest speaker state when an utterance completes, not speaker-bound transcript spans. Rapid hand-offs and overlapping speech can therefore be misattributed. Speaker awareness is compatible with wake words, custom functions, and MCP tools. It is documented for pipeline agents; do not rely on it for realtime-provider speaker attribution.

Template Variables and Live Updates

Use {{variable}} placeholders in model.system_prompt and model.first_message, then pass values in agent_config_params when creating the bot. Unresolved placeholders remain unchanged.

While a bot is in a meeting, POST /api/v1/bots/{bot_id}/update_mia can:

• merge new agent_config_params
• replace the active system_prompt
• force wake-word gating on or off with wake_word_enabled
• replace phrases with wake_words

The endpoint accepts and forwards live changes asynchronously; HTTP success does not wait for confirmation from the active agent session. Applied changes are held in memory. Update the saved agent with PUT /api/v1/mia when a change must persist to future sessions.

Virtual Avatars (Anam)

Video-based agent presence — instead of the agent appearing as a static bot tile, a photorealistic animated avatar lip-syncs to the TTS output in real time on the bot’s video track.

• Provider: Anam — BYOK, key added as avatar.anam in the integrations page
• Works in: both Realtime and Pipeline modes, on all three meeting platforms
• Avatar selection: pick any avatar from your Anam account by avatar_id (listed via Anam’s GET /v1/avatars)
• Preflight validation: MeetStream verifies the avatar_id and API key when the agent starts and returns specific errors for common misconfigurations — including the frequent personaId vs avatarId mix-up
• Fail-safe: if Anam is unreachable or hits its concurrent-session limit, MeetStream returns a diagnostic error (open-session list + auto-expiry ETA) so you can recover without guessing

BYOK API Keys

All provider API keys follow MeetStream’s BYOK (Bring Your Own Key) model:

• Keys are stored securely in MeetStream’s infrastructure — never in agent configs or databases
• Resolved at runtime when a session starts
• Configure keys through the MeetStream dashboard integrations page

Per-Bot Logging

Each bot session gets dedicated logging with:

• All agent state transitions (thinking, speaking, listening)
• User transcripts
• Latency metrics (user-finished-speaking → agent-speaking, LLM+TTS breakdown)
• Tool call results and errors

Configuration Reference

Agent configuration is set through the MIA dashboard. Below are the key configuration areas and what they control.

Agent Settings

Provider Settings

Realtime mode: Select one provider and model for the entire pipeline.

Pipeline mode: Select independently:

• STT provider + model + language
• LLM provider + model + parameters (temperature, max tokens, etc.)
• TTS provider + model + voice

Interruption Settings

Wake Word Settings (Pipeline Only)

MCP Server Settings

Custom Function Settings

Avatar Settings

Requires an Anam API key configured under Integrations → Avatar → Anam.

Example Configurations

Realtime Voice Agent

Fastest possible agent — single provider, voice responses.

Mode:              Realtime
Provider:          OpenAI
Model:             gpt-4o-realtime-preview
Response:          Voice
System prompt:     "You are a concise meeting assistant. Answer questions
                    about the discussion and help track action items."

Pipeline Chat Agent

Multi-provider setup, responds via meeting chat.

Mode:              Pipeline
STT:               Deepgram nova-3
LLM:               Google gemini-2.5-flash
TTS:               (none — chat mode)
Response:          Chat
System prompt:     "You are a silent meeting note-taker. Summarize key
                    points when asked. Keep responses brief."
Wake word:         "hey notes"

Action-Only Agent with MCP

Silently executes tools based on meeting conversation.

Mode:              Pipeline
STT:               Deepgram nova-3
LLM:               OpenAI gpt-4.1
TTS:               (none — action mode)
Response:          Action
System prompt:     "Listen for action items. When someone says 'create a
                    ticket for [task]', use the Linear tool to create it.
                    Do not speak or send chat messages."
MCP Server:        https://your-mcp-gateway.example.com/mcp
  Authorization:   Bearer <token>
  Allowed tools:   create_issue, list_issues

Realtime Agent with Anam Avatar

Voice agent with a photorealistic animated avatar rendered on the bot’s video track. Requires ANAM_API_KEY configured via Integrations → Avatar → Anam.

Mode:              Realtime
Provider:          OpenAI gpt-realtime
Response:          Voice
Avatar:            Anam (avatar_id = "f0e7a8c4-…")
System prompt:     "You are a friendly customer success agent. Greet
                    participants warmly and help them get started."

Pipeline Agent with Custom Function

Support-style agent that calls your own HTTPS endpoint to look up an order. The response’s data.order.status is captured and exposed as {{latest_order_status}} for the rest of the session, so a subsequent update_mia prompt can reference it directly.

Mode:              Pipeline
STT:               Deepgram nova-3
LLM:               OpenAI gpt-4.1
TTS:               OpenAI nova
Response:          Voice
System prompt:     "You are a support agent. When the caller asks about an
                    order, call lookup_order. When they ask to open a
                    ticket, call create_ticket."
Custom function 1: name             = lookup_order
                   method / url     = POST https://api.example.com/orders/lookup
                   parameters       = { order_id: string (required) }
                   speak_during     = "One moment while I pull that up."
                   response_vars    = latest_order_status ← data.order.status
Custom function 2: name             = create_ticket
                   method / url     = POST https://api.example.com/tickets
                   parameters       = { subject, body, priority }
                   speak_after      = false   (silent side-effect)

xAI Full Pipeline Agent

Full xAI-powered pipeline using a single XAI_API_KEY for all three components: Grok STT, Grok LLM, and Aurora TTS. Requires XAI_API_KEY configured via the Integrations page.

Mode:              Pipeline
STT:               xAI stt-1  (25 languages)
LLM:               xAI grok-3
TTS:               xAI tts-1, voice: ara
Response:          Voice
System prompt:     "You are a helpful AI meeting assistant powered by xAI.
                    Keep responses concise and natural."

Available xAI TTS voices: ara (warm, friendly), eve (energetic, upbeat), leo (authoritative, strong), rex (confident, clear)

API configuration:

1
{
2
  "mode": "pipeline",
3
  "model": {
4
    "provider": "xai",
5
    "model": "grok-3",
6
    "system_prompt": "You are a helpful AI meeting assistant. Keep responses concise.",
7
    "temperature": 0.8,
8
    "max_tokens": 150
9
  },
10
  "voice": {
11
    "provider": "xai",
12
    "model": "tts-1",
13
    "voice_id": "ara"
14
  },
15
  "transcriber": {
16
    "provider": "xai",
17
    "model": "stt-1",
18
    "language": "en"
19
  }
20
}

Related Guides

• Create an AI Agent & Bring It Into a Meeting — Step-by-step dashboard walkthrough with MCP setup
• Agent Config API Reference — Complete CRUD, field, default, template-variable, and live-update contract
• Custom Functions Guide — End-to-end setup for custom HTTP tools and response variables
• Bridge Server Architecture — Build your own AI bridge instead of using MIA
• Live WebSocket Integration — Direct WebSocket integration for custom audio processing
• Meeting Control & Command Patterns — All available commands (sendaudio, sendmsg, interrupt, etc.)