> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.meetstream.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.meetstream.ai/_mcp/server.

# MIA — MeetStream Infrastructure Agents

MIA is the AI agent layer built into MeetStream's infrastructure. It lets you deploy intelligent, voice-enabled agents into live meetings across Google Meet, Zoom, and Microsoft Teams — no infrastructure to manage.

When a MeetStream bot joins a meeting, MIA receives the live audio stream, processes it in real time through your configured AI pipeline, and responds back into the meeting via voice, chat, or silent tool actions.

Applies to: **Google Meet, Zoom, Microsoft Teams**
Support: docs.meetstream.ai · API: api.meetstream.ai

---

## How It Works

```
Meeting Platform
       │
       ▼
  MeetStream Bot ──── joins as participant
       │
       │  Live audio (binary PCM, low-latency)
       ▼
   MIA Engine
       │
       ├── STT ──► LLM ──► TTS  (pipeline mode)
       │   or
       ├── Realtime provider      (realtime mode)
       │
       ├── MCP Servers ◄──► Tool calls
       │
       ▼
  MeetStream Bot
       │
       ▼
  Meeting Platform ──── speaks, sends chat, or executes actions
```

MIA connects to the bot over two dedicated channels:

| Channel | Format | Purpose |
|---------|--------|---------|
| **Audio** | Binary PCM | Live meeting audio with speaker metadata — sub-100ms delivery, no JSON/base64 overhead |
| **Control** | JSON | Commands: send voice responses, chat messages, images, interrupts |

---

## Operating Modes

MIA supports two modes. Choose based on your latency and flexibility needs.

### Realtime Mode

A single provider handles speech-to-text, reasoning, and text-to-speech in one integrated pipeline. This is the fastest path — audio goes in, audio comes out, with minimal hops.

- **Latency**: ~300ms end-to-end
- **Best for**: Conversational agents where response speed is critical
- **Trade-off**: Provider choice is all-or-nothing — you can't mix STT from one vendor with LLM from another

### Pipeline Mode

Each stage of the pipeline — STT, LLM, and TTS — is independently configurable. Pick the best provider for each job.

- **Latency**: ~600–900ms end-to-end
- **Best for**: Fine-tuned setups where you want specific providers per capability (e.g., Deepgram for STT, GPT-4.1 for reasoning, ElevenLabs for voice)
- **Trade-off**: Higher latency due to sequential processing stages

> **Tip**: Start with Realtime mode for the fastest experience. Switch to Pipeline when you need specific provider combinations or features like wake words and adaptive interruption handling.

---

## Response Modalities

Choose how the agent interacts during the meeting:

| Modality | Behavior |
|----------|----------|
| **Voice** | Agent listens and responds with spoken audio |
| **Chat** | Agent responds via text messages in the meeting chat |
| **Action** | Agent performs tool actions silently — no voice or chat output |

In all three modalities, the agent can execute MCP tools, built-in tools, and custom functions when configured. For example, an action-only agent can silently create tickets, update CRMs, or trigger workflows based on what it hears in the meeting.

---

## Supported Providers

MIA uses BYOK (Bring Your Own Key) for providers configured through MeetStream Integrations. Agent configurations contain provider identifiers and settings, not raw provider API keys.

### Realtime Providers

The public MIA configuration API currently accepts these unified realtime providers:

| Provider identifier | Default model | Default voice |
|----------|---------------|---------------|
| `openai` | `gpt-realtime-mini` | `coral` |
| `google` | `gemini-2.5-flash-native-audio-preview-12-2025` | `Puck` |
| `xai` | Provider-managed | `Ara` |

The API validates provider identifiers. Runtime adapters for other providers are not part of the public MIA configuration contract unless they appear in this table.

### Pipeline STT Providers

| Provider identifier | Default model | Other examples |
|----------|-----------|-----------|
| `deepgram` | `nova-2` | `nova-3`, `nova-2-conversationalai`, `flux-general` |
| `openai` | `whisper-1` | — |
| `assemblyai` | `u3-rt-pro` | `universal-streaming`, `universal-streaming-multilingual` |
| `xai` | `stt-1` | — |

### Pipeline LLM Providers

| Provider identifier | Example models |
|----------|---------------|
| `openai` | `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4o`, `gpt-4o-mini` |
| `anthropic` | An Anthropic model available to your configured account |
| `google` | `gemini-2.5-pro`, `gemini-2.5-flash` |
| `xai` | `grok-3`, `grok-3-mini` |

Model names are passed to the selected provider and are not a guarantee that every model is enabled for every account.

### Pipeline TTS Providers

| Provider identifier | Default model | Voice examples |
|----------|---------------|--------|
| `openai` | `tts-1` | `alloy`, `ash`, `coral`, `nova`, `shimmer` |
| `elevenlabs` | `eleven_turbo_v2_5` | Voice IDs from your ElevenLabs account |
| `xai` | `tts-1` | `ara`, `eve`, `leo`, `rex` |

#### Default TTS model by provider

When `voice.model` is omitted, the API auto-fills a default:

| Provider | Default `voice.model` |
|----------|----------------------|
| `openai` | `tts-1` |
| `elevenlabs` | `eleven_turbo_v2_5` |
| `xai` | `tts-1` |

> **Required fields for `voice`:** `provider` and `voice_id` are always required. `model` is optional (auto-filled). ElevenLabs voice IDs are provider-specific UUIDs — do not reuse OpenAI names like `"alloy"`.

---

## Key Capabilities

### Wake Word Gating (Pipeline Mode)

Pipeline agents currently start with wake-word gating enabled unless `wake_word.enabled=false` is stored. Without an explicit block, runtime defaults are `"hey assistant"`, `"hello bot"`, and a 30-second listening window. Once activated, the agent listens and responds for the configured timeout, then returns to standby.

- Configure one or more wake phrases (e.g., "hey assistant", "hello bot")
- Adjustable activation timeout (default: 30 seconds)
- Each new user utterance during the active window extends the timeout
- **Participant-count bypass**: optionally auto-disable the gate for small meetings — set `bypass_below_participants: 1` to make the bot always-on in 1:1 calls (you + bot). Reacts live to mid-call joins/leaves, so the gate re-enables the moment a second human joins.
- **Max listening window**: optionally cap how long a single listening session can stay open (`max_listening_window`, in seconds). Once exceeded, the user must say the wake word again — useful for high-traffic shared meetings where you want a hard ceiling on always-on listening.

See [`wake_word` in the Agent Config API Reference](/guides/mia/mia-api-guide#nested-wake_word-pipeline-mode-only) for the full field list.

### Adaptive Interruption Handling

MIA distinguishes between real interruptions and background noise like "uh-huh" or "okay":

- **Adaptive mode** (pipeline): Uses ML-based barge-in detection to filter backchanneling — the agent keeps speaking through filler words but stops for genuine interruptions
- **VAD mode**: Silence-based detection — any speech above the threshold triggers an interrupt
- Configurable false interruption timeout with automatic speech resumption
- Minimum duration and word thresholds to prevent noise-triggered interrupts

### Voice Activity Detection

Configurable per mode:

- **Realtime**: Provider-native VAD with semantic understanding (OpenAI) or silence-based detection. Adjustable eagerness, silence duration, and speech probability thresholds.
- **Pipeline**: Local high-performance VAD model with configurable activation threshold, silence duration, and speech padding.

### Turn Detection

Multiple strategies for detecting when the user has finished speaking:

| Mode | Behavior |
|------|----------|
| **VAD** | Silence-based — waits for a configurable pause after speech |
| **STT** | Transcript-based — uses the STT provider's endpointing signals |
| **Multilingual** | ML turn-detector model optimized for non-English conversations |
| **Manual** | Programmatic control — you decide when the agent should respond |

Endpointing can be fixed (constant delay) or dynamic (adapts to conversation rhythm).

### MCP Tool Calling

Connect the agent to external tools and services via Model Context Protocol:

- Multiple MCP servers per agent, each with its own URL and auth headers
- Tool whitelisting — expose only the tools you want the agent to use
- Configurable per-server timeouts
- **Fail-open behavior**: If an MCP server is unreachable at session start, the agent retries without those tools rather than failing entirely
- Auth header values can reference a provisioned runtime variable only when the entire value is `env:VAR_NAME`

### Custom Function Tools

Register any HTTPS endpoint you host as an agent tool — no MCP server required. Each function is described with a JSON Schema so the LLM knows the arguments, and MIA handles the HTTP call end-to-end.

- **When to pick custom functions vs MCP**: custom functions are a single endpoint registration — drop-in for one-off integrations. MCP is preferable when you already run a multi-tool MCP server that several agents share.
- **Authentication**: handled by you through a narrowly scoped credential in the function's headers. Header values are stored with the agent configuration.
- **Retries and timeout**: automatic exponential backoff on 5xx / network errors (up to 5 retries, 120s ceiling).
- **Response cap**: long responses are truncated before being passed back to the LLM so a noisy endpoint doesn't inflate context.
- **Response variables**: pick fields out of successful JSON responses via dot-notation paths. They are available to later custom-function templates and stored-prompt rebuilds triggered by `update_mia`.
- **Speak during execution** (optional): the agent emits a filler utterance (e.g. *"One moment while I pull that up."*) while the HTTP call is in flight so the meeting doesn't go silent.
- **Speak after execution** (optional): set `speak_after_execution=false` to end the turn silently after the tool returns — useful for pure side-effect calls (create-ticket, update-CRM, etc.).
- **Limits**: up to 20 custom functions per agent, up to 20 response variables per function.

See [Custom Functions Guide](/guides/mia/mia-custom-configurations) for an end-to-end walkthrough.

### Built-in Tools

Available without any external MCP server. Enable them by listing their names in `agent.tools`:

| Tool | Description |
|------|-------------|
| `current_time` | Returns current time in ISO 8601 with optional timezone |
| `weather_now` | Returns current weather for a city (no API key required) |

### Echo Cancellation and Speaker Filtering

MIA automatically prevents the agent from hearing and responding to its own voice:

- Audio from known agent speakers is filtered out before processing
- RMS-gated echo suppression when the agent is actively speaking
- Silence gating on all incoming audio below a noise floor

### Text-Only / Chat Mode

The agent processes speech but responds only via meeting chat messages — no voice output. Useful for note-taking bots, silent assistants, or scenarios where voice responses would be disruptive.

Optional trigger keywords can gate when the agent responds in chat mode.

### First Message / Greeting

In pipeline mode, set `model.first_message` to a greeting that the agent speaks (or sends as chat) after it joins the meeting. The agent doesn't wait for user input — it proactively introduces itself.

### Speaker-Aware Responses (Pipeline Mode)

Set `agent.speaker_aware_responses=true` to prefix each transcribed user message with the active participant's display name before it reaches the LLM. Use `agent.speaker_aware_prompt` to replace the default instruction governing how the model uses those names.

- Zoom: strongest source attribution from participant-tagged SDK audio
- Google Meet: approximate active-speaker attribution over mixed audio; per-track input is best effort
- Microsoft Teams: isolated participant tracks when available; dominant-speaker fallback otherwise
- Custom prompt limit: 2000 characters

The prefix uses the latest speaker state when an utterance completes, not speaker-bound transcript spans. Rapid hand-offs and overlapping speech can therefore be misattributed. Speaker awareness is compatible with wake words, custom functions, and MCP tools. It is documented for pipeline agents; do not rely on it for realtime-provider speaker attribution.

### Template Variables and Live Updates

Use `{{variable}}` placeholders in `model.system_prompt` and `model.first_message`, then pass values in `agent_config_params` when creating the bot. Unresolved placeholders remain unchanged.

While a bot is in a meeting, `POST /api/v1/bots/{bot_id}/update_mia` can:

- merge new `agent_config_params`
- replace the active `system_prompt`
- force wake-word gating on or off with `wake_word_enabled`
- replace phrases with `wake_words`

The endpoint accepts and forwards live changes asynchronously; HTTP success does not wait for confirmation from the active agent session. Applied changes are held in memory. Update the saved agent with `PUT /api/v1/mia` when a change must persist to future sessions.

### Virtual Avatars (Anam)

Video-based agent presence — instead of the agent appearing as a static bot tile, a photorealistic animated avatar lip-syncs to the TTS output in real time on the bot's video track.

- **Provider**: [Anam](https://www.anam.ai) — BYOK, key added as `avatar.anam` in the integrations page
- **Works in**: both Realtime and Pipeline modes, on all three meeting platforms
- **Avatar selection**: pick any avatar from your Anam account by `avatar_id` (listed via Anam's `GET /v1/avatars`)
- **Preflight validation**: MeetStream verifies the `avatar_id` and API key when the agent starts and returns specific errors for common misconfigurations — including the frequent `personaId` vs `avatarId` mix-up
- **Fail-safe**: if Anam is unreachable or hits its concurrent-session limit, MeetStream returns a diagnostic error (open-session list + auto-expiry ETA) so you can recover without guessing

### BYOK API Keys

All provider API keys follow MeetStream's BYOK (Bring Your Own Key) model:

- Keys are stored securely in MeetStream's infrastructure — never in agent configs or databases
- Resolved at runtime when a session starts
- Configure keys through the MeetStream dashboard integrations page

### Per-Bot Logging

Each bot session gets dedicated logging with:

- All agent state transitions (thinking, speaking, listening)
- User transcripts
- Latency metrics (user-finished-speaking → agent-speaking, LLM+TTS breakdown)
- Tool call results and errors

---

## Configuration Reference

Agent configuration is set through the MIA dashboard. Below are the key configuration areas and what they control.

### Agent Settings

| Setting | Options | Description |
|---------|---------|-------------|
| Mode | `realtime`, `pipeline` | Single-provider vs multi-provider pipeline |
| `agent.response_modality` | `audio`, `chat`, `action` | How the agent communicates back; defaults to `audio` |
| System prompt | Free text | Defines agent personality, constraints, and behavior |
| `model.first_message` | Free text | Optional pipeline greeting spoken when the bot joins |
| `agent.speaker_aware_responses` | `true`, `false` | Prefix pipeline transcripts with the participant's name |
| `agent.speaker_aware_prompt` | Free text, max 2000 chars | Optional instruction for interpreting speaker names |

### Provider Settings

**Realtime mode**: Select one provider and model for the entire pipeline.

**Pipeline mode**: Select independently:
- STT provider + model + language
- LLM provider + model + parameters (temperature, max tokens, etc.)
- TTS provider + model + voice

### Interruption Settings

| Setting | Default | Description |
|---------|---------|-------------|
| Enable interruptions | Yes | Whether users can interrupt the agent |
| Interruption mode | VAD | Set pipeline agents to `adaptive` when LiveKit Cloud adaptive interruption handling is available |
| False interruption timeout | 2 seconds | Wait time before declaring a noise-triggered interrupt as false |
| Resume after false interrupt | Yes | Automatically resume agent speech after a false interrupt |
| Minimum speech duration | 0.5 seconds | User must speak this long to count as an interrupt |
| Minimum word count | 2 (pipeline) / 0 (realtime) | Transcribed words required to count as an interrupt |

### Wake Word Settings (Pipeline Only)

| Setting | Required | Default | Description |
|---------|----------|---------|-------------|
| Enabled | No | `true` at runtime | Whether wake word gating is active; set explicitly to `false` for always-on listening |
| Words | **Yes in an explicit enabled block** | `["hey assistant", "hello bot"]` at runtime | Trigger phrases (case-insensitive) |
| Timeout | No | 30 seconds | How long the agent stays active after hearing the wake word; rolls forward on every new utterance |
| Bypass below participants | No | 0 (off) | When the meeting has at most this many humans (the bot is excluded), wake-word gating is skipped entirely. Reacts live to joins/leaves. Set to `1` for 1:1 calls. |
| Max listening window | No | 0 (off) | Hard cap (seconds) on a single listening session. Once exceeded, user must re-trigger the wake word. Default rolling-extend behaviour preserved when `0`. |

### MCP Server Settings

| Setting | Required | Description |
|---------|----------|-------------|
| URL | Yes | Streamable HTTP MCP endpoint |
| Headers | No | Authentication headers; environment expansion requires the entire value to be `env:VAR_NAME` and a provisioned runtime variable |
| Allowed tools | No | Whitelist of tool names — omit to allow all |
| Timeout | No (default: 10s) | Request timeout per tool call |
| `agent.mcp_fail_open` | No (default: `true`) | Retry the session without MCP tools if MCP startup fails |

### Custom Function Settings

| Setting | Required | Description |
|---------|----------|-------------|
| Name | Yes | Snake-case identifier, unique per agent, max 64 chars |
| Description | Yes | Natural-language description shown to the LLM — drives tool selection |
| Method | No (default: `POST`) | HTTP method: `GET`, `POST`, `PATCH`, `PUT`, or `DELETE` |
| URL | Yes | HTTPS endpoint; must resolve to a public IP (SSRF-protected) |
| Parameters | No | JSON Schema (`type: object`) describing the LLM-supplied arguments |
| Headers | No | Extra HTTP headers used for endpoint authentication; values may use `{{variable}}`. Environment expansion requires a whole `env:VAR_NAME` value. MeetStream adds `X-Bot-ID`; `X-Agent-ID` is optional and must not be required. |
| Timeout | No (default: 30s) | Total request timeout; range 1–120s |
| Retries | No (default: 2) | Number of retries on 5xx/network; range 0–5 |
| Response cap | No (default: 15000) | Characters of response body fed back to the LLM; range 1000–50000 |
| Speak during execution | No (default: off) | Emit a filler utterance while the HTTP call is in flight |
| Speak during prompt | No | Optional fixed filler text (e.g. *"One moment..."*); defaults to *"One moment please."* if omitted |
| Speak after execution | No (default: on) | If off, the agent's turn ends silently after the tool completes |
| Response variables | No | List of `{ name, path }` extractors — dot-notation JSON paths |

### Avatar Settings

| Setting | Required | Description |
|---------|----------|-------------|
| Enabled | No (default: false) | Whether the agent renders a video avatar on its participant tile |
| Provider | No (default: `anam`) | Avatar provider — currently `anam` only |
| Avatar ID | Yes (when enabled with `anam`) | Anam `avatar_id` — pick from `GET https://api.anam.ai/v1/avatars`. Must be an `avatar_id`, not a `persona_id` |
| Avatar model | No | Optional Anam `avatarModel` override |
| Name | No | Optional persona display name |

> Requires an Anam API key configured under **Integrations → Avatar → Anam**.

---

## Example Configurations

### Realtime Voice Agent

Fastest possible agent — single provider, voice responses.

```
Mode:              Realtime
Provider:          OpenAI
Model:             gpt-4o-realtime-preview
Response:          Voice
System prompt:     "You are a concise meeting assistant. Answer questions
                    about the discussion and help track action items."
```

### Pipeline Chat Agent

Multi-provider setup, responds via meeting chat.

```
Mode:              Pipeline
STT:               Deepgram nova-3
LLM:               Google gemini-2.5-flash
TTS:               (none — chat mode)
Response:          Chat
System prompt:     "You are a silent meeting note-taker. Summarize key
                    points when asked. Keep responses brief."
Wake word:         "hey notes"
```

### Action-Only Agent with MCP

Silently executes tools based on meeting conversation.

```
Mode:              Pipeline
STT:               Deepgram nova-3
LLM:               OpenAI gpt-4.1
TTS:               (none — action mode)
Response:          Action
System prompt:     "Listen for action items. When someone says 'create a
                    ticket for [task]', use the Linear tool to create it.
                    Do not speak or send chat messages."
MCP Server:        https://your-mcp-gateway.example.com/mcp
  Authorization:   Bearer <token>
  Allowed tools:   create_issue, list_issues
```

### Realtime Agent with Anam Avatar

Voice agent with a photorealistic animated avatar rendered on the bot's video track. Requires `ANAM_API_KEY` configured via **Integrations → Avatar → Anam**.

```
Mode:              Realtime
Provider:          OpenAI gpt-realtime
Response:          Voice
Avatar:            Anam (avatar_id = "f0e7a8c4-…")
System prompt:     "You are a friendly customer success agent. Greet
                    participants warmly and help them get started."
```

### Pipeline Agent with Custom Function

Support-style agent that calls your own HTTPS endpoint to look up an order. The response's `data.order.status` is captured and exposed as `{{latest_order_status}}` for the rest of the session, so a subsequent `update_mia` prompt can reference it directly.

```
Mode:              Pipeline
STT:               Deepgram nova-3
LLM:               OpenAI gpt-4.1
TTS:               OpenAI nova
Response:          Voice
System prompt:     "You are a support agent. When the caller asks about an
                    order, call lookup_order. When they ask to open a
                    ticket, call create_ticket."

Custom function 1: name             = lookup_order
                   method / url     = POST https://api.example.com/orders/lookup
                   parameters       = { order_id: string (required) }
                   speak_during     = "One moment while I pull that up."
                   response_vars    = latest_order_status ← data.order.status

Custom function 2: name             = create_ticket
                   method / url     = POST https://api.example.com/tickets
                   parameters       = { subject, body, priority }
                   speak_after      = false   (silent side-effect)
```

### xAI Full Pipeline Agent

Full xAI-powered pipeline using a single `XAI_API_KEY` for all three components: Grok STT, Grok LLM, and Aurora TTS. Requires `XAI_API_KEY` configured via the Integrations page.

```
Mode:              Pipeline
STT:               xAI stt-1  (25 languages)
LLM:               xAI grok-3
TTS:               xAI tts-1, voice: ara
Response:          Voice
System prompt:     "You are a helpful AI meeting assistant powered by xAI.
                    Keep responses concise and natural."
```

**Available xAI TTS voices:** `ara` (warm, friendly), `eve` (energetic, upbeat), `leo` (authoritative, strong), `rex` (confident, clear)

**API configuration:**
```json
{
  "mode": "pipeline",
  "model": {
    "provider": "xai",
    "model": "grok-3",
    "system_prompt": "You are a helpful AI meeting assistant. Keep responses concise.",
    "temperature": 0.8,
    "max_tokens": 150
  },
  "voice": {
    "provider": "xai",
    "model": "tts-1",
    "voice_id": "ara"
  },
  "transcriber": {
    "provider": "xai",
    "model": "stt-1",
    "language": "en"
  }
}
```

---

## Related Guides

- [Create an AI Agent & Bring It Into a Meeting](/guides/mia/create-an-agent) — Step-by-step dashboard walkthrough with MCP setup
- [Agent Config API Reference](/guides/mia/mia-api-guide) — Complete CRUD, field, default, template-variable, and live-update contract
- [Custom Functions Guide](/guides/mia/mia-custom-configurations) — End-to-end setup for custom HTTP tools and response variables
- [Bridge Server Architecture](../bridge-server-architecture.md) — Build your own AI bridge instead of using MIA
- [Live WebSocket Integration](../live-websocket-integration-guide.md) — Direct WebSocket integration for custom audio processing
- [Meeting Control & Command Patterns](../meeting-control-and-command-patterns.md) — All available commands (sendaudio, sendmsg, interrupt, etc.)