Channels: Phone, Web Voice & Video, and Chat
chakra-convai supports three distinct channel modalities from a single shared codebase:
1. Phone (phone): Traditional telephony calls arriving through SIP trunks.
2. Web Voice & Video (web with mode: "voice"): Realtime voice streaming combined with user camera or screen sharing.
3. Web Chat (web with mode: "chat"): Low-latency typed text and photo messaging.
Each agent defined in chakra.toml declares its target channel ([[agents]] channel = "phone" | "web").
Channel Comparison
| Feature | Phone (phone) |
Web Voice (web_voice) |
Web Chat (web_chat) |
|---|---|---|---|
| Transport | SIP / Telephony trunk via LiveKit SIP | LiveKit WebRTC Room | LiveKit WebRTC Room |
| User Input | Spoken audio | Audio + Camera / Screen | Typed text + Photo uploads |
| Agent Output | Synthesized speech | Synthesized speech | Text responses |
| Silence Handling | Enabled (configurable check-ins) | Enabled | Disabled (uses idle timeout) |
| Vision Support | N/A | Full (Gemini video / Chakra vision frames) | Photos attached as user messages |
| Talk Time Limits | Enforced by daily caller limit | Enforced by daily caller limit | Excluded from talk limits |
| Identity Keying | Caller caller ID (phone number) | user.phone or user.id |
user.phone or user.id |
1. Phone Channel
Configured with channel = "phone". Phone sessions are dialed through SIP into LiveKit rooms. The caller ID becomes the user identity, linking directly to PostgreSQL caller memory and daily call duration limits.
2. Web Voice & Video
Web voice sessions connect an end-user from an iOS, Android, or Web client via WebRTC.
Multimodal Vision & Camera Frames
gemini_livePipeline: The agent natively subscribes to the user's incoming video track.chakraPipeline: The engine captures user camera frames ([web] vision_frames, default4, one frame per second) and attaches them to each user speech turn. The managed LLM inspects what the user showed while speaking. Once a turn is complete, earlier frames are evicted so subsequent turns only bill their own tokens.
3. Web Chat
When an end-user interacts over chat (or when room name starts with chat_room_prefix), the session runs in text/photo mode:
- Spoken greetings, VAD, and silence detectors are disabled.
- Image uploads from the user are converted into multimodal prompts.
- Sessions automatically close when inactive after
chat_idle_timeout_seconds.
The Standard Session Data Format
Every web and mobile application sends session metadata inside the LiveKit room's metadata string:
{
"chakra": {
"version": 1,
"mode": "voice",
"language": "si",
"user": {
"id": "usr_99812",
"phone": "0771234567",
"name": "Nimal Perera"
},
"context": {
"district": "Kandy",
"crops": ["tomato", "chilli"],
"account_tier": "premium"
}
}
}
Schema Details
mode:"voice"or"chat".language: ISO code ("si","ta","en") or descriptive language name. Locked for the session when[web] keep_language = true.user: User identity fields.user.phonetakes precedence for caller memory, falling back touser.id.context: Arbitrary key-value dictionary that is formatted and injected directly into the LLM system prompt on every turn.
Building Metadata in Python
Applications generating LiveKit room tokens in Python can use the built-in helper: