Skip to content

Voice mode should use the selected Assistant's persona/system prompt #625

Description

@philmerrell

Problem

Voice mode ignores the active Assistant entirely. A faculty member using a role-play Assistant (AI embodies a character) found voice mode drops the persona and behaves like a generic feedback chatbot — no amount of Assistant prompt tuning can fix it, because the persona is never sent.

Current behavior

  • voice_routes.py (backend/src/apis/inference_api/chat/voice_routes.py:370-375) constructs VoiceAgent with only session_id, user_id, auth_token, enabled_tools — no system_prompt and no assistant id.
  • With no system_prompt, BaseAgent.__init__ (backend/src/agents/main_agent/base_agent.py:132-137) builds the generic default prompt.
  • VoiceAgent._build_voice_system_prompt() (voice_agent.py:140-150) appends a fixed voice addendum (concise, conversational, etc.) to that default.
  • The frontend WebSocket config frame only carries session_id (voice-chat.service.ts:302-306), so there's nothing server-side to resolve an assistant from.

Voice does load prior text history for continuity (_load_text_history), but that's conversation messages, not the persona.

Proposed change

  • Frontend: include the active assistant/agent id in the voice config frame.
  • Backend: resolve the assistant's system prompt (with the same access checks the text path uses) and pass it into VoiceAgent, then append the voice addendum on top of it.
  • Consider how the voice addendum interacts with strongly-in-character personas (e.g. "confirm understanding before taking actions" may fight a role-play prompt — the addendum may need to be softened or made persona-aware).

Notes

Driving use case: instructor role-play scenarios used in courses. Related: voice selection is tracked separately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions