Voice Agent Interruption Handling (Barge-In)

Interruption handling (also known as barge-in) allows users to speak while the bot is talking, creating a more natural conversation experience.

Why Interruptions Matter#

Without interruption handling:
─────────────────────────────────────────────────────────────
Bot: "I can help you with that. Your order number one two three
     four five was shipped on December twentieth and is currently
     in transit. The estimated delivery date is December twenty..."
User: "WAIT! That's the wrong order!"
Bot: "...ninth. Is there anything else I can help you with?"
     ← Bot continues, ignoring user

With interruption handling:
─────────────────────────────────────────────────────────────
Bot: "I can help you with that. Your order number one two three—"
User: "Wait, that's the wrong order!"
Bot: [Stops immediately] "I apologize. What's the correct order number?"
     ← Natural conversation

How Interruption Detection Works#

                     ┌─────────────────────────────────────────┐
                     │              Pipeline                   │
                     │                                         │
User Audio ─────────►│─► VAD ──────────────────────────────────│
                     │     │                                   │
                     │     ▼                                   │
                     │   Is bot speaking?                      │
                     │     │                                   │
                     │     ├── No: Normal processing           │
                     │     │                                   │
                     │     └── Yes: INTERRUPTION!              │
                     │            │                            │
Bot Speaking ◄───────│◄───────────│                            │
                     │            ▼                            │
                     │      1. Stop TTS playback               │
                     │      2. Cancel LLM generation           │
                     │      3. Clear audio queue               │
                     │      4. Process user speech             │
                     │                                         │
                     └─────────────────────────────────────────┘

Configuration#

Enable/Disable Per Agent#

{
  "agent": {
    "name": "Customer Support",
    "allowInterruptions": true,
    "interruptionConfig": {
      "minSpeechDuration": 200,
      "confidenceThreshold": 0.85
    }
  }
}

Configuration Options#

Implementation#

Interruption Handler#

type InterruptionHandler struct {
    pipeline    *Pipeline
    enabled     bool
    minDuration time.Duration
    threshold   float32
    callback    func()
}

func (h *InterruptionHandler) OnVADEvent(event VADEvent) {
    if !h.enabled {
        return
    }

    if event.Type != SpeechStart {
        return
    }

    // Check if bot is currently speaking
    if !h.pipeline.IsBotSpeaking() {
        return
    }

    // Wait for minimum speech duration to avoid false positives
    time.Sleep(h.minDuration)

    // Re-check if still speaking
    if !h.pipeline.IsUserSpeaking() {
        return // Brief noise, not real interruption
    }

    // Trigger interruption
    h.triggerInterruption()
}

func (h *InterruptionHandler) triggerInterruption() {
    log.Debug("Interruption detected, stopping bot output")

    // 1. Stop audio playback immediately
    h.pipeline.ClearAudioQueue()

    // 2. Cancel LLM generation
    h.pipeline.CancelLLMGeneration()

    // 3. Stop TTS synthesis
    h.pipeline.StopTTS()

    // 4. Mark bot as not speaking
    h.pipeline.SetBotSpeaking(false)

    // 5. Enable STT for user speech
    h.pipeline.UnmuteStt()

    // 6. Notify callback
    if h.callback != nil {
        h.callback()
    }

    // Track metric
    metrics.RecordCounter("pipeline.interruptions", 1)
}

Pipeline Integration#

func (p *Pipeline) setupInterruptionHandling() {
    if !p.config.AllowInterruptions {
        return
    }

    handler := &InterruptionHandler{
        pipeline:    p,
        enabled:     true,
        minDuration: time.Duration(p.config.MinSpeechDuration) * time.Millisecond,
        threshold:   p.config.InterruptionThreshold,
        callback:    p.onInterruption,
    }

    p.vadProcessor.SetInterruptionHandler(handler)
}

func (p *Pipeline) onInterruption() {
    // Add context for LLM about interruption
    p.context.AddEvent(InterruptionEvent{
        Timestamp: time.Now(),
        BotWasSaying: p.lastBotUtterance,
    })
}

Audio Queue Management#

When interrupted, clear pending audio immediately:

type AudioQueue struct {
    queue   [][]byte
    mu      sync.Mutex
    playing bool
}

func (q *AudioQueue) Clear() {
    q.mu.Lock()
    defer q.mu.Unlock()

    q.queue = nil
    q.playing = false
}

func (q *AudioQueue) Enqueue(audio []byte) {
    q.mu.Lock()
    defer q.mu.Unlock()

    q.queue = append(q.queue, audio)
}

// In telephony provider
func (p *TwilioProvider) ClearPlayback() {
    // Clear local queue
    p.audioQueue.Clear()

    // Send clear message to Twilio
    p.sendMessage(TwilioMessage{
        Event:     "clear",
        StreamSid: p.streamSid,
    })
}

LLM Cancellation#

Cancel in-flight LLM requests:

type LLMProcessor struct {
    cancelFunc context.CancelFunc
    generating bool
}

func (l *LLMProcessor) Cancel() {
    if l.cancelFunc != nil {
        l.cancelFunc()
    }
    l.generating = false
}

func (l *LLMProcessor) Generate(ctx context.Context, messages []Message) <-chan string {
    // Create cancellable context
    ctx, l.cancelFunc = context.WithCancel(ctx)
    l.generating = true

    tokenChan := make(chan string)

    go func() {
        defer close(tokenChan)
        defer func() { l.generating = false }()

        for token := range l.llm.StreamGenerate(ctx, messages) {
            select {
            case <-ctx.Done():
                return // Cancelled
            case tokenChan <- token:
            }
        }
    }()

    return tokenChan
}

Handling Interruption Context#

Help the LLM understand the interruption:

func (p *Pipeline) processInterruption(userSpeech string) {
    // What the bot was saying when interrupted
    botContext := p.lastBotUtterance

    // Build context message for LLM
    interruptContext := fmt.Sprintf(
        "[User interrupted. Bot was saying: \"%s\". User said: \"%s\"]",
        truncate(botContext, 100),
        userSpeech,
    )

    // Add to message history
    p.context.AddMessage(Message{
        Role:    "system",
        Content: interruptContext,
    })
}

LLM Prompt for Handling Interruptions#

systemPrompt := `You are a customer support agent.

INTERRUPTION HANDLING:
- If user interrupts, acknowledge briefly and address their concern
- Don't repeat what you were saying
- Stay focused on what the user wants
- Examples:
  * "Got it, let me help with that instead."
  * "Of course, what would you like to know?"
  * "Sure, I'll look into that for you."

DO NOT:
- Say "I was saying..." or "As I was mentioning..."
- Apologize excessively for being interrupted
- Repeat the interrupted content`

Avoiding False Positives#

Filler Word Detection#

func (h *InterruptionHandler) isFillerWord(transcript string) bool {
    fillers := []string{
        "um", "uh", "hmm", "ah", "er", "like",
        "you know", "i mean", "okay", "right",
    }

    lower := strings.ToLower(strings.TrimSpace(transcript))
    for _, filler := range fillers {
        if lower == filler {
            return true
        }
    }
    return false
}

func (h *InterruptionHandler) OnTranscript(transcript TranscriptEvent) {
    if h.isFillerWord(transcript.Text) {
        return // Don't interrupt for filler words
    }

    // Proceed with interruption handling
    if transcript.IsFinal && h.pendingInterruption {
        h.confirmInterruption()
    }
}

Speech Duration Check#

func (h *InterruptionHandler) OnVADEvent(event VADEvent) {
    if event.Type == SpeechStart {
        h.speechStartTime = time.Now()
        h.pendingInterruption = true
    }

    if event.Type == SpeechEnd {
        duration := time.Since(h.speechStartTime)

        if duration < h.minDuration {
            // Too short, likely noise
            h.pendingInterruption = false
            return
        }

        // Real interruption
        h.confirmInterruption()
    }
}

Gemini Live Interruptions#

Gemini Live has built-in interruption handling:

func (g *GeminiLiveClient) HandleInterruption() error {
    // Send turn complete signal
    msg := map[string]any{
        "clientContent": map[string]any{
            "turnComplete": true,
        },
    }
    return g.conn.WriteJSON(msg)
}

// Gemini Live automatically:
// 1. Stops generating audio
// 2. Clears pending output
// 3. Listens for new user input

Best Practices#

1. Buffer Before Confirming#

Wait briefly before confirming interruption:

// Wait 200ms of continuous speech before interrupting
if speechDuration > 200*time.Millisecond {
    triggerInterruption()
}

2. Don't Interrupt During Critical Info#

func (p *Pipeline) ShouldAllowInterruption() bool {
    // Don't allow during confirmation
    if p.state == StateConfirmingAction {
        return false
    }

    // Don't allow during sensitive info
    if p.lastMessage.ContainsSensitiveInfo {
        return false
    }

    return p.config.AllowInterruptions
}

3. Track Interruption Patterns#

type InterruptionMetrics struct {
    TotalInterruptions  int
    FalsePositives      int
    AveragePosition     float64 // % into bot utterance
    CommonPhrases       map[string]int
}

func (m *InterruptionMetrics) Record(event InterruptionEvent) {
    m.TotalInterruptions++

    position := float64(event.BotPosition) / float64(event.BotTotalLength)
    m.AveragePosition = (m.AveragePosition + position) / 2

    phrase := event.UserFirstWords
    m.CommonPhrases[phrase]++
}

Troubleshooting#

Barge-in on other voice agent platforms#

Every real-time voice agent stack handles barge-in the same basic way: detect that the caller has started speaking, stop or fade out the agent's audio immediately, cancel the in-flight response and listen. Managed voice-agent APIs such as Deepgram's signal the moment the user starts speaking so the client can stop playback, and the quality difference between platforms comes from how quickly that signal arrives, how false triggers (coughs, "mm-hmm", background noise) are filtered, and whether the agent remembers what it had already said. The configuration options above control those trade-offs on Edesy.

Next Steps#

• VAD Configuration - Tune voice detection
• Turn Detection - Conversation flow
• Gemini Live - Built-in interruptions