Turn detection determines when the user has finished speaking and it's the bot's turn to respond. Getting this right is crucial for natural conversations.

What is Turn Detection?#

Natural Conversation Flow:
─────────────────────────────────────────────────────────────
User: "I want to check my order status" ─────────────┐
                                                      │ Turn boundary
Bot: "Sure, what's your order number?" ◄─────────────┘
                                        ─────────────┐
User: "It's 12345" ──────────────────────────────────┤ Turn boundary
                                                      │
Bot: "Your order has shipped..." ◄───────────────────┘

Poor Turn Detection:
─────────────────────────────────────────────────────────────
User: "I want to check my—"
Bot: [Interrupts] "How can I help?" ← Bot spoke too early

User: "I want to check my order status"
     [3 second pause]
Bot: "Sure, what's your order number?" ← Bot spoke too late

Components of Turn Detection#

                    ┌─────────────────────────────────────────┐
                    │           Turn Detection                │
                    │                                         │
User Audio ────────►│  VAD ──► Endpointing ──► Confirmation  │────► End of Turn
                    │   │          │               │          │
                    │   ▼          ▼               ▼          │
                    │ Speech   Silence        Semantic        │
                    │ Prob.    Duration       Analysis        │
                    │                                         │
                    └─────────────────────────────────────────┘

1. VAD (Voice Activity Detection)#

Detects speech vs silence:

type VADResult struct {
    IsSpeech    bool
    Probability float32
    Timestamp   time.Time
}

// VAD emits: speech_start, speech_end events

2. Endpointing#

Determines when speech has ended:

type EndpointingConfig struct {
    MinSilenceDuration time.Duration // Silence to trigger end
    MaxSpeechDuration  time.Duration // Maximum turn length
    VolumeThreshold    float32       // Minimum audio level
}

3. Semantic Confirmation#

Uses STT and context for smarter detection:

type SemanticTurnDetector struct {
    stt           STTProvider
    pendingText   string
    lastWordTime  time.Time
}

func (d *SemanticTurnDetector) IsCompleteTurn(transcript string) bool {
    // Check for complete sentence
    if endsWithPunctuation(transcript) {
        return true
    }

    // Check for question patterns
    if startsWithQuestion(transcript) && len(transcript) > 20 {
        return true
    }

    // Check for trailing silence after content
    if time.Since(d.lastWordTime) > 500*time.Millisecond {
        return true
    }

    return false
}

Configuration#

Basic Configuration#

{
  "agent": {
    "turnDetection": {
      "mode": "vad",
      "vadThreshold": 0.8,
      "silenceDuration": 300,
      "maxTurnDuration": 30000
    }
  }
}

Mode Options#

Mode	Description	Latency	Accuracy
`vad`	VAD + silence timer	⚡ Fastest	Good
`semantic`	VAD + STT analysis	🚀 Fast	Better
`hybrid`	Combines both	🚀 Fast	Best

VAD Mode#

Simple silence-based detection:

func (d *VADTurnDetector) OnVADEvent(event VADEvent) {
    switch event.Type {
    case SpeechStart:
        d.turnStartTime = time.Now()
        d.isSpeaking = true

    case SpeechEnd:
        // Wait for configured silence duration
        time.AfterFunc(d.silenceDuration, func() {
            if !d.isSpeaking {
                d.emitEndOfTurn()
            }
        })
        d.isSpeaking = false
    }
}

Semantic Mode#

Uses transcript content for smarter detection:

func (d *SemanticTurnDetector) OnTranscript(event TranscriptEvent) {
    d.pendingText = event.Text
    d.lastWordTime = time.Now()

    if event.IsFinal {
        // Final transcript from STT endpointing
        d.emitEndOfTurn()
        return
    }

    // Analyze for completeness
    if d.isSemanticComplete(event.Text) {
        // Give brief pause for continuation
        time.AfterFunc(200*time.Millisecond, func() {
            if d.pendingText == event.Text {
                d.emitEndOfTurn()
            }
        })
    }
}

func (d *SemanticTurnDetector) isSemanticComplete(text string) bool {
    text = strings.TrimSpace(text)

    // Ends with punctuation
    if strings.HasSuffix(text, ".") ||
       strings.HasSuffix(text, "?") ||
       strings.HasSuffix(text, "!") {
        return true
    }

    // Short affirmative/negative responses
    shortResponses := []string{"yes", "no", "okay", "sure", "thanks", "bye"}
    lower := strings.ToLower(text)
    for _, resp := range shortResponses {
        if lower == resp {
            return true
        }
    }

    return false
}

Hybrid Mode#

Combines VAD and semantic analysis:

type HybridTurnDetector struct {
    vadDetector      *VADTurnDetector
    semanticDetector *SemanticTurnDetector
    pendingEndOfTurn bool
}

func (d *HybridTurnDetector) Process(event any) {
    switch e := event.(type) {
    case VADEvent:
        d.vadDetector.OnVADEvent(e)

        if e.Type == SpeechEnd {
            // VAD says speech ended, check semantic
            if d.semanticDetector.isSemanticComplete(d.pendingText) {
                d.emitEndOfTurn()
            } else {
                d.pendingEndOfTurn = true
            }
        }

    case TranscriptEvent:
        d.semanticDetector.OnTranscript(e)

        if d.pendingEndOfTurn && d.semanticDetector.isSemanticComplete(e.Text) {
            d.emitEndOfTurn()
            d.pendingEndOfTurn = false
        }
    }
}

STT Endpointing Integration#

Leverage STT provider's endpointing:

Deepgram#

{
  "sttConfig": {
    "endpointing": 300,
    "utterance_end_ms": 1000,
    "interim_results": true
  }
}

Google#

{
  "sttConfig": {
    "singleUtterance": false,
    "voiceActivityTimeout": {
      "speechEndTimeout": "1s"
    }
  }
}

Use Case Configurations#

Fast-Paced Support#

Quick responses for simple queries:

{
  "turnDetection": {
    "mode": "vad",
    "silenceDuration": 200,
    "vadThreshold": 0.75
  }
}

Thoughtful Conversations#

Allow pauses for complex topics:

{
  "turnDetection": {
    "mode": "semantic",
    "silenceDuration": 500,
    "allowThinkingPauses": true
  }
}

Elderly/Accessibility#

More patient turn detection:

{
  "turnDetection": {
    "mode": "hybrid",
    "silenceDuration": 700,
    "vadThreshold": 0.85,
    "maxTurnDuration": 60000
  }
}

IVR/Commands#

Quick command detection:

{
  "turnDetection": {
    "mode": "vad",
    "silenceDuration": 150,
    "shortResponseMode": true
  }
}

Handling Edge Cases#

Trailing Filler Words#

func (d *SemanticTurnDetector) stripFillers(text string) string {
    fillers := []string{" um", " uh", " like", " you know", " so"}
    for _, filler := range fillers {
        text = strings.TrimSuffix(text, filler)
    }
    return text
}

func (d *SemanticTurnDetector) isSemanticComplete(text string) bool {
    // Strip trailing fillers before checking
    text = d.stripFillers(text)
    // ... rest of logic
}

Multi-Sentence Turns#

func (d *SemanticTurnDetector) isMultiSentenceTurn(text string) bool {
    sentences := splitSentences(text)

    // If first sentence is a question, they might continue
    if len(sentences) > 0 && isQuestion(sentences[0]) {
        return false // Wait for more
    }

    // If we have 2+ complete sentences, probably done
    if len(sentences) >= 2 {
        return true
    }

    return false
}

Phone Number Dictation#

func (d *SemanticTurnDetector) isPhoneNumberComplete(text string) bool {
    // Extract digits
    digits := extractDigits(text)

    // US phone: 10 digits
    if len(digits) == 10 {
        return true
    }

    // International: 11-15 digits
    if len(digits) >= 11 && len(digits) <= 15 {
        return true
    }

    return false
}

Metrics and Debugging#

Turn Detection Metrics#

type TurnMetrics struct {
    TurnDurations    []time.Duration
    SilenceBeforeTurn []time.Duration
    InterruptedTurns int
    FalseEndpoints   int
    MissedEndpoints  int
}

func (m *TurnMetrics) Record(turn TurnEvent) {
    m.TurnDurations = append(m.TurnDurations, turn.Duration)
    m.SilenceBeforeTurn = append(m.SilenceBeforeTurn, turn.SilenceBeforeResponse)

    if turn.WasInterrupted {
        m.InterruptedTurns++
    }
}

func (m *TurnMetrics) Analyze() TurnAnalysis {
    return TurnAnalysis{
        AvgTurnDuration:    average(m.TurnDurations),
        AvgSilenceBeforeResponse: average(m.SilenceBeforeTurn),
        InterruptionRate:   float64(m.InterruptedTurns) / float64(len(m.TurnDurations)),
    }
}

Debug Logging#

func (d *TurnDetector) SetDebugMode(enabled bool) {
    d.debug = enabled
}

func (d *TurnDetector) debugLog(format string, args ...any) {
    if d.debug {
        log.Printf("[TurnDetection] "+format, args...)
    }
}

// Output:
// [TurnDetection] VAD: speech_start at 0ms
// [TurnDetection] Interim transcript: "What is my"
// [TurnDetection] Interim transcript: "What is my order"
// [TurnDetection] VAD: speech_end at 1500ms
// [TurnDetection] Silence timer started: 300ms
// [TurnDetection] Final transcript: "What is my order status"
// [TurnDetection] Semantic: complete sentence detected
// [TurnDetection] End of turn emitted at 1800ms

Best Practices#

1. Start Conservative#

// Start with longer silence duration
config := TurnDetectionConfig{
    SilenceDuration: 400 * time.Millisecond,
    VADThreshold:    0.8,
}

// Tune based on metrics
if avgInterruptionRate > 0.1 {
    config.SilenceDuration += 100 * time.Millisecond
}

2. Context-Aware Adjustment#

func (d *TurnDetector) adjustForContext(context *ConversationContext) {
    // Shorter patience after bot asks question
    if context.LastBotMessageWasQuestion {
        d.silenceDuration = 250 * time.Millisecond
    }

    // Longer patience for complex topics
    if context.Topic == "technical_support" {
        d.silenceDuration = 500 * time.Millisecond
    }
}

3. Recover from Errors#

func (d *TurnDetector) handlePrematureEnd() {
    // If user continues speaking right after we ended turn
    if d.userSpeakingWithin(200 * time.Millisecond) {
        d.cancelCurrentResponse()
        d.resumeListening()
        d.silenceDuration += 100 * time.Millisecond // Be more patient
    }
}

Next Steps#

VAD Configuration - Voice activity detection
Interruptions - Handle barge-in
Latency Optimization - Reduce response time