Skip to content

Add prompt optimization guidance for turn detection to U3-Pro streaming docs#663

Open
devin-ai-integration[bot] wants to merge 4 commits intomainfrom
devin/1771947216-turn-detection-prompt-optimization
Open

Add prompt optimization guidance for turn detection to U3-Pro streaming docs#663
devin-ai-integration[bot] wants to merge 4 commits intomainfrom
devin/1771947216-turn-detection-prompt-optimization

Conversation

@devin-ai-integration
Copy link
Contributor

@devin-ai-integration devin-ai-integration bot commented Feb 24, 2026

Add prompt optimization guidance for turn detection to U3-Pro streaming docs

Summary

Adds a new #### Optimizing prompts for turn detection subsection to the "Using prompts" section of the U3-Pro streaming model selection page (/docs/universal-streaming/select-the-speech-model#using-prompts).

The new content includes:

  • A benchmark table from internal testing (100 samples) comparing the default prompt vs. prompt omitted on accuracy, recall, and false positive rate
  • An explanation of how the default prompt reduces FP rate from 34% → 24% vs. no prompt
  • Four key takeaways for prompt design (default prompt is already optimized, concrete rules work best, filler word instructions are high-signal, recall is near-perfect across all approaches)

This was requested based on learnings from internal turn detection evaluation runs.

Updates since last revision

  • Renamed "Strict punctuation rules (default)" to just "Default" in the benchmark table
  • Renamed "No prompt" to "Prompt omitted" to clarify it means the prompt parameter was omitted entirely, not that the default prompt was used
  • Removed the "Turn-signal framing" row from the table per reviewer feedback
  • Removed the #default-prompt hyperlink from the explanatory text
  • Simplified the second key takeaway to be more general (no longer references the removed row)
  • Reworded the first key takeaway to clearly distinguish "prompt omitted entirely" (83%) from "built-in default prompt" (88%)

Review & Testing Checklist for Human

  • Verify benchmark numbers are approved for public docs. The accuracy/recall/FP data comes from an internal Slack discussion — confirm these are okay to publish externally.
  • Check the "default prompt" vs "prompt omitted" language is clear to external readers. The distinction between "the built-in default prompt" and "omitting the prompt parameter entirely" may still be confusing to users unfamiliar with the system internals. Read through the section fresh and confirm it makes sense.
  • Review rendered output in preview deployment. Confirm the markdown table, bold text, and numbered list render correctly in the Fern docs site. Check heading hierarchy (#### under ###) looks right in the sidebar/TOC.

Suggested test plan: Open the preview deployment URL and navigate to the "Using prompts" section. Verify the new "Optimizing prompts for turn detection" subsection appears with a properly formatted two-row table and numbered list. Confirm the section reads naturally in context with the surrounding content.

Notes

@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions
Copy link

@github-actions
Copy link

@github-actions
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants