From 6188c36e0ef550880ef0e2ad375270f792ac48e2 Mon Sep 17 00:00:00 2001 From: Justin Dunham Date: Thu, 30 Jul 2026 09:21:48 -0700 Subject: [PATCH 1/2] Add learn page on improving support-ticket deflection with docs. --- ...-improve-support-ticket-deflection-docs.md | 61 +++++++++++++++++++ learn/ai-for-docs/sidebars.yaml | 2 + 2 files changed, 63 insertions(+) create mode 100644 learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md diff --git a/learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md b/learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md new file mode 100644 index 000000000..40e20f803 --- /dev/null +++ b/learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md @@ -0,0 +1,61 @@ +--- +seo: + title: Use AI to improve support-ticket deflection with your docs + description: Use AI to find where developers get stuck in your docs before they file a ticket, then close what's missing with search, FAQs, and troubleshooting content that measurably reduces support volume. +--- + +# Use AI to improve support-ticket deflection with your docs + +A support queue full of the same three questions is rarely a support problem. It is usually a documentation problem wearing a support ticket as a disguise, because the answer already exists somewhere in the docs, just not where a developer could find it fast enough to skip the ticket. + +AI changes how quickly a team can catch that pattern. Instead of waiting for a support lead to notice a spike, you can point AI at your support logs and your docs together and ask where the two disagree. This article walks through how to use AI to read that signal, write what is missing, and measure whether the fix reduced ticket volume. + +## What deflection means for API docs + +At Redocly, we track "deflection" as one of three developer experience metrics, alongside acquisition and adoption. [Key Metrics for Docs](https://redocly.com/blog/key-metrics-for-docs) defines it plainly: deflection means a developer finds the answer in your docs or search before they ever open a ticket, so the support team spends its time on real problems instead of repeat questions. + +The metric is concrete, not a vibe. You can track it as the reduction in support tickets per integration, the percentage of developer questions answered through docs or search, and satisfaction scores on your self-service flows. None of those numbers move by themselves, though. They move once you find the specific pages, missing sections, or confusing examples that are pushing developers toward the contact form instead of the page that was supposed to help them. + +That is where AI earns its place in the workflow. A support team can read a handful of tickets and notice a trend, but it cannot read every ticket, every search query, and every doc page in the time it takes a queue to grow. AI can, and it can do it every week instead of during an occasional review. + +## Where AI fits into deflection work + +Two AI-assisted practices do most of the work here: reading support and search data for patterns, and testing docs against the tasks a stuck developer needs to finish. AI can spot the pattern faster than a person reading tickets one at a time, but a writer should still confirm each fix before it ships, because AI can misread a ticket's real cause and point you at the wrong page. + +### Mining support logs for missing content + +Feed AI a sample of recent support tickets alongside your current docs, then ask it to sort each ticket into one of three buckets: the answer already exists in the docs, the answer exists but is hard to find, or the answer does not exist yet. That third bucket is where you write new content. The first two point to search and navigation problems instead, which calls for a different fix. + +Redocly's own dashboards do a version of this at the topic level. An internal example tracked weekly searches, doc reads, and support conversations for a single feature, single sign-on, side by side, so the team could see exactly where self-service was breaking down for that one topic ([Q2 2025 updates](https://redocly.com/blog/updates-2025-05)). You do not need custom dashboards to start this practice; a spreadsheet with ticket subject, matching doc page, and outcome will surface the same pattern. + +### Testing docs the way a stuck developer would + +Reading tickets tells you what already went wrong. Testing tells you what is about to go wrong next. [Use AI as a usability tester](https://redocly.com/learn/ai-for-docs/ai-usability-testing) describes giving AI a real task, like setting up a webhook and verifying its signature, using only your public docs, then watching where it asks a clarifying question or gets stuck completely. Those stalls are the same moments that would otherwise turn into a ticket. + +The method comes from Redocly's internal Phronesis practice, where employees work through real customer workflows using only the published docs each week. That human version of the test led to a 630 percent increase in free trial conversions by surfacing where the docs were confusing or incomplete ([Phronesis](https://redocly.com/blog/phronesis)). AI can run the same test at a scale no team could staff on its own, continuously rather than once a quarter. + +## Turning AI findings into content developers can search + +Finding the missing content is half the job. Developers still need to find it before they give up and file a ticket, and a fix that nobody can search for does not deflect anything. + +Write new sections around the task a developer is trying to finish, not the internal name for the feature: "verify a webhook signature," not "signing configuration." [Use AI to help developers find and understand your APIs faster](https://redocly.com/learn/ai-for-docs/ai-help-developers-find-understand-apis) makes the same point about onboarding docs, and it applies just as well to troubleshooting content, because assistants and search both retrieve short, specifically titled sections more reliably than long pages that bury the answer halfway down. + +When the missing piece is at the spec level instead, such as an undocumented error response, pair this work with [Use AI to find gaps in your documentation coverage](https://redocly.com/learn/ai-for-docs/ai-find-gaps-documentation-coverage), since that side of the problem needs a different kind of check than a support-log review. + +## Measuring whether deflection is working + +Deflection has a reactive side and a proactive side, and a healthy program watches both. Reactively, watch for a spike in tickets tagged to one topic right after a release, because that usually means a change broke something the docs have not caught up with yet. Proactively, track the steady reduction in tickets about developers not being able to find the docs at all, since that is the number that shows self-service is replacing support over time ([Beyond anomaly detection](https://redocly.com/blog/beyond-anomaly-detection)). + +Pick one topic to start, the way Redocly's single sign-on example did, and track three numbers together: searches for that topic, page reads, and tickets. If reads and searches climb while tickets for that topic fall, the new content is doing its job. If tickets stay flat instead, the content probably is not surfacing where developers are looking, so send AI back to check search and navigation before writing anything new. + +## Best practices for an AI-assisted deflection loop + +- Run AI over your support queue monthly, not only after a bad week, so drift shows up before it becomes a spike. +- Test five to ten critical tasks with AI on every major doc or product change, because a page that passed last month may not pass after a release. +- Write new sections with the exact task language developers use in tickets and search queries, not the internal name for the feature. +- Track tickets, searches, and page reads together for at least one topic before expanding the practice, so you can show the loop works before asking for more support-team time. +- Treat a ticket AI could already answer from your docs as a signal to move that answer higher on the page or into the search index, instead of a ticket to close and forget. + +## How Redocly can help + +Support-ticket deflection lives or dies on whether a developer can search their way to an answer before reaching for the contact form, and that is exactly what [Revel](https://redocly.com/revel) is built for. As the external developer portal where partners and customers onboard, Revel pairs hosted search and AI assistant features with your published docs, so the fixes an AI-assisted deflection loop turns up have somewhere to surface the moment a developer needs them, instead of sitting on a page nobody scrolls to. diff --git a/learn/ai-for-docs/sidebars.yaml b/learn/ai-for-docs/sidebars.yaml index 7dd3c8899..b197ad99a 100644 --- a/learn/ai-for-docs/sidebars.yaml +++ b/learn/ai-for-docs/sidebars.yaml @@ -30,6 +30,8 @@ label: Use AI to build a searchable API catalog for your team - page: ai-help-developers-find-understand-apis.md label: Use AI to help developers find and understand your APIs faster +- page: ai-improve-support-ticket-deflection-docs.md + label: Use AI to improve support-ticket deflection with your docs - page: ai-check-terminology-consistency-across-docs.md label: Use AI to check terminology consistency across your docs - page: ai-usability-testing.md From ab6b836dad3ca2423df425145524488ced3a1d50 Mon Sep 17 00:00:00 2001 From: Justin Dunham Date: Thu, 30 Jul 2026 11:30:59 -0700 Subject: [PATCH 2/2] Refresh learn/ai-for-docs draft from latest Sonnet 5 run. --- ...-improve-support-ticket-deflection-docs.md | 62 ++++++++++--------- 1 file changed, 32 insertions(+), 30 deletions(-) diff --git a/learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md b/learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md index 40e20f803..22197feee 100644 --- a/learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md +++ b/learn/ai-for-docs/ai-improve-support-ticket-deflection-docs.md @@ -1,61 +1,63 @@ --- seo: title: Use AI to improve support-ticket deflection with your docs - description: Use AI to find where developers get stuck in your docs before they file a ticket, then close what's missing with search, FAQs, and troubleshooting content that measurably reduces support volume. + description: How to use AI to find the doc pages behind repeat support tickets, fix the real cause, and measure whether deflection improves. --- # Use AI to improve support-ticket deflection with your docs -A support queue full of the same three questions is rarely a support problem. It is usually a documentation problem wearing a support ticket as a disguise, because the answer already exists somewhere in the docs, just not where a developer could find it fast enough to skip the ticket. +Most support teams answer the same handful of questions every week, and most of those questions already have an answer somewhere in the docs. The problem usually is not missing content. It is content that nobody can find, trust, or apply at the moment they need it. -AI changes how quickly a team can catch that pattern. Instead of waiting for a support lead to notice a spike, you can point AI at your support logs and your docs together and ask where the two disagree. This article walks through how to use AI to read that signal, write what is missing, and measure whether the fix reduced ticket volume. +That outcome has a name: "deflection," the share of developer questions resolved without a ticket. This article covers how to use AI to find the doc pages that generate tickets without anyone noticing, fix what is really wrong with them, and confirm the fix moved the number instead of just feeling better. -## What deflection means for API docs +## What deflection means for your docs -At Redocly, we track "deflection" as one of three developer experience metrics, alongside acquisition and adoption. [Key Metrics for Docs](https://redocly.com/blog/key-metrics-for-docs) defines it plainly: deflection means a developer finds the answer in your docs or search before they ever open a ticket, so the support team spends its time on real problems instead of repeat questions. +Deflection sits alongside acquisition and adoption as one of three measurable outcomes for a documentation program, according to [key metrics for docs](https://redocly.com/blog/key-metrics-for-docs). Acquisition brings developers to your site, adoption gets them integrating, and deflection measures whether they can keep solving problems on their own once they are in production. When deflection works, a mature API program frees up support to handle harder problems instead of repeating the getting-started guide. -The metric is concrete, not a vibe. You can track it as the reduction in support tickets per integration, the percentage of developer questions answered through docs or search, and satisfaction scores on your self-service flows. None of those numbers move by themselves, though. They move once you find the specific pages, missing sections, or confusing examples that are pushing developers toward the contact form instead of the page that was supposed to help them. +Reduction in tickets tagged something like "can't find docs" is itself a trackable goal, tracked the same way you would track any product metric ([beyond anomaly detection](https://redocly.com/blog/beyond-anomaly-detection)). Effective deflection is not about avoiding support: it is about helping developers move faster and succeed without waiting on a reply. So a rising ticket count on a topic is not a support problem first. It is a docs signal. -That is where AI earns its place in the workflow. A support team can read a handful of tickets and notice a trend, but it cannot read every ticket, every search query, and every doc page in the time it takes a queue to grow. AI can, and it can do it every week instead of during an occasional review. +## Find the tickets your docs should already answer -## Where AI fits into deflection work +Start with the tickets themselves, not the docs. Export a sample of resolved tickets, including whatever doc link an agent eventually sent, and ask an AI model to cluster them by the underlying question rather than by product area or severity. A prompt like this works well as a starting point: -Two AI-assisted practices do most of the work here: reading support and search data for patterns, and testing docs against the tasks a stuck developer needs to finish. AI can spot the pattern faster than a person reading tickets one at a time, but a writer should still confirm each fix before it ships, because AI can misread a ticket's real cause and point you at the wrong page. +```markdown +You are reviewing 200 resolved support tickets for a REST API. +Group them by the underlying developer question, not by the +product area or ticket tag. For each cluster, list: +1. The question in plain language +2. How many tickets match it +3. Which doc page (if any) an agent linked in the resolution +4. Whether that page should have answered the question already +``` -### Mining support logs for missing content +The clusters usually split into two kinds. Some point to a page that exists but fails, because it buries a prerequisite, uses a stale example, or answers a different question than the one developers meant to ask. Others point to nothing: no page addresses the workflow at all. Both are worth fixing, but they call for different work, so keep them in separate lists before you touch a single doc. -Feed AI a sample of recent support tickets alongside your current docs, then ask it to sort each ticket into one of three buckets: the answer already exists in the docs, the answer exists but is hard to find, or the answer does not exist yet. That third bucket is where you write new content. The first two point to search and navigation problems instead, which calls for a different fix. +## Turn ticket clusters into doc fixes -Redocly's own dashboards do a version of this at the topic level. An internal example tracked weekly searches, doc reads, and support conversations for a single feature, single sign-on, side by side, so the team could see exactly where self-service was breaking down for that one topic ([Q2 2025 updates](https://redocly.com/blog/updates-2025-05)). You do not need custom dashboards to start this practice; a spreadsheet with ticket subject, matching doc page, and outcome will surface the same pattern. +When a page exists but underperforms, the fix is rarely "write more words." [Use AI to help developers find and understand your APIs faster](https://redocly.com/learn/ai-for-docs/ai-help-developers-find-understand-apis) makes the case for organizing pages around a single task, with prerequisites and errors on the same page as the operation they belong to, so a developer under deadline pressure does not have to piece an answer together from three tabs. -### Testing docs the way a stuck developer would +Two causes are easy to miss because they live outside the words on the page itself. First, drift: when your live API changes and the docs do not, [use AI to detect drift between your docs and your live API](https://redocly.com/learn/ai-for-docs/ai-detect-drift-docs-live-api) shows up in support as a ticket that reads "your sample returns 404" rather than a docs complaint. Second, AI-readability: when an assistant pulls a confusing or outdated fragment and hands a developer a wrong answer with confidence, that also lands back in your queue, a pattern covered in [optimizations to make to your docs for LLMs](https://redocly.com/blog/optimizations-to-make-to-your-docs-for-llms). -Reading tickets tells you what already went wrong. Testing tells you what is about to go wrong next. [Use AI as a usability tester](https://redocly.com/learn/ai-for-docs/ai-usability-testing) describes giving AI a real task, like setting up a webhook and verifying its signature, using only your public docs, then watching where it asks a clarifying question or gets stuck completely. Those stalls are the same moments that would otherwise turn into a ticket. +Small naming choices matter too. Unclear property names in a schema, like `usrNm` instead of `username`, lead to support tickets and frustrated developers on their own, independent of how well the surrounding paragraph reads ([use AI to accelerate and improve reviews](https://redocly.com/learn/ai-for-docs/ai-reviews)). Run AI review on both the words and the schema behind them, because a ticket cluster about "the field names are confusing" is a review problem, not a writing problem. -The method comes from Redocly's internal Phronesis practice, where employees work through real customer workflows using only the published docs each week. That human version of the test led to a 630 percent increase in free trial conversions by surfacing where the docs were confusing or incomplete ([Phronesis](https://redocly.com/blog/phronesis)). AI can run the same test at a scale no team could staff on its own, continuously rather than once a quarter. +## Where AI assistants fit in the deflection loop -## Turning AI findings into content developers can search +At Redocly, we use an internal AI Assistant and MCP servers as a first line of support before a question ever reaches a subject-matter expert, answering routine questions about internal components and APIs directly ([how AI fits into modern API documentation](https://redocly.com/learn/ai-for-docs/ai-modern-api-docs)). That pattern deflects a real share of questions, but only when the assistant retrieves from pages that are current and well scoped. -Finding the missing content is half the job. Developers still need to find it before they give up and file a ticket, and a fix that nobody can search for does not deflect anything. +An assistant sitting in front of a wrong or outdated page does not remove a ticket. It just delays it and adds a wrong answer to the transcript before a human gets involved. Treat the assistant as a lever on top of good docs, not a replacement for fixing the page underneath it. -Write new sections around the task a developer is trying to finish, not the internal name for the feature: "verify a webhook signature," not "signing configuration." [Use AI to help developers find and understand your APIs faster](https://redocly.com/learn/ai-for-docs/ai-help-developers-find-understand-apis) makes the same point about onboarding docs, and it applies just as well to troubleshooting content, because assistants and search both retrieve short, specifically titled sections more reliably than long pages that bury the answer halfway down. +## Measure whether deflection improved -When the missing piece is at the spec level instead, such as an undocumented error response, pair this work with [Use AI to find gaps in your documentation coverage](https://redocly.com/learn/ai-for-docs/ai-find-gaps-documentation-coverage), since that side of the problem needs a different kind of check than a support-log review. +Before you credit an AI-assisted fix, set a baseline. Track ticket volume for the affected question for a few weeks before you ship anything, then compare the same window after. Useful signals include the reduction in support tickets per integration, the percentage of developer questions answered through docs or search instead of a ticket, and satisfaction scores on self-service flows ([key metrics for docs](https://redocly.com/blog/key-metrics-for-docs)). -## Measuring whether deflection is working +Watch time-to-first-successful-call as well, not just page views, since a page can get more traffic while still failing to get anyone unstuck. Re-test the same onboarding tasks after any navigation change, because a fix that helps one cluster can bury the page another cluster needed without anyone catching it right away ([use AI to help developers find and understand your APIs faster](https://redocly.com/learn/ai-for-docs/ai-help-developers-find-understand-apis)). When the number moves, publish it next to the docs roadmap. That is what turns deflection from a nice idea into a budget line your support team will keep funding. -Deflection has a reactive side and a proactive side, and a healthy program watches both. Reactively, watch for a spike in tickets tagged to one topic right after a release, because that usually means a change broke something the docs have not caught up with yet. Proactively, track the steady reduction in tickets about developers not being able to find the docs at all, since that is the number that shows self-service is replacing support over time ([Beyond anomaly detection](https://redocly.com/blog/beyond-anomaly-detection)). +## Build a short list before you commit engineering time -Pick one topic to start, the way Redocly's single sign-on example did, and track three numbers together: searches for that topic, page reads, and tickets. If reads and searches climb while tickets for that topic fall, the new content is doing its job. If tickets stay flat instead, the content probably is not surfacing where developers are looking, so send AI back to check search and navigation before writing anything new. +Not every cluster deserves the same response. Rank clusters by ticket volume and by how many production accounts they touch, then estimate the effort each fix needs: a rewritten paragraph, a moved section, or a new page entirely. A cluster with fifty tickets and a one-paragraph fix should ship before a cluster with five tickets and a rewrite that spans three pages, even if the second cluster feels more interesting to fix. -## Best practices for an AI-assisted deflection loop - -- Run AI over your support queue monthly, not only after a bad week, so drift shows up before it becomes a spike. -- Test five to ten critical tasks with AI on every major doc or product change, because a page that passed last month may not pass after a release. -- Write new sections with the exact task language developers use in tickets and search queries, not the internal name for the feature. -- Track tickets, searches, and page reads together for at least one topic before expanding the practice, so you can show the loop works before asking for more support-team time. -- Treat a ticket AI could already answer from your docs as a signal to move that answer higher on the page or into the search index, instead of a ticket to close and forget. +Share the ranked list with support leads before you start writing. They usually know which questions repeat because a customer is confused, and which repeat because the underlying workflow genuinely has no good answer yet. That distinction changes whether AI should help you rewrite a page or flag a missing product capability for engineering instead. ## How Redocly can help -Support-ticket deflection lives or dies on whether a developer can search their way to an answer before reaching for the contact form, and that is exactly what [Revel](https://redocly.com/revel) is built for. As the external developer portal where partners and customers onboard, Revel pairs hosted search and AI assistant features with your published docs, so the fixes an AI-assisted deflection loop turns up have somewhere to surface the moment a developer needs them, instead of sitting on a page nobody scrolls to. +When ticket clusters point to a findability problem rather than a missing page, the fix depends on where developers look for answers first. [Revel](https://redocly.com/revel) is the external developer portal built for that kind of self-service discovery, with search and quickstarts positioned so partners and customers can resolve a question without filing one. Pairing that public surface with [Reef](https://redocly.com/reef) as an internal catalog keeps support and docs teams working from the same current source, so the fix you make after reading a ticket cluster reaches readers before the next round of tickets arrives.