Automating support with AI without destroying CSAT

We’ve deployed AI-assisted support triage for eleven clients in the past eighteen months. The ones that went badly all made the same mistake: they let the AI resolve too much, and CSAT dropped before anyone noticed.

The right division of labour

AI is genuinely good at: reading incoming tickets, classifying them by type and urgency, pulling relevant order data, drafting a first response, and routing to the right human. It is not good at: handling complaints from upset customers, resolving disputes, processing refunds without policy checks, or anything that requires judgement about edge cases.

The line we draw for every client: AI classifies and drafts, human reviews and sends — except for the lowest-stakes ticket types (order status enquiries with clear answers, shipping tracking requests, password resets) where we allow fully automated responses.

The build

Intake classification

Every incoming ticket gets classified into one of eight categories: order status, shipping, returns, product question, complaint, technical issue, billing, and other. Classification accuracy runs above 94% on the stores we’ve built this for.

Data enrichment

For order-related tickets, the system pulls order data, tracking status, and previous support history automatically and attaches it to the ticket before a human sees it. This alone cuts average handle time by 40%.

Draft generation

For classified ticket types with clear resolution paths, the AI drafts a response using the order data and a set of approved response templates. The human reviews, edits if needed, and sends. Not auto-sends — reviews and sends.

Escalation rules

Tickets containing specific signals — refund, lawyer, fraud, wrong, terrible, disgusting, not received — skip AI drafting and go straight to a senior support agent queue. Do not pass go.

What to measure

CSAT (obviously), first response time, average handle time, and escalation rate. If escalation rate goes up after launch, the classification is putting things in the wrong bucket. Fix the classifier, not the escalation rules.

Results across our clients

Average handle time down 38%. First response time down 61%. CSAT unchanged or slightly up (faster responses help). Support headcount flat despite growing ticket volume. Roughly the outcome you’d want.