Pilot to production
A 90-day plan to take a stalled AI pilot to every agent who wants it on a 220-agent customer care team. The company is invented. The method is how I work.
Fictional · every name, number and detail is made upFix what stalled the pilot, then scale.
Larkspur's pilot worked for the agents who kept using it. Six of fifteen stopped by week four. The drafts missed the company's tone, nobody owned the tool, the data rules were never written down, and a rumor said the pilot was really a headcount study.
This plan fixes those first: a named owner, written data rules, a jobs statement from the sponsor, and fourteen agent champions. Then it rolls out in two waves over 90 days.
At day 90 the sponsor makes one call, using numbers agreed in week one: scale to dispatch and sales, fix and re-measure, or stop.
A pilot that worked, then stalled
- Company
- Larkspur Home Services, a fictional regional repair company: plumbing, heating and cooling, electrical.
- Size
- 1,900 employees across 38 branches.
- Customer care
- 220 agents and 8 team leads at two sites. Scheduling, billing questions and complaints by phone, email and chat.
- The pilot
- 15 agents for 6 weeks. An enterprise AI assistant drafting email and chat replies, summarizing calls and searching the policy knowledge base.
What the pilot showed
- Agents who kept using it said call notes took noticeably less time.
- Six of fifteen stopped by week four. Drafts ignored Larkspur's tone rules and quoted an outdated knowledge base.
- A rumor spread that the pilot was measuring who to cut. Nobody in leadership answered it.
- The assistant wasn't approved to connect to the ticketing system, so some agents pasted customer details into personal AI accounts.
- When the pilot ended, nobody owned the tool.
Lowest areas: staff trust, data and privacy. Try the scorer
The goal: by day 90, every agent who wants the assistant uses it safely, and Larkspur knows whether it helps.
90 days, four phases
Discover
Interviews with every group on the stakeholder map. Ticket data, pilot usage and a fresh readiness score. Baselines for after-call work, first reply time, satisfaction and escalations pulled per team for the eight weeks before the pilot, and frozen.
Set foundations
Owner named. Data rules and jobs statement written. Ticketing connection approved; if it isn't by week 4, wave 1 waits. Champions chosen and trained. Cost case drafted: licenses, protected hours, training time.
Wave 1
80 agents on three teams, including one skeptical team on purpose. Metrics reviewed weekly.
Wave 2 and handoff
The remaining 125 agents. Operations takes over the day-to-day. Day-90 review with the sponsor in week 13.
The questions come before the tools
Two weeks of conversations, grouped by who is in the room. The agents' answers carry the most weight, because they decide whether this gets used.
- What will you decide at day 90, and what would you need to see to decide it?
- If this works, what do agents get back with the time saved?
- What have staff been told about jobs, and will you put it in writing?
- Who pays for this after year one?
- Which ticket types take the most time today?
- Who on your team already uses AI on their own?
- What would make you want to switch it off?
- Walk me through your last hard ticket. Where did the time go?
- In the pilot, where did the assistant help, and where did it get in the way?
- What should it never do?
- Which systems would the assistant read from, and who approves each one?
- How does access end when someone leaves?
- What do we log today, and who looks at it?
- What do the vendor's terms say about training on our data and where it is stored?
- What customer data is off-limits for the tool?
- How long should conversation records be kept?
- Do customers need to be told when a reply was drafted with AI?
- Does our call-recording notice already cover transcription and AI summaries?
Who decides, and who lives with it
Plotted by influence over the decision and stake in the outcome. Each group carries the one thing it needs from this rollout.
Keep satisfied
- IT DirectorNo new tools that IT never approved.
- Security and privacy leadClear data lines and a record of who did what.
- Legal counselA disclosure decision on record.
- Finance partnerA cost case past year one.
Partner closely
- VP Customer Experience, sponsorProof it helps before anything scales.
- Support Operations Manager, program ownerProtected time and real authority to run it.
Keep informed
- Dispatch and sales leadsA playbook they can reuse in the next wave.
- Branch managersTo know what customers might see.
Bring in early
- Agents, 220A real say, and a straight answer on jobs.
- Team leads, 8No penalty on handle-time targets while their teams learn.
- HR partnerTraining inside paid hours, and fair reviews.
Agents have the least formal power on this map and the most real power. They decide whether the tool gets used.
Who runs it after launch
Three layers, light enough that a 220-agent team can keep it going. For the 90 days I run the weekly layer with the program owner; from week 9 the owner runs it and I observe. Handoff is done when the owner has run three weeks without me.
Steering group
VP Customer Experience (chair), IT Director, security and privacy lead, legal counsel, HR partner, Finance partner, program owner.
Decides new use cases and policy changes, and hears the day-90 evidence. The sponsor, as chair, makes the day-90 call.
Program owner
The Support Operations Manager, a day a week during the rollout, four hours a week after handoff.
Owns the rollout plan, metrics, training, the champion network, the vendor relationship, and the policy knowledge base, reviewed monthly with a named author per section.
Champions
14 agents across both sites, about one per 15 peers, with two protected hours a week.
Own team huddles, the prompt library, first-line help and raising friction.
How a new use case gets in
Who decides what
| Add a new use case | Steering group |
|---|---|
| Connect a new system | IT Director and security and privacy lead |
| Change the data rules | Security and privacy lead and legal counsel, reported to steering |
| Change prompts or training | Program owner |
| Pause the tool for a team | Program owner, same day, no meeting needed |
| Pause the tool for everyone | Program owner or security and privacy lead, same day; steering told within 24 hours |
The rails, and when each one has to be in place
Six gates, seventeen checks. Wave 1 does not start until its checks are done; wave 2 until the rest are. The program owner confirms each check; the security and privacy lead and legal counsel sign the Data and Transparency checks; steering sees the signed list before wave 1.
Data
Access
Human review
Transparency
Incidents
Responsible use
A starting checklist for a fictional company, not legal advice. Regulated sectors such as healthcare, financial services and government add their own requirements. Involve counsel.
Fourteen champions and two waves
Who the champions are
About one per 15 agents, chosen by peers and team leads together. They aren't only the most technical people. At least two are skeptics from the pilot, including from the six who stopped, because in my experience a convert persuades more colleagues than an enthusiast does.
Each gets two protected hours a week, credit in their performance review, and first look at new features.
What they do
- Run a 15-minute team huddle each week.
- Collect prompts that work into a shared library, by ticket type.
- Answer first-line questions before anything goes to IT.
- Bring friction to the program owner every week, unfiltered.
Re-onboarded in week 4 with the fixes: a tone guide, an updated knowledge base and an approved ticketing connection.
Two teams with strong leads, plus one skeptical team on purpose. Problems surface while the group is still small.
Wave 1 champions coach wave 2. Access switches on only after training.
Building trust
- Week 3: the sponsor holds an open session at both sites and puts the jobs statement in writing, covering how saved time will be used and what AI won't replace.
- Usage is reported by team, never by name. Nobody's individual usage or prompts go into a performance review, and team leads do not get prompt-level logs.
- Agents help write the tone and usage guide.
- Every two weeks, a short note: what we heard, and what we changed.
- A friction channel where "this isn't working" is a welcome message.
Training
- A 45-minute hands-on session on real tickets, in paid hours, before access is switched on.
- Office hours twice a week for the first month.
- Handle-time targets eased for each team's first two weeks.
- The champions' prompt library, kept current.
What I'd track, and what each number is for
Usage shows whether people use it. Value shows whether it helps. Guardrails make sure speed doesn't cost quality or trust. After-call work is the notes and updates an agent finishes after a call ends.
| Metric | Why it matters | Day-90 target | Source |
|---|---|---|---|
| Usage | |||
| Weekly active agents, as a share of licenses | Shows whether it's part of daily work | 65% or more | Vendor admin analytics |
| Active within 14 days of training | Shows whether training lands | 80% of trained agents | Vendor admin analytics |
| Habit rate: weekly actives using it 3+ days a week | Separates habit from novelty | Half of weekly actives | Vendor admin analytics |
| Value | |||
| After-call work per ticket | The time agents get back | Below pre-rollout baseline | Support platform |
| First reply time, email and chat | What customers notice | Below pre-rollout baseline | Support platform |
| Guardrails | |||
| Customer satisfaction | Speed can't cost quality | Within one point of baseline, or better | Post-contact survey |
| Escalations to team leads | Bad drafts show up here first | No more than baseline plus 5% | Support platform |
| Flags raised per 1,000 uses | Whether agents feel safe flagging | Rising in month one, then steady | In-tool flag |
| Data incidents, open | The Scale call needs this at zero | Zero open at day 90 | Security lead's incident log |
| Trust | |||
| Agent confidence pulse, three questions monthly, anonymous | Whether fear is going down | Higher each month | Pulse survey |
| Responsible use | |||
| Unused licenses and automations | Cost, and nothing left running that nobody uses | First review done by day 90; unused licenses switched off | Admin console |
Targets are invented for this fictional company, not industry benchmarks. Vendor usage data usually runs a day or two behind, so the dashboard gets read weekly. Pair every number with time on the floor talking to agents. See how this looks as a dashboard.
One call, three possible answers
Scale
Weekly actives at 65% or more, after-call work below baseline, satisfaction within one point of baseline or better, zero open data incidents.
Next: dispatch and sales, using this playbook.
Fix
Weekly actives between 40% and 65%, or one guardrail slipped.
Next: fix the cause, re-measure for 30 days, then the same three answers once. A second fix is a stop.
Stop
Weekly actives under 40% at day 90, or a data incident that can't be closed.
Next: switch it off and return the budget. Keep the lessons.
Stop is an acceptable answer. Saying so on day one is part of why agents trust the rollout.
This one is fictional. Yours would start with a conversation. Open a room and I'll build the plan for your team before we talk.