DialCompare
Menu

Buying guide

How to test an AI receptionist before you switch your number

A practical trial plan: caller scenarios to run, the outcomes to expect, the failure and escalation cases to force, calendar and CRM checks, and go or no-go criteria for a pilot.

DialCompare editorial · Published

Every vendor in our shortlist offers some way to try the receptionist before paying: free simulated scenarios before going live (Smith.ai), a two-week trial that accepts real forwarded calls (Upfirst), or shorter trials of one to two weeks (RingCentral, Frontdesk, Dialzara). A trial only tells you something if you run the same scenarios against each candidate and write down what happened. This guide is a proposed test plan. We have not run these tests ourselves, and nothing below is a result; it is the script we would use.

Set the trial up so the results mean something

  • Start in a controlled test: consenting colleagues as the callers, a test phone number for the receptionist, and dummy customer and calendar records rather than real ones. Nothing in this stage touches a real caller.
  • Move to live traffic only after the scripted cases below pass and the business has confirmed the caller notice, call recording and data handling requirements that apply in its locations. Then forward a slice first, such as after-hours calls or overflow when the line is busy, and keep your existing voicemail as a fallback.
  • Give the receptionist the same knowledge you would give a new hire on day one: hours, address, services, prices you are willing to quote, who handles what, and the questions you never want answered on the phone.
  • Brief the team. Someone should be reachable for transfer tests, and everyone should know the trial is running so a confused caller is handled kindly.
  • Decide what “good” means before the first call. Pick the accuracy, transfer and booking standards from the go or no-go list at the end, and note your budget from the pricing guide.
  • Turn on recordings and transcripts for the controlled test only with every participant’s consent, and keep a simple log: date, scenario, what the caller said, what the receptionist did, pass or fail.

Caller scenarios to run

Run each scenario from at least two phones, once calmly and once with background noise or interruptions. In the controlled stage every caller is a consenting colleague using dummy details; have one who does not know the script make a few of the calls. Expected outcomes are ours; adjust them to your business.

  1. A new customer asks about hours and location. Expect a correct answer and an offer to help further, not a transfer.
  2. A new customer asks for a price. Expect either the figure you approved or a clear statement that a person will follow up, never an invented number.
  3. A booking request with a specific day and time. Expect the receptionist to check availability, book the slot, confirm the details back, and send the confirmation you configured.
  4. A reschedule and a cancellation of that booking. Expect the original slot freed and the new one confirmed.
  5. A caller asks for a named person. Expect a transfer during hours, or a message taken outside them with callback wording that matches what your business has actually approved and can keep.
  6. An existing customer with an account question. Expect the receptionist to recognize the limits of what it knows and take a message rather than guess.
  7. A simulated urgent call. Route this scenario to a test destination you control, never to a real emergency service. Expect immediate escalation to that destination or the instruction you set, with no small talk. Passing shows the routing worked in the test; it does not certify emergency reliability.
  8. A frustrated caller who interrupts and repeats. Expect calm handling, no loops, and an offer of a human callback.
  9. A caller who speaks Spanish or another language you serve. Expect either correct handling in that language or a graceful handoff.
  10. A rambling caller who takes eight minutes. Expect the call to reach a useful outcome; on a per-minute plan, note the cost this call would have added.
  11. Out-of-scope questions. Ask something medical, legal or financial that the receptionist should not answer. Expect a refusal and a handoff, not advice.
  12. A robocall and a two-second hang-up. Check whether either shows up as a billable unit; vendors differ on what counts (Goodcall, Upfirst).

Failure and escalation cases to force

A receptionist is judged by what it does when the easy path is closed.

  • Transfer to a line that nobody answers. Expect a message taken and a notification sent, not a dropped call.
  • Transfer during a call while the caller is mid-sentence. Expect a warm handoff, or a clear explanation that a blind transfer is happening.
  • Ask a question that is not in the knowledge base. Expect “I do not have that information” and a next step, not a confident guess.
  • Give conflicting details, such as two different dates for the same appointment. Expect the receptionist to ask which one you mean.
  • Call from a blocked number or with caller ID withheld. Expect the same courtesy and the billing treatment you were told to expect.
  • Call at the exact boundary of your after-hours rules. Expect the right routing on both sides of the boundary.
  • Run the simulated urgent path twice on different days against the test destination. Once is not evidence, and no test here certifies emergency handling.

Calendar and CRM checks

After the booking scenarios, check the systems behind the phone rather than the call itself.

  • The appointment sits in the correct calendar, in your time zone, with the buffer you set and without a double booking.
  • The confirmation text or email went to the caller, and the reminder sequence, if any, was scheduled.
  • The contact record in your CRM has the caller’s name, number, reason for calling and the notes you expected, in the fields you expected.
  • A repeat caller did not create a duplicate contact.
  • The right people were notified, and nobody who should not see caller details received them.
  • Recordings and transcripts are stored where you expect, for as long as you expect, and can be deleted if a caller asks.

Pilot go or no-go criteria

Set these before the trial, then hold to them. Ours are a starting point.

  • Accuracy. Every scenario with a known answer was answered correctly on every attempt, or the receptionist declined and handed off. One invented price or one piece of advice on an out-of-scope topic is a no-go.
  • Escalation. Every forced failure case ended with a person or a message, never a dropped caller.
  • Bookings. Every booking, reschedule and cancellation matched the calendar and the confirmation, with no duplicates.
  • Caller experience. Two people who did not run the trial listened to the recordings and would be comfortable with those calls representing the business.
  • Cost. The busy-month estimate from the pricing guide, including overage, rounding and add-ons, fits the budget you set.
  • Data. Retention, deletion and who can see recordings meet your obligations to callers.

If any criterion fails, extend the trial with the failing scenarios only, or move to the next candidate on the comparison table. If all pass, and the business has confirmed its caller notice, recording and data handling requirements, forward a slice of live traffic for another two weeks and re-run the simulated urgent and booking scenarios before you change your published number.

Keep the record

Keep the log after the pilot. Vendors change models, prompts and pricing, and a receptionist that passed in spring can drift by autumn. Re-run the escalation and booking scenarios after any vendor update, and check the provider profile for the current terms before renewing. Our comparison method explains what we verify and what we do not.

Sources