How to Run a Reply Speed Test on Your Own Business (and Your Competitors)
A one-week mystery-shopper test for your own inbox: ten messages, three time slots, four channels and a simple scoring sheet out of six.
In this article
A reply speed test is a simple mystery-shopper exercise: you message a business as an ordinary customer would, at set times and on set channels, and record how long the first useful reply takes and whether it answered the question. For a small online shop, 10 test messages spread over one week, three time slots (morning, evening and after midnight) and four channels (WhatsApp, Instagram, Messenger and website chat) is enough to show where buyers are waiting. Run it on your own business first, then on two or three competitors, and keep the competitor results private.
Key takeaways
- Plan 10 messages across 3 time slots and 4 channels, sent from a phone and accounts not linked to your business.
- Score each message on speed (0 to 3), whether it answered (0 to 2) and whether it offered a next step (0 to 1).
- Most shops find the after-midnight slot is the weak one. That's a coverage gap, not a staff problem.
- Testing competitors is fine if you behave like a normal customer: short questions, no fake orders, no named results in public.
- Re-run the same test a month after any change so you compare like with like.
Why run a reply speed test at all?
Because your own reports only see the conversations that got a reply, and you rarely experience your shop as a buyer does. A test shows the real wait from the customer's side, including at 1 am, on the channel nobody checks. It takes about an hour of effort spread over a week and costs nothing.
Owners usually think they reply faster than they do. You remember the busy afternoon when you answered everyone in two minutes. You don't remember the Instagram DM that sat until morning because the notification was buried under twenty others.
This kind of test isn't new. The 2011 Harvard Business Review study "The Short Life of Online Sales Leads" was essentially a large mystery shop: researchers sent web enquiries to 2,241 US companies and timed the responses. Only 37% replied within an hour. You're doing the same thing on a smaller scale, for the channels your buyers actually use.
What you need before you start
- A phone number and accounts that aren't linked to your business. A friend's phone, a second SIM, or a personal Instagram account that doesn't follow your page. If your team recognises the sender, the test is worthless.
- A private browser window on mobile data for website chat, so the widget doesn't remember you.
- Ten questions written in advance. Real ones your buyers ask: price, size, delivery charge, delivery time to a specific area, stock of a specific colour, return policy, payment options.
- A scoring sheet (below), in a notebook or a spreadsheet.
- Silence. Don't tell your team the test is running. If you have staff, tell them afterwards that it happened and that it was about the system, not about catching anyone out.
How do you run a reply speed test step by step?
Write ten realistic questions, assign each to a channel and a time slot, then send them across one week from accounts your team won't recognise. Note the exact time sent and the time of the first reply that actually answers. Score each one the same day, then total the scores by time slot and by channel.
- Write the questions. Mix easy ones ("Black one in L size available?") with ones that need a person ("Can you deliver to Gazipur by Thursday?").
- Build the schedule. Three slots: morning (9 to 10 am), evening (8 to 10 pm) and after midnight (12 to 2 am). Spread the ten messages so every channel is tested in at least two slots.
- Send one message per slot. Don't send three at once; you'll only measure one moment.
- Record the times. Screenshot the chat so you have the timestamps. On WhatsApp, the message info screen shows when it was delivered and read.
- Stop the clock at the first real answer. An auto-greeting or "we'll get back to you" doesn't count.
- Reply politely and close the conversation. "Thanks, I'll decide and come back" is fine. Don't leave someone chasing you.
- Score and total. Same day, while you remember the details.
The scoring sheet
Keep the scoring simple enough that you'll actually fill it in. Speed earns up to 3 points, a real answer up to 2 and a clear next step 1, so each message scores out of 6. Here is a filled-in example for one shop's own test (illustrative numbers).
| # | Channel | Slot | Question | Wait for first real reply | Speed (0-3) | Answered? (0-2) | Next step? (0-1) | Total (of 6) |
|---|---|---|---|---|---|---|---|---|
| 1 | Mon morning | Price of the blue panjabi? | 4 min | 2 | 2 | 1 | 5 | |
| 2 | Mon evening | Size L in stock? | 38 min | 1 | 2 | 0 | 3 | |
| 3 | Messenger | Tue after midnight | Delivery charge to Sylhet? | 9 h 10 min | 0 | 2 | 1 | 3 |
| 4 | Website chat | Wed morning | Do you take card payment? | No reply | 0 | 0 | 0 | 0 |
| 5 | Wed evening | Can I exchange if it doesn't fit? | 12 min | 1 | 1 | 0 | 2 | |
| 6 | Thu after midnight | Is this cotton? | 10 h 25 min | 0 | 2 | 0 | 2 | |
| 7 | Messenger | Fri evening | Delivery to Gazipur by Sunday? | 6 min | 1 | 2 | 1 | 4 |
| 8 | Fri after midnight | Price for 3 pieces? | 8 h 40 min | 0 | 1 | 0 | 1 | |
| 9 | Sat morning | Any discount this week? | 2 min | 2 | 2 | 1 | 5 | |
| 10 | Website chat | Sun evening | How long does delivery take? | 50 sec | 3 | 2 | 1 | 6 |
Speed points: under 1 minute = 3, 1 to 5 minutes = 2, 5 to 60 minutes = 1, over an hour or no reply = 0. Answered: 2 if the question was fully answered, 1 if partly ("price is in the post"), 0 if not. Next step: 1 if the reply moved towards an order, such as asking for size or address.
How do you read the results?
Total the scores by time slot first, then by channel. A low after-midnight total means nobody covers nights. One weak channel usually means nobody is watching that app. Fast replies that score low on "answered" mean the replies are quick but unhelpful, which is the hardest problem to fix and the most important.
In the example above, the slot totals are morning 10 out of 18, evening 15 out of 24, after midnight 6 out of 18. Website chat scored 6 on Sunday and 0 on Wednesday, which suggests the widget is only watched when someone happens to be at the laptop. Every night-time message did get an answer, but between eight and a half and ten and a half hours later.
That shop doesn't need to hire anyone for daytime. It needs nights covered and someone responsible for website chat. If you want to turn the results into ongoing numbers, our first response time guide shows how to measure the same thing from real conversations every week.
Compare against the targets in how fast you should reply to customer messages: a first useful reply in under a minute, and a person within five minutes during opening hours.
Is it ethical to mystery-shop your competitors?
Yes, if you behave like a genuine customer. Ask a normal question, keep the exchange short, don't place fake orders and don't name or shame anyone with the results. The UK Market Research Society's mystery shopping guidelines say competitor visits shouldn't use more of a business's time than a normal enquiry.
The MRS guidelines (2013) are written for professional researchers, but the principles carry over well to a shop owner checking a few rival pages:
- Keep it short. One question, a polite close. Don't string someone along for twenty messages.
- Never place a fake order. In cash-on-delivery markets a fake order costs the seller real courier fees and a returned parcel. Don't do to others what hurts you.
- Don't single out a person. You're testing the business's system, not an individual admin.
- Report in aggregate. MRS says data "must be reported at an aggregated level". Write "the three competitor pages averaged 3.1 out of 6", not "Page X ignored me".
- Don't publish it. Screenshots of a rival's slow reply on your own page look petty to buyers and can be unfair. They might have had a family emergency that night.
What do you do after the test?
Fix the weakest slot first, not the average. If nights scored lowest, give night-time messages an automatic first answer from your price list and policies, and let a person follow up in the morning. If one channel scored lowest, bring all channels into one inbox so nothing sits unseen. Then re-test after a month.
When you re-test, use the same ten questions, the same slots and the same scoring. Change one thing at a time between tests so you know what made the difference. If you turned on night-time answers and the after-midnight total went from 6 to 15, you have your answer. Our guide to setting up after-hours AI replies goes through the night shift step by step.
Where WisperBot fits
WisperBot, a reply-first inbox for WhatsApp, Instagram, Messenger and website chat, is one way to close the gaps a test usually finds. All four channels land in one inbox, and a Smart Bot can answer common questions in seconds from your own information, during the hours you set for each day, such as 10 pm to 9 am, in your timezone. It hands over to a person when a question needs judgement. Run your test first, though. If your scores are already good across all slots, you may not need anything new.
Frequently asked questions
How many test messages do I need for a reply speed test?
Ten is enough for a small shop to spot patterns, as long as they cover every channel you use and at least three time slots, including after midnight. Fewer than six and you'll mostly be measuring luck. If you get hundreds of messages a day, measure from your real conversation data as well.
Is it legal to mystery-shop a competitor's Facebook page?
Sending a genuine-style product question to a public business page is ordinary customer behaviour. Problems start if you place fake orders, harass staff, collect personal information or publish results that damage a named business. If you're unsure about the rules where you are, keep it to short, genuine questions and private, aggregated notes.
Should I tell my team about the test?
Not before it runs, or they'll watch the inbox more closely that week and the results won't be real. Tell them afterwards, share the scores by time slot and channel, and make clear the point is to fix gaps in coverage, not to catch anyone out.
What is a good score on a reply speed test?
Using the scoring sheet in this guide, an average of 5 out of 6 or more across all slots is strong. Many small shops score well in the morning and evening and close to zero after midnight. Focus on the slot with the lowest total rather than the overall average.
How often should I run a reply speed test?
Once before you change anything, then a month after each change, using the same questions and schedule. After that, every quarter is enough, plus once before a big sale or festival when message volume will jump. Rotate the sending phone and accounts so your team doesn't start to recognise them.
Sources
- Market Research Society: Mystery Shopping Research Guidelines (January 2013)
- Harvard Business Review: The Short Life of Online Sales Leads (March 2011)
Last checked: 8 October 2026.
