On this page
Most lead vendor "tests" aren't tests. A contractor buys a small package, works the leads between jobs, closes one or none, and makes a gut call. The vendor blames follow-up; the contractor blames the leads; nobody learns anything. A real test decides in advance what success means, tracks every lead the same way, and runs long enough to see sales.
Step 1: Define success before you start
Write down the numbers that would make you scale, keep, or cut. Typical success metrics:
- Cost per sit at or below a target
- Close rate within a range of your existing sources
- Cost per sale below your break-even point
- Accuracy: share of leads with valid contact info inside your area
Set targets from your current sources so the new vendor competes against something real.
Step 2: Size the test properly
Too small and random luck decides; too large and a bad vendor gets expensive. Aim for enough appointments to see a pattern — enough that one lucky or unlucky sale doesn't swing the result. For in-home projects, plan for the test to run 60–90 days so later sales have time to close.
Step 3: Agree on the rules with the vendor
Before the first lead arrives, get in writing:
- The definition of a valid lead or qualified appointment — see what is a qualified appointment
- The replacement or credit policy and the reporting window
- Delivery method and speed
- Exclusivity — see what to verify with exclusive leads
- How you'll share outcome data
Step 4: Control your side of the test
Most failed tests are lost on the contractor's side. For the test period:
- Respond fast. Commit to calling every lead within minutes. Measure it with a response time audit.
- Follow up consistently using the same cadence for every lead. See the follow-up cadence guide.
- Assign appointments fairly. Don't give test appointments only to your weakest rep — or your strongest.
- Log outcomes daily in your CRM with a reason for every non-sale.
Step 5: Track every lead through the funnel
| Stage | Count | Rate |
|---|---|---|
| Leads or appointments received | ||
| Valid (accurate, in area) | ||
| Contacted | ||
| Booked | ||
| Sat | ||
| Sold | ||
| Revenue |
Score each lead with the lead quality scorecard so you can explain the numbers, not just report them.
Step 6: Review at checkpoints
- Week 2: accuracy and contact rates. Raise data problems early with specific examples.
- Week 4–6: booking and show rates; early sales.
- Day 60–90: cost per sale and revenue versus your targets.
Step 7: Decide with rules, not feelings
| Result | Decision |
|---|---|
| Beats targets on cost per sale | Scale gradually and keep tracking |
| Close to targets with a clear fixable issue | Fix the issue, extend the test |
| Accuracy problems the vendor won't address | Cut |
| Good accuracy, poor outcomes, and your process held up | Cut or renegotiate |
| Poor outcomes and your follow-up slipped | Re-run the test properly |
Common testing mistakes
- Judging after two weeks, before in-home sales cycles finish
- Running two new vendors at once in the same territory, so results blur
- Comparing a new vendor's cost per lead with an old vendor's cost per sale
- Not telling the vendor about problems until the test is over
Commission-only arrangements change the math of a test: if you pay only when jobs close, the vendor carries most of the risk. See commission-only lead generation. Before signing anything, run through our questions to ask an appointment setting company.
Frequently asked questions
How long should I test a new lead vendor?
Long enough for leads to move through your full sales cycle — usually 60 to 90 days for in-home projects — with enough volume that a single sale doesn't decide the result.
What metrics should I use to evaluate a lead vendor?
Accuracy, contact rate, booking rate, show rate, close rate, cost per sit, cost per sale, and revenue. Compare them with your existing sources over the same period.
How many leads do I need for a fair test?
Enough to see patterns rather than luck. If one sale changes your conclusion, the test was too small or too short.
Should I tell the vendor what's wrong during the test?
Yes. Share specific examples early — wrong numbers, out-of-area addresses, missing decision-makers. Good vendors adjust; how a vendor responds is part of the test.
RunsForYou Team
Written by the team that runs outbound calling, homeowner qualification, and appointment booking for home service contractors at RunsForYou (Runs For You LLC).
Related services