Skip to content

Comparing your options

An embedded testing team, a CRO agency, or your first in-house hire.

All 3 can work. Which one fits depends on how many tests you need, how fast you need them, and whether the learning stays in your company or leaves with a vendor.

How to decide

Price is the last thing to compare, not the first.

The 3 options buy different things, so the monthly number on its own tells you very little. What decides it is how many tests you actually need each month, how soon you need the first one to run, and who is doing the work once the pitch is over.

The question most teams skip is what happens to the learning. A year of tests is worth more than any single win, and where that record lives when the engagement ends is the difference between a program and a series of projects.

I have worked all 3 ways. I built and ran the in-house testing team at DMV.org, and we brought in Widerfunnel to run tests on the insurance channel. Both did things the other could not. This page is written by us, so read it with that in mind.

Best fit repeatGROWTH A CRO agency In-house team
Who does the work repeatGROWTH Me, on every test A CRO agency An account lead, then whoever is free In-house team 1 person, plus whoever you can borrow
Roles you get repeatGROWTH Research, design, build, QA and analysis from the same team A CRO agency Account manager, analyst and developer, split across your account In-house team Whatever you hire, plus other departments as support
Typical monthly cost repeatGROWTH From $6,495 A CRO agency $5,000 to $25,000 In-house team $35,000 or more for a 3-person team, fully loaded
Time to first test repeatGROWTH Under 2 weeks A CRO agency 4 to 8 weeks In-house team 3 to 6 months including hiring
Test volume repeatGROWTH 2 to 12+ a month A CRO agency 2 to 8 a month In-house team 15+ a month once the team owns the roadmap
Breadth at once repeatGROWTH 1 or 2 funnels at a time A CRO agency Several channels in parallel with a bigger bench In-house team Limited by your own headcount
Flexibility repeatGROWTH Conversion is the focus, and what we learn gets shared with the channels it affects A CRO agency Tends to stay inside the contracted lane In-house team Often pulled into other departments as support
Day to day access repeatGROWTH In your Slack, same day A CRO agency Scheduled calls and a monthly report In-house team In the building, in every standup
Documentation repeatGROWTH Every test documented and handed to you, wins and losses A CRO agency Selected case studies In-house team Internal, if someone keeps it up

Agency and in-house figures are ranges we see in the market, not quotes. Ask any agency for their number and what it includes, since tools and developer time are often billed separately.

Where the money goes

A retainer pays for a team, not for tests.

An agency retainer covers an account manager, an analyst and a developer, whether or not all 3 are busy on your account that month. An in-house team costs more again, and earns it back in volume once the team owns the roadmap.

We sit in between. Fewer people, no account layer, and the person you talk to is the person building the test. That is the whole reason the number is what it is.

Plenty of teams split the work. Some clients design and let us build and measure. Others hand us the whole thing. Both work, and the scope changes what it costs.

Where we are not the right call

3 situations where something else fits better.

You need 10 or more tests a month

That is an agency with a bigger bench. We would be the bottleneck, and you would feel it by month 2.

Pick an agency

Testing is a culture change, not a project

An in-house lead who sits in your standups will change how the company makes decisions. We work alongside your team, but we do not change how it runs.

Pick an in-house hire

Your traffic cannot reach a clear result

Below roughly 30,000 sessions a month, most tests take months to read. Spending on traffic first usually returns more.

Spend on traffic first

Not sure which one you need?

The evaluation answers it with your own numbers. We review your analytics, test the experience with real users, and tell you what we would do first.

If the answer is an agency or an in-house hire, the evaluation will say so.

Request an evaluation