Use this framework to choose the right UX evaluation method for your stage and budget:
| Heuristic Evaluation |
Early design, low budget, internal validation |
Fast (1-3 days), cheap ($2-5K), finds 75% of issues |
Misses novel problems, no real user feedback |
$ |
3-5 days |
| Usability Testing |
Validate prototype, test with real users |
Finds real-world issues (100%), user quotes, task metrics |
Slower (2-3 weeks), more expensive ($5-15K) |
\[ | 2-4 weeks |
| **A/B Testing** | Optimize live product, high traffic | Quantitative data, large sample, real behavior | Requires traffic, can't explain WHY, only WHAT | \]$ |
2-6 weeks |
| Analytics Review |
Understand current behavior |
Based on actual usage, free, large sample |
No “why,” only “what,” miss non-user perspectives |
Free |
1-2 days |
Decision Tree:
Step 1: What stage are you at?
→ Early concept/wireframe → Use Heuristic Evaluation - Rationale: Too early for user testing (no working prototype), heuristic eval catches obvious violations
→ Functional prototype → Use Usability Testing - Rationale: Now you can watch real users interact, catch issues heuristic eval misses
→ Live product → Use Analytics + A/B Testing - Rationale: Real usage data reveals what users actually do (not what they say they’ll do)
Step 2: What’s your budget?
→ <$5,000 → Use Heuristic Evaluation (3 evaluators, $2-4K) or Guerrilla Usability Testing (5 users recruited at coffee shop, $500)
→ $5K-$20K → Use Formal Usability Testing (8 users × $75/user + facilitator = $5-10K) + Heuristic Evaluation ($2-4K) = comprehensive feedback
→ $20K+ → Use Agency-Led Research (full discovery, usability testing, A/B testing, reporting)
Step 3: What questions do you need answered?
| “Does this violate usability principles?” |
Heuristic Evaluation |
Experts compare to established heuristics |
| “Can users complete tasks?” |
Usability Testing |
Watch real users struggle or succeed |
| “Which version converts better?” |
A/B Testing |
Quantitative comparison of alternatives |
| “Why are users dropping off?” |
Usability Testing + Analytics |
Combine what (analytics) with why (user observation) |
| “What are common failure modes?” |
Heuristic Evaluation |
Systematic inspection finds edge cases |
Combining Methods for Maximum Coverage:
Phase 1: Early Design (Week 1-2)
- Heuristic Evaluation (3 evaluators, 8 hours each)
- Output: List of 20-30 violations, prioritized by severity
- Cost: $2,400
Phase 2: Validate Fixes (Week 3-4)
- Usability Testing (5 users, 60 min each, think-aloud protocol)
- Output: Task success rates, time on task, user quotes
- Cost: $2,500 ($375 user compensation + $2,125 facilitator)
Phase 3: Launch (Week 5-12)
- Analytics Monitoring (track key metrics: task completion, error rates, dropout points)
- Output: Quantitative validation, identify new problem areas
- Cost: Free (built into product)
Phase 4: Optimize (Month 3-6)
- A/B Testing (test 2-3 variations of problem areas)
- Output: Data-driven optimization
- Cost: $5-10K (engineering time to implement variants)
Heuristic Evaluation Efficiency:
Coverage vs. Cost Comparison:
| 1 evaluator |
35% of issues |
$800 |
$23/issue |
1 day |
| 3 evaluators |
75% of issues |
$2,400 |
$32/issue |
1 day (parallel) |
| 5 evaluators |
85% of issues |
$4,000 |
$47/issue |
1 day (parallel) |
| Usability testing (5 users) |
85% of issues |
$2,500 |
$29/issue |
2 weeks |
Key Insight: 3 evaluators find 75% of issues at $32/issue in 1 day. Diminishing returns after 3 evaluators. Usability testing finds remaining 10-25% (novel issues heuristic eval misses) at similar cost/issue but slower timeline.
When to Skip Evaluation (High Risk):
Low Risk (Evaluation Optional): - Minor feature addition to proven product - Internal tool with tech-savvy users - Cosmetic changes (color, font) with no functional impact
High Risk (Evaluation Critical): - New product category (no existing mental models) - Safety-critical (medical, security, automotive) - Physical hardware (can’t patch post-launch) - Vulnerable users (elderly, disabled, children)
Red Flags You Need Evaluation:
Key Insight: Heuristic evaluation is fast and cheap ($2-4K, 1-3 days) but finds only 75% of issues. Usability testing is slower (2-4 weeks) but finds novel issues heuristic eval misses. Best practice: Do both—heuristic eval early to catch obvious violations, usability testing later to validate fixes work for real users. Combined cost $5-7K prevents $100K+ in returns and support costs.