A/B Testing in UX Design: When and How to Run a Valid Experiment
Learn when A/B testing fits UX, how to define hypotheses and metrics, run valid experiments, interpret results and avoid false product conclusions.
Usability Testing Services
Watch people try the tasks that matter in your product or prototype, and see exactly where they stop, guess or give up — before you decide what to change.
- For live products and prototypes
- The tasks people must get right
- Custom scope
- Fixed or phased, set in the proposal
- Moderated or unmoderated
- Chosen for the question
- Prioritized findings
- Tied to what participants did
In short
See where people get stuck, and what to fix first.
What's included
- Study plan
- Participant screening
- Task sessions
- Prioritized findings
Answers you'll have after the sessions
- Can people finish the tasks that matter?
- Where exactly do they get stuck, and why?
- Which problems should the team fix first?
Where it stops
Experts review the product against known patterns.
Usability Testing
Real users try real tasks while we watch.
Not included
- How common a problem is across all your users
- Redesign or development — scoped separately
From the tasks you choose to the fixes you make
What your team holds after the study, step by step.
-
Study plan
The tasks, who should take part and the method, agreed before any session.
-
Task sessions
Moderated or unmoderated, on your product or prototype, with your team invited to watch.
-
Observation record
What each participant did, where they hesitated and what they said.
-
Prioritized findings
Problems ranked by their effect on the task, each with a recommendation.
You receive
- A findings report
- Session notes and consented clips
- A review with your designers
Watch the task, not the opinion.
Usability testing services answer one question: can the right people do what the product is for? Each method is chosen for the part of that question it answers best.
-
Which tasks, on which build, answer your question?
Method: A study plan and a pilot session Your product team, with the live product or a prototype
Why this method: One pilot catches unclear tasks and broken builds before real participants arrive.
-
Who should take part?
Method: Participant screening Your customers where reachable, or recruited participants screened for fit
Why this method: Findings from the wrong people send the fixes in the wrong direction.
-
What goes wrong at each step, and why?
Method: Moderated task sessions A small group per user type, thinking aloud with a facilitator
Why this method: Follow-up questions show why a step fails, not only that it does.
-
Does the same problem repeat across more attempts?
Method: Unmoderated remote tests A wider group, recorded on their own devices
Why this method: More attempts at a lower cost per session, for patterns that do not need follow-up.
-
Which problems block the task, and which only slow it down?
Method: Synthesis and a prioritization review Every session, reviewed with your product and design leads
Why this method: Each finding is tied to the task, what was observed and how many participants hit it.
Limits and confidence. Usability testing shows where and why people struggle, not how many of all your users will. Findings are reported as counts of participants, such as “4 of 6”, never as percentages of your market, and the report says what the sessions could not cover.
When usability testing is the right step
It fits when there is something to try and a decision it should inform. Four signs it is the right step now.
-
You can name the tasks that matter
Signing up, paying, finding a record, finishing a setup — the few jobs the product must not fail at, and the people who do them.
-
There is something to try
A live product, a beta or a clickable prototype that participants can actually use.
-
A design decision is waiting
A redesign, a new flow or a release that the team wants to check before committing to it.
-
The team will watch and act
Designers and product owners can join the sessions and turn the findings into changes.
Let's watch your users try it
Tell us which tasks matter and what you are about to decide. We will come back with a study plan and a realistic start.
Scope, access and who does what
Tasks, participants, method and the review rhythm are agreed before the first session.
- Scope
- Custom fixed or phased, set in the proposal
- Timeline
- Agreed after scope, access and recruitment
-
You provide
- The product
- Access to the live product, a staging build or a clickable prototype.
- The tasks
- The jobs people must be able to do, and the decision the study should inform.
- Participants
- Customers we can invite, or a profile we can recruit and screen against.
- Your team
- A product owner who agrees the plan, and designers who join the sessions.
-
Who does what
- ANODA
- Plans the study, screens participants, runs the sessions, analyses them and writes the prioritized findings.
- Your team
- Provides access and context, helps reach users, watches the sessions and decides what to change.
-
Boundaries
- Outside usability testing
- Redesign, development, analytics setup and statistical surveys of your whole user base.
- After the sessions
- Fixes can be designed with us or your own team, and retested, scoped separately.
Which task must people get right?
Tell us about the product, who uses it and the decision the sessions should inform.
Message received.
Within 15 minutes, we’ll email you initial feedback and follow-up questions.
Usability Testing Questions: What to Ask Before, During and After a Test
Build a better usability test with questions for screening, tasks, follow-ups and debriefs, plus guidance on what not to ask and how to use the answers.
Usability Testing: common questions
Who can test whether people can complete the main tasks in our product?
A usability testing agency with researchers who plan studies, screen participants and facilitate sessions without leading people to the answer. Ask how they choose tasks and participants, how they report findings from a small group, and who on their team will run the sessions. A good agency will also tell you when an expert review or a smaller test would answer your question for less.
How are findings prioritized?
Each finding is tied to the task it affects, what participants did and how many of them hit it. Problems that stop people from finishing a key task come first, then those that slow them down or cause errors. The team weighs that order against effort in a review.
What do usability testing services at ANODA include?
A study plan with tasks, participant criteria and method, screening and recruitment when scoped, moderated or unmoderated sessions on your product or prototype, an observation record, and prioritized findings with recommendations, reviewed with your team. Your designers and product owner are invited to watch the sessions live, so the findings are not news to them when the report arrives.
Should we run usability testing, a UX audit or UX research?
Run usability testing when you need to see real people try specific tasks. A UX Audit is faster when experts can find the problems without recruiting users. UX Research fits when the question is what people need rather than how the interface works, and Product Discovery when the product itself is not defined yet.
What do you need from us to start?
Access to the live product, a staging build or a prototype, the tasks that matter and the decision the study should inform, customers we can invite or a profile to recruit against, and a product owner who agrees the plan.
What do we get at the end?
A findings report with problems ranked by their effect on the task, session notes with clips where participants agreed to recording, and a review with your designers and product owner. The findings are yours to act on with any team.
How are scope, timing and price set?
By the number of tasks and user groups, whether sessions are moderated or unmoderated, how participants are recruited, and whether a retest is needed. We set a custom scope and estimate in the proposal, fixed or phased, and agree the timeline once access and recruitment are clear.
What is not included?
Redesign, development, analytics setup and statistical surveys of your whole user base. User testing services show why and where people struggle; they cannot prove how many of all your users will. Fixes and retests can follow with us or your own team, scoped separately.