A little while ago, I was talking with a UX researcher from another company who had been working in the field for about 8 or 9 years. He had a ton of experience with different research methods and described himself as knowledgeable about quant research. He even used the mixed-methods moniker to describe his work a few times during our conversation.

Specifically, we were discussing a quant study he had recently completed and whether the sample was large enough to support the findings. (Sound familiar? Hahahaha) Early in his career, he had been taught that an entire study was either statistically significant or it was not. That idea shaped the question he asked me during our conversation. Was the study statistically significant? His confusion was not caused by a lack of research experience or a broad unfamiliarity with quant methods. It came from treating statistical significance as a label for the entire study rather than something applied to individual findings within it.

I then explained that statistical significance usually applies to a specific result within a study. One comparison may meet the statistical threshold while another comparison from the same study may not. Both results can come from the same participants, the same survey, and the same overall sample.

The researcher then asked the question that revealed the real source of the confusion. If the sample size stayed the same, how could one result be statistically significant while another was not? To answer that, we needed to look beyond the number of participants. We needed to discuss the size of each result and the amount of variation in the data. That’s the catalyst for this week’s post.

Statistical Significance

The value of statistical significance is that it helps us judge whether an observed result is unlikely to be explained by normal sampling variation alone and may reflect a real pattern in the broader population. It usually applies to one specific comparison, difference, or relationship in the data, not to the entire study.

Suppose we compare 2 versions of an experience and observe a difference in task-completion rates. A statistical test helps us evaluate how unusual a difference that large would be if there were no real difference between the designs in the broader population. When the evidence meets the threshold selected for the research, we call that particular result statistically significant. That conclusion is narrower than many people realize.

This distinction matters because statistical significance does not tell us how large or useful a finding is. In fact, the American Statistical Association has warned researchers against treating statistical significance as a measure of the size or importance of an effect. The observed difference, uncertainty, research design, and practical consequences all need to be considered together.

In my last 5 orgs, I advocated for using a 90% confidence level for our quant research. Compared with a 95% confidence level, this accepts slightly more uncertainty and can reduce the sample size required to achieve the same level of precision, making it a practical tradeoff among confidence, recruiting effort, time, and cost. A 95% confidence level is the most commonly used convention across many research fields, including academia. Higher-stakes research may require different or stricter standards, but the appropriate threshold should reflect the decision being made and the relative consequences of false positives and false negatives. But for my orgs, at those points in time, 90% was a better, more practical default. I made sure everyone understood the tradeoffs and socialized the idea before any studies were run.

Every org is different, and you’ll need to determine the appropriate threshold based on your specific situation. And keep in mind, using a 90% confidence standard does not make every finding in a study statistically significant by default. It simply establishes the threshold your org will use when evaluating each result.

An org-wide standard also makes research decisions more transparent and collaborative. It prevents researchers from keeping the reasoning behind sample-size decisions to themselves. If a stakeholder questions a study’s validity based on a common sample-size misconception, you can refer back to the standard everyone agreed to before the research began. As a best practice, the threshold should also be selected before analyzing the data because changing it after seeing the results would allow the desired outcome to influence the analytical standard.

Effect Size

Once we cleared up the idea that significance applies to individual findings, the rest of the conversation became wayyyyyyy easier. The concept that resolved the confusion was effect size.

Effect size describes how large an observed difference is. For example, a 2-percentage-point change in task completion is a much smaller effect than a 20-percentage-point change. Larger effects are generally easier to distinguish from normal variation, while smaller effects usually require more participants to evaluate confidently.

For example, consider a study comparing 2 versions of the same GUI. One result shows that average satisfaction increased from 5.1 to 5.2 on a 7-point scale. Another result shows that task completion increased from 55% to 85%.

Both findings came from the same study and the same group of participants. The difference in satisfaction is very small, while the difference in task completion is much larger. The completion result may meet the statistical threshold even when the satisfaction result does not. Is this all making more sense now?

The exact answer would also depend on how consistent the responses were and which statistical test was appropriate.

4 factors work together when we evaluate a quant result:

• Sample size: How much information the study contains

• Effect size: How large the observed difference is

• Variability: How consistent or spread out the results are

• Confidence level: How much uncertainty the study is willing to accept

Sample size matters, but it cannot answer the question by itself. Two findings from the same sample can produce different statistical conclusions because the effects and the amount of variation may be different.

This is where the common recommendation of 30 participants often causes confusion. MeasuringU addressed this issue masterfully in its article, “Do Statistics Really Require 30 Participants?” That article explains that 30 is not a universal minimum for conducting statistical analysis. Statistical methods can be used with smaller samples, although small samples usually provide less precision and make smaller effects harder to detect.

And remember, a result that does not meet the statistical threshold can still provide useful information. We all know this to be true, but the term statistical significance often confuses people and muddies the waters, making the problem even worse. It may suggest a possible direction, support evidence from another research method, or show that a larger sample would be needed before making a confident claim.

That was my answer to my researcher friend’s original question. The sample size did not make the entire study statistically significant or insignificant. It gave us a certain amount of information, and each result had to be evaluated based on the size of the effect, the variation in the data, and the confidence standard we had selected.

Different Methods, Different Samples

The conversation also reminded me why sample-size guidance becomes confusing so quickly in UX research. We use the term UX research to describe methods that answer very different questions, so the same sample-size logic cannot be applied to all of them.

Christian Rohrer’s framework organizes UX research methods in a 2 × 2 matrix, which I absolutely love. One dimension separates what people say from what people do. The other separates qual understanding from quant measurement.

I’m sure you’ve seen this before in Christian’s Nielsen Norman Group article, ”When to Use Which User-Experience Research Methods”:

Qual Attitudinal

For qual attitudinal research, I usually think about data saturation rather than statistical significance. Data saturation occurs when new interviews mostly repeat themes we have already heard and stop changing our understanding in a meaningful way.

The results from this type of research are often directional. They can give us enough evidence to make a reasonable decision or identify what should be investigated next, but they do not tell us exactly what percentage of the full population holds each opinion.

Qual Behavioral

Qual behavioral research follows different logic. For example, in a moderated usability test, the purpose is often to find interface problems that the team can fix. A large sample is usually unnecessary when that is the research objective.

That means it doesn’t matter if only 1 of 8 participants cannot find the button needed to complete an important task. The session does not tell us how frequently the full user population will encounter a usability problem, but that’s because frequency and severity are separate considerations. When you find a usability problem in a qual behavioral research study, you fix it no matter how many participants encountered that problem.

Quant Attitudinal and Behavioral

The 2 quant quadrants are where statistical significance becomes more relevant. Even within these quadrants, the type of question affects what we can measure, compare, and conclude.

Remember the earlier example comparing two versions of the same GUI. Satisfaction increased from 5.1 to 5.2, while task completion increased from 55% to 85%. The same participants produced both results, but the much larger difference in task completion was more likely to meet the statistical threshold.

This shows why different types of quant data need to be interpreted differently:

• Rating scales let us compare how people evaluated an experience. We need to consider both the difference between the ratings and how much individual responses varied.

• Rankings show the order in which people preferred several options. They do not tell us how much more the first-ranked option was preferred over the second.

• Behavioral measures let us compare results such as task completion, errors, time on task, and conversion. Each measure must be analyzed separately because the size and consistency of the differences may vary.

• Open-ended questions still produce qual data, even when they appear in a large survey. We can categorize and count the responses, but statistical significance does not apply directly to the original written comments.

This brings us back to the main point from earlier.

How I Report Mixed Results

Once we accept that significance applies to individual results, reporting a quant study becomes more specific. Instead of announcing that the study was significant, I describe what happened for each important measure.

For every major result, I try to report four things:

1. What we observed: The scores, percentages, or differences found in the sample.

2. How large the effect was: The actual size of the difference between the results.

3. How confident we are: Whether the result met the statistical threshold and how much uncertainty remains.

4. Why the result matters: Whether the difference is large enough to affect a user or business decision.

A report might explain that participants completed a task more often with Design B and that the difference met our 90% confidence threshold. The same study might show a small improvement in satisfaction that did not meet the threshold.

Those findings should be reported separately. The task-completion result may provide strong enough evidence to influence the decision, while the satisfaction result may remain directional or require a larger sample. For example, a result that does not reach statistical significance still contains valuable information. The observed difference may suggest a direction, support findings from interviews or usability testing, or help the team estimate the sample needed for another study.

Statistical thresholds should support the interpretation rather than replace it. Again, the American Statistical Association advises against making scientific, business, or policy decisions based only on whether a result crosses a statistical threshold. The study design, effect size, assumptions, uncertainty, and consequences of the decision all contribute to the conclusion.

Conclusion

By the end of our conversation, the researcher I was talking to no longer needed one answer about whether the entire study was statistically significant or not. We could now look at each important finding and discuss the effect size of the result, the amount of uncertainty, and whether the evidence was strong enough to support a decision.

The bottom line is that a study with only 32 respondents may produce statistically significant findings when the effects are large, while smaller effects within the same study may remain statistically unresolved and be treated as directional.

If you take away only one thing from this article, it should be this:

The most important analytical skill in this situation was knowing that statistical significance describes evidence for a particular result. Effect size helps us understand how large that result is, and our research context helps us decide whether it matters.

I hope this help clarify things. As always, thanks for reading.

Summary: After MS Forms produced a questionable winner from a 32-person force-ranking exercise, I exported the raw data and compared the default output with Borda scoring, pairwise wins, and the Schulze method. The analysis showed that different methods can produce different winners, which is why UX researchers need enough statistical knowledge to recognize when a tool’s default calculation does not match the question they are trying to answer.

A couple of weeks ago, I used MS Forms to run a stack-ranking exercise at work. 32 participants ranked 8 items from most important to least important. When I opened the results, Forms had already calculated the rankings, as it should, and identified a winner.

The result did not look wrong to the untrained eye, but within minutes of looking at the raw data, I knew something was off. For example, the item Forms ranked 1st appeared consistently near the top of everyone’s lists, but another item received far more first-place rankings and performed better when compared directly with most of the other options. So I knew what I needed to do. I needed to determine the most appropriate way to reanalyze the data because Forms’ default analysis sucked for this scenario. Thanks Microsoft.

Study Design Constraints

Let’s back up a little. Stack ranking was the best approach here because the stakeholders needed to understand relative preference across a short list of comparable items. Think of a study asking users to prioritize 8 common tasks they might complete through a self-service portal:

1. Check the status of a request

2. Update account information

3. Download a document

4. Make a payment

5. Contact support

6. Review recent activity

7. Manage notification settings

8. Add or remove an authorized user

These are concise, distinct tasks that participants can reasonably compare. Ranking them forces people to make tradeoffs that a standard rating scale might not reveal. Someone could reasonably rate all 8 tasks as important, but a forced ranking shows which ones they would prioritize when everything cannot be treated equally.

Another factor was the deadline was super tight, aren’t they all. Hahahaha. Based on our previous recruiting and survey conversion rates, I expected between 30 and 50 completed responses within the timeline we had to work in. I would have preferred a larger sample because it would have provided greater precision and made smaller differences between items easier to interpret. A larger sample would also have reduced the likelihood that a few individual responses could change the final order.

The 32 responses were still very helpful, relevant and statistically valid within the guardrails established for the study. I presented the findings as directional evidence, avoided claims about small differences and did not use the results for subgroup comparisons. And here’s the important takeaway, I explained those limits before launching the survey. The stakeholders agreed upfront that the study would still provide enough evidence to guide the immediate decision.

This upfront conversation is a very mature research step that many UX Researchers flat out skip. Getting agreement on the strength of the evidence before collecting it reduces arguments about the method during the final readout. It also gives the research team credibility because the standard for interpreting the results is established before anyone knows which item will win.

The Real World Requires Judgment

MS Forms produced an aggregate ranking automatically. Based on the output, it appeared to be applying a basic positional weighting method. Items received more value when participants placed them near the top and progressively less value in each lower position.

I could not confirm the exact formula because Forms did not explain the calculation and I don’t have the patience to go look it up. The charts Forms pooped out weren’t necessarily wrong but they didn't pass the sniff test. This type of scoring can work well when you want to find the item that most people ranked near the top. But that does not always mean it was the item people preferred most strongly or chose over the other options most often. This kind of nuanced and expert judgement is something I see missed by most UXers in the field today.

Third-party research platforms have to select a default calculation that can work across many use cases. Even a product from a company such as Microsoft cannot determine the most appropriate interpretation for every study.

This type of discernment will become more important as automated and AI-generated analysis becomes common. The output may be polished and mathematically correct while still answering a different question from the one the research was designed to address.

Testing Alternatives

So what did I do about it? First, I downloaded the individual responses into Excel and kept one row for each participant. I converted each answer into a numerical ranking from one through eight and confirmed that every respondent had used each rank once. I then tested two alternative approaches.

The first method I tried was a combined Borda scoring with pairwise wins. Borda assigns points according to where an item appears in each participant’s ranking. Pairwise analysis compares every item directly with each of the other seven items.

For example, the analysis would compare “check the status of a request” with “contact support” and count how many participants placed each task higher. The same comparison would then be completed for every possible pair.

I used the number of pairwise wins as the primary measure and the Borda score as the tie-breaker. This approach emphasized direct preference while still recognizing items that participants placed consistently near the top.

The second approach was the Schulze method. Schulze also begins with pairwise comparisons, but it considers the strength of the paths connecting the items. This allows it to resolve situations where preferences form cycles and no item defeats every alternative cleanly.

Schulze is a strong method when the analysis needs to produce one formal winner from complicated pairwise results. Its practical weakness is explainability. The calculation is more difficult for stakeholders to review, understand and reproduce.

Different Methods, Different Stories

The following example uses fictional tasks and a reconstructed response pattern. It illustrates the analytical issue without using any company information or reproducing the actual study content.

Something like this:

Method #1: MS Forms

The default weighted method, the MS Form’s method, selected “check request status” because participants consistently placed it near the top. It had broad support and relatively few low rankings.

Method #2: Schulze

Schulze selected “contact support.” It received the most 1st votes and performed strongly in the direct comparisons. The method looked at how often participants ranked “contact support” above each of the other 7 tasks, then considered the strength of those head-to-head wins across the full set of comparisons. That gave it the strongest overall path through the rankings, even though it did not have the highest positional score.

Method #3: Borda and Pairwise Combo

The pairwise-plus-Borda approach produced a tie between “download a document” and “contact support.” Each defeated six of the other seven tasks. “Download a document” then won the Borda tie-break because participants placed it more consistently across the complete rankings.

A similar problem appears in the ForceRank.it article “Counting Votes Is Hard.” Four people ranked 9 possible topics for a technical presentation. Three people placed Item 1 first, but ForceRank’s original point-based scoring method selected Item 2. Item 2 performed consistently well across all four rankings, while Item 1 was the majority favorite but was placed last by one participant. ForceRank eventually replaced its original method with the Schulze method because the original calculation failed the Majority Criterion. Schulze evaluates the items through pairwise comparisons and produces a winner that better reflects the group’s overall preferences.

Conclusion

For this study, I chose pairwise wins with Borda as the tie-breaker. That combo method was understandable, reproducible and appropriate for a directional sample of 32 respondents. It also made it possible to explain why the methods produced different results without presenting one calculation as universally correct.

I compared every item directly with the other seven and counted how many comparisons it won. When two items had the same number of wins, I used the Borda score as the tie-breaker. I also reported first-place votes and the full rank distributions so stakeholders could see the preference pattern behind the final order.

MS Forms collected the responses and provided a reasonable default summary. My responsibility was to determine whether that summary matched the stakeholder’s question.

The most important analytical skill in this project was knowing when the default answer needed a second look. The tools will keep getting better, but our responsibility stays the same. We need to understand the method well enough to know when the default output does not fit the research question. I hope sharing this story was helpful. Thanks for reading.

Over the past couple years, I have written many articles about UX ideas that sound reasonable in theory but get used very differently in the real world. I realized these 5 articles fit together because they all address the same underlying issue. Many UXers take something valuable, strip away its original intent, and use their own bastardized version to defend bad UX work.

This week’s post is a reference list for 5 articles that push back on that pattern. Each one attempts to bring the conversation back to the user, where it belongs. I hope you enjoy it and consider sharing the link with others.

5 Things UXers Need to Stop Doing

1. Stop Weaponizing Jakob’s Law

This article explains what Jakob’s Law actually means and how it gets twisted during real-world UX discussions. UXers often find one product that uses a pattern they already like, or point to something Google does, and claim that users must already understand it.

This post offers a more rigorous exercise for identifying patterns across comparable products so teams can distinguish a genuinely familiar convention from a novel design idea.

2. Stop Weaponizing Conversational UX

This article pushes back on the idea that Conversational UX simply means making interface copy sound casual or friendly. I hear this misinterpretation more and more every day in the context of chatbots. Instead, Conversational UX is about how interactions unfold in the real world. You know, rather than giving UX charlatans an excuse to make deals with the marketing dark-pattern devil.

This article includes a practical content-testing exercise that helps UXers identify when a GUI is supporting the user’s task and when supposedly conversational language is being used to interrupt, manipulate, shame, or push an upsell.

3. Stop Throwing Away Time-on-Task Data

This article is about using time-on-task data responsibly in small-sample usability studies.

It explains why the median is usually a better starting point than the average, especially when one participant experiences a severe delay that skews the data. Instead of throwing out the extreme case or allowing it to redefine the typical experience, the article recommends reporting the median, range, individual cases, and observed behavior together so teams understand both the general experience and the specific failure paths.

4. Weaponized Institutional Knowledge in UX

This article looks at what happens when institutional knowledge stops being useful context and starts being used to shut down UX work.

Claims like “our users are different” frequently become conversation enders that allow teams to ignore established best practices without providing any evidence for the exception. The post argues that best practices should remain the starting point and that the burden of proof should fall on the person making the counterclaim, not on the researcher being asked to disprove an unsupported belief.

5. Why Your UI Style Guide Sucks

This article is about the difference between documenting how a UI control looks and defining how it actually works. Using button design as an example, it walks through the interaction states and attributes that are routinely missing from modern style guides, leaving developers to make important UX decisions on their own.

The broader point is that UXers need to understand and document the functional behavior behind the controls they use, not just their colors, spacing, and visual appearance.

Conclusion

I think these articles work well as a set because they all challenge practices that have become too easy for UX teams to accept.

These posts are not about making UX more complicated or demanding research for every minor decision. They are about maintaining a reasonable standard and refusing to let convenient shortcuts replace evidence and a clear understanding of user behavior.

Thanks for reading. I hope having these articles in one place makes them easier to use and share. I would love to hear about the bad UX practices you regularly encounter and believe our profession should stop accepting.

I don’t know about you, but I still see a surprising amount of UX debates around things we figured out decades ago. Do any of these sound familiar?

• “Fewer clicks are always better.”

• “Text is clutter and makes the design look less clean.”

• “These icons don’t need labels.”

• “We don’t need to do any IA testing; we’ll just add search.”

• “The AI chatbot will handle all of this.”

• “Let’s not run research unless we can get a bigger sample.”

None of these statements are automatically incorrect by default, but each is certainly misguided. They are based on common misinterpretations and misrepresentations of replicated, well-established HCI behavioral research. And what’s worse is that they sound just enough like the truth to sneak by a design review or stakeholder conversation without sounding obviously wrong. That is what makes them dangerous.

Over the last 20+ years, I’ve seen these 10 things come up time and time again. I wanted to write them out here to inform those of you who may hold these misunderstandings. I also wanted to help those of you who know the facts but need support identifying them in the real world and combating them before they cause trouble in your own day-to-day work.

So, drumroll please… with that in mind, here are the 10 most common UX misunderstandings I see in the real world, in no particular order.

1. Believing search or an AI chatbot can compensate for bad navigation

The common misunderstanding for this is that users will “just search,” or now, “just ask the chatbot.” That sounds modern and user-centered, but it often becomes an excuse to avoid the harder work of information architecture.

These 3 sources together help explain what we already know about this:

1. MeasuringU’s article, Search vs. Browse on Websites, frames search as something that supports navigation, not something that replaces it. Their point is practical: even with good navigation and a user-refined taxonomy, some items will still be difficult to find, and some users will still prefer to search. The stronger pattern is not search instead of navigation, but good search supporting good navigation.

2. Baymard Institute's article titled Ecommerce Search UX 2026 makes the same broader point from another direction. Across their ecommerce testing, roughly half of participants turned to search as their preferred product-finding strategy, while the other half used the main navigation. That means product finding has to support both behaviors well.

3. NN/g’s article, What Is Your Site’s AI Chatbot for? Users Can’t Tell, adds the newer version of this mistake. Users often do not know what site-specific AI chatbots are for, and chatbots have to prove they solve problems that existing site features do not already solve.

For example, think about a financial platform that buries critical wire transfer limit information under a vague settings menu and assumes users will search with the right query on the first try. (Side note: we also know that users have trouble writing good queries, but don’t tell your chief AI officer. hahaha) Or think about a healthcare admin tool that hides provider eligibility logic in help documentation and assumes the chatbot will answer in a way that maps to the user’s specific plan.

Traditional navigation bars should be built to match the user’s mental models and use terms they are familiar with at a glance. Search should support known-item finding, recovery, and cross-category lookup. AI should have a clear job, clear boundaries, and obvious value beyond what normal navigation and search already provide.

2. Optimizing for click count instead of information scent

This misunderstanding claims that fewer clicks automatically mean better UX. This is the old three-click rule popularized by charlatan PMs who are relying only on a few A/B test studies run in Silicon Valley a decade ago. During the prime “move fast and break things” era, no less. It keeps coming back because it is easy to understand, easy to measure, and easy to say in a meeting, but it is built on flimsy data.

Check out these sources for a better explanation:

1. NN/g’s article, The 3-Click Rule for Navigation Is False, directly rejects the idea that every important piece of information must be reachable within three clicks. Their point is not that click count never matters. It is that click count is a weak proxy for usability. A longer path with clear labels and a strong sense of direction can work better than a shorter path full of vague choices.

2. NN/g’s article, Information Scent: How Users Decide Where to Go Next, explains the deeper principle. Users decide where to go based on the cues they get from link labels, surrounding context, and their prior experience. The real question is not “How many clicks does this take?” The better question is “Can users confidently predict where this path will lead?”

3. MeasuringU’s article, Using Tree-Testing To Test Information Architecture, gives teams a practical way to evaluate this. Tree testing helps measure whether users can find items in a navigation structure, which is much more useful than simply counting the number of steps in a proposed flow.

This shows up when a product team combines several workflow steps into one crazy dense screen because the task becomes “shorter.” Think the typical bad dashboards we’ve all been seeing lately. hahahaha. A design like that may collapse several task flows into a single overloaded menu because it technically reduces navigation depth.

The research-based recommendation here is to optimize for task-path clarity, not shortness. Clicks still matter, but they are not the main unit of usability. A click with strong information scent is cheap. A click into uncertainty is expensive. Think about how much better it is to file your taxes one field at a time via TurboTax compared to one long, scrolling tax web form. Wayyyyyy more clicks, but a wayyyyyy better experience.

3. Misusing progressive disclosure to hide essential choices

This one claims that hiding information automatically makes an interface simpler. It’s a misinterpretation of the “less is more” idea. Progressive disclosure is a real and useful UX standard, but it is often misused as a way to make the screen look cleaner while pushing important decision-making work onto the user.

These 4 NN/g articles explain it well:

1. NN/g’s article, Progressive Disclosure, defines the pattern as a way to defer advanced or rarely used features so interfaces are easier to learn and less error-prone. That is the key phrase: advanced or rarely used. Progressive disclosure works when the hidden information is secondary. It breaks down when the hidden information is essential to the user’s decision.

2. NN/g’s article, Tooltip Guidelines, makes this boundary very clear. Tooltips should not contain information that is vital to task completion. Users should not have to discover a tooltip in order to understand a requirement, a consequence, a fee, a permission rule, or a decision-critical explanation. (You hear that, Adobe? Hahaha.)

3. NN/g’s article, Don’t Use Split Buttons for Navigation Menus, shows another version of the same problem. Split buttons often create ambiguity because one part of the control does one thing and another part does something else. That may look compact, but it can make the user slow down, hesitate, or misunderstand what the control will do.

4. NN/g’s article, Accordions on Desktop: When and How to Use, is also useful here because it frames accordions as a pattern that requires care. Accordions can help organize long or complex content, but users should have control over what is expanded, and the content structure needs to support the task rather than hide what people came to find.

Have you ever seen critical information hidden behind an “i” icon, required policy rules revealed only on hover, pricing caveats tucked into a tooltip, advanced filters buried in collapsed panels, or an accordion that contains the exact information users need before they can make a choice? If so, that’s this misunderstanding’s impact in the real world. It also shows up in split buttons where the visible action does one thing, the caret does another thing, and the user has to understand that distinction before safely using the control. I think you get it.

The recommendation here is to make decision-critical information visible at the moment of decision. (Just-in-time and contextual.) Use progressive disclosure for advanced options, supporting details, definitions, examples, or extra configuration.

4. Thinking form “simplicity” means fewer labels or fewer steps, rather than less user effort

This misunderstanding is that a form is simpler when it looks cleaner, shorter, or less text-heavy. In reality, forms are simple when users can complete them with low uncertainty, low memory load, and low recovery effort.

Check these sources out:

1. Baymard Institute’s article, Mobile Form Usability: Never Use Inline Labels, calls inline labels a prime example of false simplicity. They look simple because the screen appears less cluttered, but they become harder to use once the user starts typing, reviewing, or correcting information. Baymard observed users deleting their own input just to see the label again.

2. Baymard Institute’s article, 3 Types of False Simplicity, explains the broader pattern. Some design choices appear simpler at first glance but actually make the interface more complicated to use. This is one of the most common form-design traps: reducing what is visible on the screen while increasing the amount of work users have to do in their heads.

3. NN/g’s article, Placeholders in Form Fields Are Harmful, makes the same point from another angle. Placeholder text inside a form field makes it harder for users to remember what belongs in the field, check their answers, and fix errors. This is especially important in longer or more complex forms.

4. NN/g’s article, Marking Required Fields in Forms, adds a related recommendation: required fields should be clearly marked. A form should not make users infer which fields matter, guess what they can skip, or discover requirements only after submission.

In complex interfaces, this gets painful quickly. Think about onboarding a B2B onboarding experience. The form may ask for IDs, dates, roles, plan types, addresses, exceptions, documents, thresholds, and routing logic. When placeholder labels disappear, input instructions are hidden under a hover, fields have no groupings, or required-field logic is unclear, users are left managing the form in their heads.

Function before style. Keep all labels persistent. Group related fields. Put instructional text where users need it. Mark required fields clearly. Break complicated tasks into meaningful sections when that supports comprehension, but do not split or compress steps just to create the illusion of simplicity. These are the attributes of an easy-to-use form.

5. Designing validation as a gate instead of a recovery system

The common misunderstanding for this is that validation exists to catch users when they make a mistake. That framing is not totally wrong, but it is very, very incomplete. Validation should help users recover, not merely block them from moving forward.

Read more here:

1. NN/g’s article, Error-Message Guidelines, says effective error messages should be highly visible, constructive, and respectful of user effort. That last part matters. The user already did work. A good error message should help preserve that work and guide them toward repair.

2. NN/g’s article, 10 Design Guidelines for Reporting Errors in Forms, gives a more specific form-level recommendation. Error messages should be placed next to the field causing the problem, and they should be explicit, human-readable, polite, precise, and constructive. The user should not have to hunt for the problem or translate the system’s complaint into an action.

3. Baymard Institute’s article, Usability Testing of Inline Form Validation, adds the timing issue. Inline validation can save users time and effort, but it has to be implemented carefully. Avoiding premature validation, removing errors once input is corrected, and using positive inline validation can all improve the experience.

4. Baymard Institute’s checkout guidance, Ecommerce Checkout UX Guide, also calls out a critical recovery issue: if submission fails, preserve all entered data and guide users directly to the error. Clearing the form after an error is a major failure because it forces the user to redo work they already completed.

UI component validation often has to handle complex rulesets. For example, a payment field may fail because of a card type, billing address, account status, or fraud rule.

The recommendation here is to design validation around error recovery. Show the problem near the source. Explain what went wrong in human language. Preserve the user’s work. Validate at the right time, not too early and not too late. (Again, just-in-time and contextual.) Confirm success when that confirmation helps the user move forward.

6. Treating filters and sorting as nice-to-haves

This one states that once a list, table, catalog, or search-results page exists, filtering and sorting are optional enhancements. (Spoiler, they are not. hahaha.) For many tasks, filtering and sorting are the main way users turn a large information space into something manageable.

These 4 sources detail this well:

1. Baymard Institute’s research page, E-Commerce Product Lists & Filtering UX, frames filters and sorting as central to product finding. With the right filters and a clear filtering interface, users can narrow a huge list down to a smaller set of relevant options. With solid sorting, users can order items by the attributes they care about.

2. Baymard Institute’s article, Filtering UX: 5 Essential Filter Types, makes the point even more directly. If users cannot adequately filter product lists, their ability to tailor the list to their needs is severely restricted, and they may be unable to find what they need.

3. Baymard Institute’s article, 4 Essential Sort Types, covers the other half of the problem. Sorting helps users manipulate product lists in a way that supports exploration and selection. Missing or poorly chosen sort options can make users work harder to compare the options that matter to them.

4. NN/g’s article, Filters vs. Facets: Definitions, is especially helpful for complex interfaces. Filters and faceted navigation both reduce a large set of content to a narrower display, but facets add more power by letting users narrow content across multiple aspects of the information space.

In complex interfaces, this is not just an ecommerce problem. It shows up in admin consoles, data tables, CRMs, analytics tools, document repositories, audit logs, claims queues, security event lists, and research repositories. Users may need to narrow by status, date, owner, segment, severity, region, product line, policy type, plan type, or exception reason. If the interface gives them a massive list and a weak search box, it is asking them to manually do what the system should help them do. (Again, users are not experts in writing queries, and they need your help with the context.)

The recommendation here is to treat filtering and sorting as core task infrastructure. Identify the criteria users actually use to make decisions.

Display applied filters clearly and make them easy to remove. For large or complex content sets, use facets that reflect how users think about the domain, not just how the database happens to be structured.

7. Forcing account creation before the user completes the main task

The misunderstanding is that account creation is a harmless step because the user will need an account anyway. From the business side, that may feel quasi-reasonable, but it’s really not. From the user side, it often feels like an interruption that happens before the task they actually came to do.

Check out these 3 Baymard Institute articles to learn more:

1. Baymard Institute’s article, Make “Guest Checkout” Prominent, explains that promoting guest checkout reassures users who are resistant to creating an account while still allowing returning users to sign in. Baymard also recommends limiting the account-selection step to “Checkout as Guest” and “Sign In to My Account” rather than presenting several competing options at once.

2. Baymard Institute’s article, Save Account Creation for the Confirmation Step, explains why timing matters. Saving account creation for the confirmation step removes one additional decision from checkout until the user has finalized the order. That reduces friction during the primary task.

3. Baymard Institute’s article, 4 Ways to Improve the Post-Checkout UX, adds that users may be more willing to create an account after checkout because much of the necessary information has already been entered. At that point, account creation can often be reduced to creating a password rather than completing a whole new workflow.

Examples of this are when a user tries to download a document and gets forced into account creation. Or a B2B buyer wants to view implementation documentation and gets a lead form instead. Cringe.

The research-based recommendation is to delay account creation until it is necessary, or until the user has received enough value to understand why it is being requested.

8. Assuming icon-only controls are self-explanatory

This misunderstanding claims that the meaning of an icon is always obvious and that users will learn your unique icon language over time. UXers, PMs, engineers, and stakeholders often become fluent in their own product language, which makes this one seem true, even though there are decades’ worth of research proving this is not the case. Users do not automatically share that fluency.

These 3 NN/g sources detail the foundational findings:

1. NN/g’s article, Icon Usability, explains that truly universal icons are rare. Only a very, very, very small number of icons have relatively broad recognition with most users. Outside of those extremely rare instances, most icons remain ambiguous because the same symbol can mean different things in different interfaces.

2. NN/g’s video, Yes, Icons Need Text Labels, makes the practical recommendation clear. Because most icons are ambiguous, a text label should be present alongside the icon to clarify its meaning. Labels reduce guesswork.

3. NN/g’s video, Tips for Icon Usability, adds a broader testing point. Teams should not assume users recognize an icon just because the icon is visually clean, common inside the product, or familiar to the design team. Icons need to be understandable to the actual people using the interface.

In complex interfaces, this gets especially painful because the actions are often abstract. A trash icon is one thing. A “reconcile variance” icon, or “map field” icon are other beasts entirely.

The recommendation here is clear as day, and still, many of you will try to disregard it. It is a best practice to label all icons. Period. Full stop. It’s best for usability, accessibility, and code compliance standards.

9. Making hover menus appear instantly because it feels responsive

The common misunderstanding for this is that instant hover behavior is better because it makes the interface feel fast. But fast is not always the same as usable. When hover behavior triggers too easily, the interface can feel jumpy, fragile, and distracting.

These 3 Baymard Institute articles are all you’ll need for this one:

1. Baymard Institute’s article, Provide a Hover Delay of 300–500 MS for Hover-Based Drop-Down Menus, explains the issue directly. Hover-based drop-down menus can be useful for exposing categories and subcategories, but they are prone to flickering when they trigger too quickly. Baymard recommends a 300–500 ms hover delay to reduce accidental triggering.

2. Baymard Institute’s article, 2024 Product Finding Research Update, shows that this is not just an old or theoretical problem. Baymard notes that menu flickering was first identified as a main navigation issue in 2014 and still appeared in their latest product-finding testing ten years later.

3. Baymard Institute’s article, Drop-Down Usability, is useful for the broader pattern. Drop-downs can be appropriate, but they need to match the user’s task and the structure of the available options. Just because something can be hidden behind a triggered menu does not mean that is the best interaction pattern.

They show up in side navigation, data-table row actions, dense dashboards, chart controls, map layers, nested settings menus, and toolbars. A user trying to move from one area of the screen to another accidentally opens a panel that covers the thing they were trying to inspect. A user trying to select a menu item has to steer diagonally through a narrow activation path. A user with less precise pointer control gets punished for moving like a human. We’ve all done these things ourselves, yet we keep designing hover-only stuff. Why?

The design recommendation here is to treat hover as fragile. Add an appropriate delay and avoid large accidental-trigger zones. Preserve the user’s path to the submenu. Do not cover important content unnecessarily. Make sure the same action can be completed without relying on hover alone, especially for touch, keyboard, and assistive technology users. When in doubt, don’t hide it under a hover.

10. Believing UX findings are invalid unless the sample size is large

This common misunderstanding claims that usability findings do not count unless the study has a large sample. Some teams hear “five users” and immediately dismiss the study as anecdotal. Other teams go too far in the other direction and treat five users as a magic number that works for every research question.

Read these 4 MeasuringU articles for a better understanding:

1. MeasuringU’s article, Do Statistics Really Require 30 Participants?, addresses another common myth: that statistics automatically require at least 30 participants. Their point is that rigid sample-size rules are often misapplied. The appropriate sample size should come from the study goal, metric, method, and intended claim.

2. MeasuringU’s article, Sample Sizes for Usability Studies: One Size Does Not Fit All, gives the most useful framing. Sample size depends on the type of study. MeasuringU separates usability studies into problem discovery, estimation, and comparison. Those are different research goals, and they require different sample-size logic.

3. MeasuringU’s article, Why You Only Need to Test with Five Users Explained, explains the well-known small-sample argument with important caveats. Five users are more likely to reveal common problems than rare ones. The point is not that five users will find everything. The point is that small formative studies can be useful when the goal is to identify common usability problems and iterate.

4. MeasuringU’s article, What Do You Gain from Larger-Sample Usability Tests?, explains when larger samples matter. It is easy to see common problems with a small sample, but larger samples are needed to detect uncommon issues, estimate performance more precisely, or support stronger summative claims.

In real product orgs, this mistake often becomes a convenient veto. A researcher brings evidence that users cannot find a setting, misunderstand a filter, miss an error, or misinterpret an icon.

The research-based recommendation here is to match the method to the claim. Small formative studies are appropriate when the goal is to identify and understand usability problems, especially common ones. Larger samples are appropriate when the goal is benchmarking, estimating rates, comparing designs, or detecting less common issues. Do not pretend “small-n research” can answer every question. Also do not pretend every UX question requires survey-scale data before the team is allowed to fix a broken interaction.

Conclusion

I hope you can see the bigger pattern across all 10 of these misunderstandings is that teams often mistake reduced visible complexity for reduced user effort. That is the trap!

Complexity does not disappear because we collapsed it, renamed it, or hid it behind an icon. It either gets handled by the interface, or it gets handed to the user. And users already have enough work to do, so let’s stop being aesthetically and ego-driven and put their needs first for a change, shall we? Thanks for reading!

As UX researchers, we almost always need data that lives in systems we do not own. 🎵Tale as old as time.🎵 Unfortunately, many CRMs are structured from the sales team’s point of view, which is not always useful when we are building study participant lists. Endless fields for accounts, territories, renewal dates, account owners, and deal stages may work well for sales because they are often managing relationships at the account level. But us researchers need a different structure. Depending on the study, a useful dataset might need one row per person, one row per survey response, one row per support issue, or one row per task, session, or behavior. The data can be accurate in the source system and still require cleaning before it can support the research work.

This is a big reason basic data cleaning skills are critical for being a pro UX researcher.

When the structure of the data does not match the research need, you want the confidence to clean it yourself instead of waiting for someone else to find the time to do it for you. Lucky for us, it is not that hard. so, that is what this week’s article is all about.

What’s in Your Rows?

One of the first cleaning decisions is deciding what one row should represent. This sounds basic, but it affects almost every cleaning choice that follows. Before removing duplicates, splitting cells, standardizing categories, or fixing missing values, the research team needs to understand the unit of the dataset.

In business data, one row can mean many different things. It might represent one company, one account, one household, one support ticket, one purchase, one product event, one survey response, or one person. None of those structures is automatically wrong. The right structure depends on the research purpose.

For us UX research recruiting, one row usually needs to represent one individual person. That person may need to be invited, screened, scheduled, tracked, thanked, and excluded from future outreach for a period of time.

This issue comes up often in B2B2C orgs. A CRM export may have one row per customer company, with several contacts stored inside that same row. That can make sense for sales because the account is the main unit of work. It is less useful for UX research when the team needs to contact individual people for study participation.

For example, a CRM export might look like this:

That row contains useful info, but it is not yet structured for recruiting. Maria, James, and Priya are three different people. They may have different roles, different product experiences, and different levels of fit for a study. If they remain bundled inside one company row, the research team cannot easily track outreach, screening status, participation history, or eligibility at the person level.

A research-ready version would look more like this:

See the difference? This change is small in the spreadsheet, but meaningful for the research workflow. The company-level info still matters, so it stays attached to each person. The difference is that the cleaned file now supports the way the research team needs to work. Each row can be treated as a possible participant rather than as part of a bundled account record.

Keep It Traceable

Once the team knows what the cleaned data needs to become, the next step is to keep the cleaning work traceable. An important best practice that I see soooooo many UX researchers dismiss is keeping the original export untouched. The cleaning should happen in a separate workspace, and the final cleaned version should be easy to compare back to the source data.

In your spreadsheet, you might use tabs like this:

• Raw Export for the untouched source data.

• Cleaning Workspace for the active cleanup work.

• Cleaned Data for the version that is ready for recruiting or analysis.

• Review Needed for records that require human judgment.

• Removed Records for records that were excluded.

• Duplicate Records for records that may appear more than once.

• Cleaning Notes for decisions, rules, and changes made during the cleanup process.

This structure matters because cleaning involves judgment. You’ll need to decide whether two rows are duplicates, whether a category should be standardized, whether a contact should be excluded, whether a name and email address appear to belong together, stuff like that. Those decisions should not disappear into one final spreadsheet.

Traceability is especially important when UX is using data from a system owned by another department. If a stakeholder asks where a contact came from, the team should be able to point back to the original CRM row. If an account owner questions why a person was included or excluded, the team should be able to explain the cleaning decision. If the same source data needs to be refreshed later, the team should be able to repeat the process without starting over.

A few simple fields can make this easier:

• Original Row ID connects each cleaned record back to the source export.

• Cleaning Notes explains unusual decisions or changes.

• Review Needed separates records that are ready to use from records that require human judgment.

These fields may feel administrative, but they make the cleaned data easier to trust.

The goal is to avoid creating a file that only one person understands. Cleaned research data should be usable by the person who made it, but it should also be understandable to another researcher who opens it later. That is part of making data cleaning a research practice rather than a private spreadsheet task.

Make Each Field Usable

Once the overall structure of the dataset is defined, the next step is making sure each field contains the type of info it is supposed to contain. This sounds obvious, but many business exports contain fields that have gradually accumulated formatting inconsistencies over years of use. Those inconsistencies may not bother the team that owns the system, but they often create unnecessary work for researchers trying to filter, sort, join, or segment the data.

Many of these problems are small on their own. Together, they make the spreadsheet harder to work with and increase the chance of mistakes during recruiting or analysis. Before doing anything more advanced, it helps to standardize the basic building blocks of the dataset.

Common cleanup tasks include:

• Removing extra spaces before or after values.

• Standardizing capitalization where appropriate.

• Converting email addresses to lowercase.

• Fixing inconsistent date formats.

• Removing hidden line breaks or non-printable characters.

• Splitting fields that contain several pieces of information into separate fields.

• Making sure every column contains one type of information.

For example, an email column should contain email addresses. A segment column should contain one segment value. A date column should contain dates that Excel recognizes as dates. If a field contains several different kinds of info, it becomes much harder to filter the data, compare records, or identify patterns.

This is also a good point to standardize delimiters before doing more advanced cleaning. One export may separate multiple values with commas, another with semicolons, and another with line breaks. Choosing one consistent delimiter early makes the remaining cleaning work much simpler.

Expanding Rows

As discussed earlier, one of the more common problems in CRM exports is finding several related values stored inside the same row. Let’s use the same example as before to illustrate this point.

Suppose a CRM export contains this info:

The research team will eventually needs three rows instead of one, like this:

The important part of this transformation is preserving the relationships between the values. Maria’s email should stay with Maria. James’s email should stay with James. The company, segment, and any other account-level info should remain attached to all three contacts because that info still applies to each person. This is where I see many cleaning mistakes happen. Don’t worry, I mess this stuff up all the time too! It is easy to split one column and accidentally lose the relationship to another column. If the names are separated first and the emails are separated independently, there is no guarantee they will still align correctly. Once that relationship is lost, it often requires manual work to reconstruct it.

When I perform this type of cleaning, I usually think about whether the values belong together before deciding how to split them apart. If two fields describe the same person, they should be expanded together. The goal is to create rows that still represent real people with the correct info attached to them.

Missing & Unclear Data

Spoiler alert, no dataset is perfect and records will be incomplete. Others will contain info that is difficult to interpret with confidence. Both situations deserve attention because they affect recruiting decisions.

In almost all cases, it is better to leave uncertainty visible than to replace it with info that may be wrong.

Simple review fields can make these situations much easier to manage:

• Missing Email

• Missing Role

• Unknown Segment

• Needs Review

• Unclear Name-Email Match

Consider a CRM row like this:

James and Priya can probably be matched with reasonable confidence. Maria is much less obvious because the remaining email appears to be a shared mailbox rather than an individual contact. That record should be flagged for review instead of being assigned automatically.

The same thinking applies to missing info. A contact without an email address may still belong in the cleaned dataset if another team can supply the missing info later. A contact without a job title may still be useful if the study includes a screener that verifies the participant’s role. The point is to make the uncertainty easy to identify so the research team can make informed decisions during recruiting.

Dedupe 101

Duplicates are one of the most common problems in data. They are also one of the easiest problems to underestimate. A spreadsheet may look like it contains hundreds of possible participants, but that number can shrink quickly once duplicate people, companies, accounts, or email addresses are removed.

The same person might be invited twice. One company might be overrepresented in a study. A team might think it has a larger sample than it really does. Duplicate records can also make it harder to track who has already participated, who has already declined, and who should not be contacted again for a while.

Common duplicate patterns to look out for:

• The same email address appearing in multiple rows.

• The same person listed with slightly different names.

• The same company listed under different spellings.

• A parent company and subsidiary both appearing as separate accounts.

• The same participant appearing in several old recruit lists.

• The same contact appearing once in a CRM export and once in a marketing list.

Email address is often the strongest field for deduping individual contacts, but it should not be the only field the team checks. People can have more than one email address. Companies can change domains. Some records may use a generic inbox instead of an individual email. Exact matches are helpful, but they do not catch every duplicate.

This is why deduping should include both automatic checks and human review. Excel can highlight repeated emails, repeated company names, or repeated combinations of name and company. The team should then move questionable records to a review tab instead of deleting them immediately.

A practical deduping pass might include these steps:

1. Standardize email addresses by trimming spaces and converting them to lowercase.

2. Highlight duplicate email addresses.

3. Check duplicate combinations of name and company.

4. Review similar company names before merging them.

5. Move uncertain duplicates to a separate review tab.

6. Document which records were removed or kept.

Match the Real World

A cleaned contact list still needs to match the reality of the study. A person can have a valid name, a valid email address, and a company attached to the record, but still be wrong for the research. This is why cleaning has to connect back to the recruiting criteria.

If a study needs healthcare administrators from enterprise accounts, the spreadsheet needs consistent fields that support that filter. The segment field should not contain several versions of the same category. The role field should not mix job titles, departments, notes, and guesses. The customer tier field should be clear enough that the team can tell which accounts belong in the sample.

Useful fields for recruiting might include:

• Cleaned segment

• Cleaned role

• Customer tier

• Product line

• Region

• Account owner

• Likely eligibility

• Eligibility confidence

• Account owner approval needed

• Do not contact status

The cleaned fields should support the recruiting decision without hiding the original data. It is often better to create a cleaned version of a field than to overwrite the source value. For example, the original CRM value might say ENT, while the cleaned segment field says Enterprise. That makes the data easier to use while still preserving the source context.

For example, a simple mapping table might look like this:

The research team should also be careful with company-level signals. Even when a company is in the right segment or owns the right product, that does not mean every contact at that company is a “good” study participant. Some contacts may not use the workflow being studied, and others, like senior sponsors, may be important to the account without being close enough to the day-to-day experience. We all know, this is where a good screener comes into play.

The same logic applies to outreach appropriateness. Some contacts should not be used even if the record looks complete. Generic inboxes, distribution lists, former employees, personal email addresses, and contacts who require account owner approval should be flagged before outreach begins.

A useful recruiting-ready status field might include:

• Ready for outreach

• Needs screener

• Needs account owner approval

• Needs data review

• Do not contact

• Not eligible for this study

This keeps the cleaned file connected to the real recruiting workflow. The final spreadsheet should help the team decide who can be contacted, who needs more review, and who should be excluded from the current study.

14 Cleaning Formula Templates

⚠️ Disclaimer: These examples assume your CRM export has been converted into an Excel table named CRM. Replace the table name and column names with the names used in your own spreadsheet.

1. Create an original row ID

Use this when you want every cleaned record to trace back to the original export row.

="CRM-"&TEXT(ROW()-ROW(CRM[#Headers]),"00000")

Example result:

CRM-00001
CRM-00002
CRM-00003

This is useful before splitting one CRM row into several person-level rows.

2. Clean a contact name

Use this to remove extra spaces and hidden characters from a name field.

=TRIM(CLEAN([@[Contact Name]]))

Example:

"  Maria Lopez  " → "Maria Lopez"

3. Clean an email address

Use this to remove extra spaces, remove hidden characters, and convert the email address to lowercase.

=LOWER(TRIM(CLEAN([@Email])))

Example:

"  Maria.Lopez@Acme.com  " → "maria.lopez@acme.com"

4. Flag missing recruiting fields

Use this when you want to quickly identify records that are not ready for outreach.

=TEXTJOIN("; ",TRUE,
IF([@[Clean Email]]="","Missing email",""),
IF([@[Clean Role]]="","Missing role",""),
IF([@[Clean Segment]]="","Missing segment",""))

Example result:

Missing email; Missing role

If the result is blank, none of those fields are missing.

5. Create a simple ready-for-review status

Use this when you want a cleaner status field instead of a long list of missing fields.

=IF(OR([@[Clean Email]]="",[@[Clean Role]]="",[@[Clean Segment]]=""),"Needs Review","Ready")

Example result:

Ready

or

Needs Review

6. Flag generic inboxes

Use this to identify email addresses that may not belong to an individual person.

=IF(OR(
ISNUMBER(SEARCH("info@",[@[Clean Email]])),
ISNUMBER(SEARCH("support@",[@[Clean Email]])),
ISNUMBER(SEARCH("admin@",[@[Clean Email]])),
ISNUMBER(SEARCH("billing@",[@[Clean Email]])),
ISNUMBER(SEARCH("contact@",[@[Clean Email]]))
),"Generic Inbox","Likely Individual")

Example result:

Generic Inbox

⚠️ Disclaimer: This formula does not prove the contact is unusable, but it tells the research team to review it before outreach. It is only a quick review aid. A blank email may still need to be caught by a separate missing-field check, and a generic inbox list should be adjusted for your organization. You may want to add other patterns like hello@, team@, customerservice@, or department-specific shared inboxes that show up often in your data.

7. Flag duplicate email addresses

Use this to identify repeated email addresses in a cleaned contact list.

=IF([@[Clean Email]]="","Missing Email",IF(COUNTIF(CRM[Clean Email],[@[Clean Email]])>1,"Duplicate Email","Unique"))

Example result:

Duplicate Email

This is one of the fastest ways to catch repeated contacts before recruiting begins.

8. Flag duplicate name-and-company combinations

Use this when email is missing or when you want a second deduping check.

=IF(OR([@[Clean Contact Name]]="",[@[Clean Company]]=""),"Needs Review",IF(COUNTIFS(CRM[Clean Contact Name],[@[Clean Contact Name]],CRM[Clean Company],[@[Clean Company]])>1,"Possible Duplicate","Unique"))

Example result:

Possible Duplicate

This is useful because the same person can sometimes appear with different email addresses.

9. Standardize messy category labels with a lookup table

Create a small mapping table named SegmentMap.

Raw ValueClean ValueENTEnterpriseEnterprise CustomerEnterpriseHealth CareHealthcareMedicalHealthcare

Then use this formula in your cleaned segment column.

=XLOOKUP(TRIM([@Segment]),SegmentMap[Raw Value],SegmentMap[Clean Value],"Review Needed")

Example result:

ENT → Enterprise

This keeps the original CRM value visible while giving the research team a cleaner field for filtering and sampling.

10. Split multiple names into separate rows with TEXTSPLIT

Use this when a cell contains several names separated by semicolons.

=TEXTSPLIT(SUBSTITUTE(A2,"; ",";"),,";")

Example input:

Maria Lopez; James Chen; Priya Shah

Example output:

Maria Lopez
James Chen
Priya Shah

This is helpful for quick cleanup, but it only splits the names. If the row also has multiple emails that need to stay matched to the right names, Power Query is usually safer.

11. Split multiple emails into separate rows with TEXTSPLIT

Use this when a cell contains several emails separated by semicolons.

=TEXTSPLIT(SUBSTITUTE(B2,"; ",";"),,";")

Example input:

maria.lopez@acme.com; jchen@acme.com; priya@acme.com

Example output:

maria.lopez@acme.com
jchen@acme.com
priya@acme.com

This is useful for inspection, but be careful. Splitting names and emails separately can break the relationship between the person and the email address.

12. Power Query pattern for splitting names and emails together

Use this when a CRM row has multiple names in one cell and multiple emails in another cell, and you need one row per person.

For this example, let’s assume your Excel table is named CRM and starts with a row like this:

⚠️ Disclaimer: This template assumes the names and emails are separated by semicolons. If your data uses a different delimiter, change the ";" in the code to whatever your file uses.

Power Query M example:

let
    Source = Excel.CurrentWorkbook(){[Name="CRM"]}[Content],

    AddIndex = Table.AddIndexColumn(
        Source,
        "Original Row Number",
        1,
        1,
        Int64.Type
    ),

    AddOriginalRowID = Table.AddColumn(
        AddIndex,
        "Original Row ID",
        each "CRM-" & Text.PadStart(Text.From([Original Row Number]), 5, "0"),
        type text
    ),

    RemoveOriginalRowNumber = Table.RemoveColumns(
        AddOriginalRowID,
        {"Original Row Number"}
    ),

    SplitCleanList = (value as any, optional makeLower as nullable logical) as list =>
        let
            RawText = if value = null then "" else Text.From(value),
            Parts = List.Transform(
                Text.Split(RawText, ";"),
                each Text.Trim(_)
            ),
            NonBlank = List.Select(
                Parts,
                each _ <> ""
            ),
            Cleaned = if makeLower = true
                then List.Transform(NonBlank, each Text.Lower(_))
                else NonBlank
        in
            Cleaned,

    AddPersonTable = Table.AddColumn(
        RemoveOriginalRowNumber,
        "Person Rows",
        each
            let
                Names = SplitCleanList([Contact Names], false),
                Emails = SplitCleanList([Contact Emails], true),
                NameCount = List.Count(Names),
                EmailCount = List.Count(Emails),
                MaxRows = List.Max({NameCount, EmailCount}),
                SplitReview =
                    if MaxRows = 0 then
                        "Review Needed: missing names and emails"
                    else if NameCount <> EmailCount then
                        "Review Needed: name-email count mismatch"
                    else
                        "Split Complete",
                PersonTable =
                    if MaxRows = 0 then
                        #table(
                            {"Contact Name", "Email", "Split Review"},
                            {{null, null, SplitReview}}
                        )
                    else
                        Table.FromColumns(
                            {
                                Names,
                                Emails,
                                List.Repeat({SplitReview}, MaxRows)
                            },
                            {"Contact Name", "Email", "Split Review"}
                        )
            in
                PersonTable
    ),

    ExpandPersonRows = Table.ExpandTableColumn(
        AddPersonTable,
        "Person Rows",
        {"Contact Name", "Email", "Split Review"},
        {"Contact Name", "Email", "Split Review"}
    ),

    KeepUsefulColumns = Table.SelectColumns(
        ExpandPersonRows,
        {
            "Original Row ID",
            "Company",
            "Contact Name",
            "Email",
            "Segment",
            "Tier",
            "Account Owner",
            "Split Review"
        }
    )

in
    KeepUsefulColumns

This keeps the first name with the first email, the second name with the second email, and so on. It also keeps the company-level fields attached to each new person row.

Example output:

If the number of names and emails does not match, the template keeps the available values and marks the row with Review Needed: name-email count mismatch. If both the names and emails are missing, it keeps one review row instead of silently dropping the original CRM record.

⚠️ Disclaimer: This template assumes that names and emails are listed in the same order when the counts match. It does not prove that the name and email belong together. Use the match-confidence formula after this step as a review aid.

13. Add a match-confidence flag after splitting names and emails

Use this after the rows have been expanded. It gives the team a quick signal about whether the contact name and email address appear to belong together.

=LET(
name,LOWER(TRIM([@[Contact Name]])),
email,LOWER(TRIM([@[Email]])),
handle,IFERROR(TEXTBEFORE(email,"@"),email),
first,IFERROR(TEXTBEFORE(name," "),name),
last,IFERROR(TEXTAFTER(name," ",-1),""),
IF(OR(name="",email=""),"Review Needed",
IF(OR(
AND(LEN(first)>=3,ISNUMBER(SEARCH(first,handle))),
AND(LEN(last)>=3,ISNUMBER(SEARCH(last,handle)))
),"Likely Match","Review Needed"))
)

Example:

Maria Lopez + maria.lopez@acme.com → Likely Match
James Chen + jchen@acme.com → Likely Match
Maria Lopez + teamlead@acme.com → Review Needed

⚠️ Disclaimer: This formula checks the part of the email address before the @ symbol and looks for the person’s first name or last name. It also avoids checking very short name fragments because those can create false matches. This should still be treated as a review aid, not final proof that the name and email belong together.

14. Create a recruiting-ready status

Use this when you want one field that tells the team what to do next.

=IF([@[Do Not Contact]]="Yes","Do Not Contact",
IF([@[Generic Inbox]]="Generic Inbox","Needs Review",
IF([@[Missing Fields]]<>"","Needs Data Review",
IF([@[Eligibility Confidence]]="Low","Needs Screener",
"Ready for Outreach"))))

Example results:

Ready for Outreach
Needs Screener
Needs Data Review
Do Not Contact

⚠️ Disclaimer: This kind of field helps turn a cleaned spreadsheet into a working recruiting list. The formula assumes you already have helper columns such as Do Not Contact, Generic Inbox, Missing Fields, and Eligibility Confidence in your table. You can rename those fields or change the status logic to match your own recruiting process. Treat the output as a workflow flag, not as a final decision about whether someone should be contacted.

Conclusion

Us researchers often talk about study design, moderation, synthesis, and storytelling, but some studies are shaped much earlier. The quality of the research can be affected by the quality of the data used to find, select, and contact participants. This matters in our orgs because researchers depend on systems owned by sales, support, customer success, product, or marketing. and alllllllll that data still needs to be cleaned before it can support recruiting, sampling, screening, or outreach.

Basic data cleaning helps the team understand what one row represents, which contacts are duplicated, which fields are usable, and which records need review. It also helps prevent outreach mistakes that can damage trust with participants and internal partners. This is why data cleaning should be treated as part of ResearchOps. In all orgs, UX research often starts with data someone else owns, and the useful work is turning that data into something research can responsibly use.

For the record, I am usually allergic to self-help type books. But Muriel M. Wilkins’ Leadership Unblocked: Break Through the Beliefs That Limit Your Potential landed differently with me. This book is not about UX in any way, but as I read it, I could not stop thinking, “This is really a book about UX.” I was doing the transference in my head the whole time.

Wilkins is writing about the negative self-talk that limits leadership potential. But I’ve noticed these same patterns of negative self-talk show up all over UX work as well. What made the book especially useful to me is that the framework did not feel like generic self-help advice. The ideas are grounded in peer-reviewed research across psychology, neuroscience, cognitive science, behavioral science, and change management research, with references to work from reputable institutions like Harvard and Stanford. That made the “self-help” side of the book feel more connected to the real world, which I really appreciated. It is not just a book about leadership in the formal sense. It is a self-coaching framework for noticing the beliefs that shape how we respond when work gets frustrating, ambiguous, political, or hard. And that makes it highly transferable to UX. Here’s a breakdown of the book.

The Book’s Core Idea

Wilkins describes 7 hidden blockers:

1. I need to be involved.

2. I need it done now.

3. I know I’m right.

4. I can’t make a mistake.

5. If I can do it, so can you.

6. I can’t say no.

7. I don’t belong here.

Like I said, the book is research-based, drawing from the author’s decades of coaching conversations and research from a bunch of different fields of study. That matters to me because these blockers are not presented as random personality quirks or case study anecdotes. They are treated as learned patterns that can shape how people interpret situations, protect themselves, and respond under pressure in the real world.

That is where the self-coaching framework comes in. The work is not just to name the “blocker,” the intrusive negative self-talk. It is to notice it, understand how that blocker might have benefited you in the past, challenge the current cost of still carrying it, reframe it, and practice a different behavior.

Wilkins provides reframing examples for each of the 7 blockers in the book. That part especially caught my attention because reframing is also one of the most practical moves UX professionals can make. UX work constantly requires us to reframe vague requests, problem statements, stakeholder assumptions, product risks, and team conversations.

So I started asking myself a different question. What would these reframes look like if they were written specifically for UX?

The 4 Traps

As I transferred the book into UX-speak, the 7 blockers started to organize into 4 larger traps in my head:

1. The Over-Functioning Trap

2. The Pacing and Perfection Trap

3. The Partnership Trap

4. UX Leader Identity Trap

These traps are not about UX executives or people who formally run UX teams. They are about UX professionals at any level who are trying to create better product decisions through research, design, content, accessibility, systems thinking, and user understanding.

The traps are not signs that a UX professional is doing anything wrong. That is the tricky part to explain, and the book does a great job exploring how most of these blockers start with something valuable that once served or protected you, but now can block you.

Wanting to be involved can come from knowing that UX is often brought in too late. Wanting work done quickly can come from knowing that teams need timely input. Wanting to avoid mistakes can come from caring about quality. Knowing you are right can come from having strong evidence. Saying yes can come from wanting to be collaborative. Feeling out of place can come from years of being treated as support instead of a strategic contributor.

The Over-Functioning Trap

The Over-Functioning Trap combines 2 blockers:

1. I need to be involved.

2. I can’t say no.

This trap shows up when UX tries to personally carry too many decisions. This is where we confuse being included with providing value. We confuse being responsive with being valuable.

The Reframes:

“I need to be involved” becomes:

I need to be involved where I add the most value.

and

“I can’t say no” becomes:

I can’t say no when the work truly matters.

That is a hard shift because many UX professionals have been trained by their organizations to equate access with progress. If we finally get invited, we go. If someone finally asks for UX input, we provide it. If a team finally wants help, we say yes. But the more mature move is not always more involvement. The key takeaway here is sometimes the mature move is better involvement.

The Pacing and Perfection Trap

The Pacing and Perfection Trap combines 2 blockers:

1. I need it done now.

2. I can’t make a mistake.

This trap shows up when UX is either rushing from panic or waiting for perfect clarity before acting. On one side, urgency can make every decision feel equally important. A stakeholder asks for something quickly, the roadmap is moving, the release date is close, and suddenly UX is reacting at the speed of everyone else’s anxiety. We confuse moving quickly with contributing meaningfully.

On the other side, the fear of mistakes can make our credibility feel like something we have to protect at all costs. We want the research plan to be perfect. We want the design rationale to be airtight. We want the recommendation to be fully defensible. We want the artifact to be polished before anyone sees it. We confuse being careful with being untouchable within our orgs sometimes.

The Reframes:

“I need it done now” becomes:

I need it done with intention, not reaction.

and

“I can’t make a mistake” becomes:

I can make a mistake and still have impact.

With this pairing, both sides can create real problems, and I’m sure you can see how these two go hand in hand. If you rush, you will make more mistakes. But if you strive for pixel perfection, you may wait too long to influence the work while it is still flexible enough to change.

The Partnership Trap

The Partnership Trap combines 2 blockers:

1. I know I’m right.

2. If I can do it, so can you.

This trap shows up when UX assumes being right is enough to create influence. We all know that is a very easy trap to fall into because UX often does know something important. We may have the research. We may have watched the usability sessions. We may best understand the workflow. We may see the content problem, the accessibility risk, the broken mental model, or the disconnect between what the team thinks and what users actually do. But evidence and knowledge does not automatically create alignment.

Sometimes evidence can make our conclusions feel obvious before other people even begin to understand them.

We commonly confuse having knowledge with already having alignment, and we also confuse what we have learned with what others should already know. Confirmation bias can make the evidence feel obvious because it supports what we already believe, while the false consensus effect can make us assume other people will interpret the situation the same way we do.

The Reframes:

“I know I’m right” becomes:

I might be right, but I still need to translate.

and

“If I can do it, so can you” becomes:

If I can do it, I can help others learn it.

This is one of the most important UX lessons in the book. Let me clarify: this does not mean the dreaded “democratizing UX” meme. I am not talking about teaching others to do UX. I am talking about teaching people how to best utilize UXers in the real world. Our value is not proving that UX has the better answer. Our value is helping people understand what our expertise means for the decision. Bottom line: translation is part of the job. UX expertise becomes more useful when other people can act on it.

UX Leader Identity Trap

The UX Leader Identity Trap Trap connects to the final blocker:

• I don’t belong here.

This trap shows up when UX waits for permission to contribute to strategic work. A strategic room can make UX professionals feel like they are there to support, not contribute. The conversation may be about revenue, roadmap tradeoffs, operational efficiency, engineering constraints, sales pressure, market positioning, or delivery commitments. Because the language is not always explicitly about users, it can be tempting to wait until someone asks a UX-shaped question.

If a decision affects adoption, behavior, trust, comprehension, usability, accessibility, retention, or customer effort, UX has something to contribute. UX may not own the final decision, but that does not mean UX has no authority to shape it.

The Reframe:

“I don’t belong here” becomes:

I do belong here because my work impacts outcomes.

Our value is not waiting to be invited into strategy. Our value is bringing UX expertise into strategic work across the organization.

Again, this is not only for people with formal leadership titles. A researcher can bring evidence into a roadmap conversation. A designer can make a product risk visible before a solution is locked. A content designer can show how language affects comprehension and trust. An accessibility specialist can explain how an experience creates exclusion or risk. Design systems folks can connect pattern decisions to speed, consistency, and quality.

We need to internalize that UX belongs anywhere the decision depends on how people will understand, use, trust, or experience the product. In other words, basically every strategic decision. Hahahaha

Repeatable Self-Coaching Loop

The practical value of the book is that it gives you a way to work with these blockers instead of just feeling bad about them.

For UX, I would make the loop (the framework) very simple:

Notice. Name. Reframe. Practice. Unblocked.

• Notice the reaction.

• Name the blocker.

• Reframe the belief.

• Practice a different behavior.

• Get a little more unblocked, Rinse and repeat

Loop Triggers:

• Before saying yes to another request, notice whether you are trying to be helpful or trying to prove UX has value.

• Before joining another meeting, ask whether your presence will change the quality of the decision.

• Before pushing evidence harder, ask whether people understand what the evidence means.

• Before waiting for perfect confidence, ask whether a useful next step would create more learning.

• Before staying quiet in a strategic conversation, ask whether user behavior, experience quality, or product risk is actually part of the decision.

That is the usefulness of this book for UX. It gives us a way to slow down the automatic reaction and choose a more intentional move.

Conclusion

Leadership Unblocked is not a UX book, but you should buy it and read it as if it were one.

The book gave me language for patterns I see all the time in my day-to-day work. But more importantly, it gave me a way to reframe things and center my efforts around what is actually within my control.

UX professionals are often working inside systems that really do bring UX in late, undervalue discovery, misuse data, reward speed over learning, and treat experience quality as something to clean up after the real decisions have already been made. But even in those systems, our beliefs shape our next move. That is why the self-coaching frame works in the real world. And that is how UX gets unblocked.

I recently wrote a couple of articles about A/B testing, and after publishing them, subscribers started reaching out with a bunch of follow-up questions. I responded to a handful of them until I realized I had already talked about this stuff in older posts!

So rather than keep sending people five different links in separate replies, I wanted to put everything in one place. This post is basically a reference list for my thoughts on A/B testing, sample sizing, multivariate testing, chi-squared testing, and the ways experiment results can mislead teams when they are read too narrowly. The shared theme across these articles is that A/B testing is useful when it is framed well, sized for the real world, and the data isn’t weaponized or misinterpreted. Enjoy!

Top 5 A/B Testing Articles

1. A/B Test Strategies That Work
   This article is about making A/B tests more useful before the test ever goes live. It focuses on the strategy behind the comparison, including how to define a better hypothesis, choose a meaningful outcome, and avoid running tests that look rigorous but do not actually help the team make a better product decision.
   

   

2. A/B Test Sample Sizing
   This post is a companion piece to the A/B Test Strategies That Work article above. It explains why sample size matters, what inputs shape it, and how concepts like baseline rate, minimum detectable effect, confidence, and power connect back to the decision the team is trying to make.
   

   

3. Another Way A/B Tests Mislead
   This article looks at how A/B tests can still mislead teams even when the test itself seems reasonable. It is mostly about interpretation, especially the risk of treating a winning variation as automatically better without asking what changed, what the metric actually captured, and what the result does not explain.
   

   

4. How to Conduct a Multivariate Test Using Excel or Google Sheets
   This post is a practical walkthrough for comparing multiple variables instead of forcing every product question into a simple A/B structure. It shows how a multivariate test can help separate the effects of different design elements, and it keeps the method approachable by using tools many teams already have access to.
   

   

5. More Meaningful Comparisons With Chi-squared Testing
   This article is about using chi-squared testing to make better comparisons when the data is categorical. That matters for A/B testing because a lot of product and UX outcomes show up as counts, choices, completions, failures, clicks, selections, or other patterns that are usually compared too casually with percentages alone.

Conclusion

I think these articles work well as a set because they cover different parts of the same broader problems I see in my day-to-day work. Knowing this stuff better is where UX researchers can really add a lot of value.

If your team uses A/B testing, is trying to use it more often, or is trying to get better at interpreting experiment results, these 5 posts are a good place to start. They are not meant to make experimentation feel more complicated for the sake of it. They are meant to make the method more useful in the real-world conditions where product decisions actually happen.

Thanks for reading, and I hope having these articles in one place makes them easier to use. If any of them help you have a better conversation about A/B testing, sample sizing, or statistical comparisons in your org, I would love to hear about it.

When I talk to other UX researchers about the idea that research should operate more like a service, I often get the same kind of pushback. People hear “service” and assume I mean something passive. They think I am talking about researchers waiting for requests, taking orders from other disciplines, and producing whatever study someone asks for. That is not what I mean.

When I say UX research should think of itself as a service, I mean service in the professional sense. A good service has judgment, standards, expertise, and responsibility for helping the people around it get better outcomes. It does not simply fulfill requests. It helps clarify what is actually needed.

This distinction matters because UX research can drift into some unhelpful patterns that I know you are all familiar with. One version is too removed from the work and only appears when a formal study is needed. Another version is too reactive and mostly responds to the next incoming request. Neither version is ideal for building digital products in the real world, where uncertainty is constant and decisions are being made long before a formal research plan exists.

The service model I am talking about is much closer to servant leadership than order-taking. The researcher is there to help the team do better work. That includes helping other UXers think through tradeoffs, helping Product folks understand uncertainty, and helping the broader team avoid building too much confidence around things they have not actually learned yet.

Service Makes Research More Strategic

The main argument is that service does not make research less strategic. It makes research more strategic because it puts research closer to the decisions that shape the product.

Over the last 20+ years, I’ve noticed that the researchers who have the most influence are often the ones who are useful throughout the work. They are close enough to see how decisions are forming, where assumptions are hardening, and where evidence would actually change direction. That kind of influence rarely comes from waiting until someone asks for a formal study. It comes from being involved early enough to shape the questions the team is asking in the first place.

And once you demonstrate this, your influence is likely to grow organically because, by serving others up real-world results, you place yourself front and center in the minds of your colleagues as someone who can help them succeed with your research knowledge.

This is where I think researchers sometimes get trapped by the language of strategy. We can start to believe that strategic research has to be big, formal, highly visible, or removed from the day-to-day mess of product work. In my experience, the opposite is often true. Research becomes more strategic when it is close to the real decisions, close to the people making those decisions, and close to the moments where uncertainty could send the work in the wrong direction.

Service Starts During Shaping

The best place for research to become useful is usually earlier than people think. Before there is a prototype, before the design direction has hardened, and sometimes before the team has fully agreed on the problem, there is already research work to do.

Project shaping is where Product, UX strategy, UX Design, and UX research should be working together. Product may be thinking about value, priority, constraints, and business direction. UX strategy and design may be thinking about experience direction, workflow implications, information architecture, and how the work fits into the broader product ecosystem. Research can add another layer by making user uncertainty visible before it gets buried inside requirements.

During project shaping, researchers can help the team answer practical questions:

• Which problem are we actually trying to solve?

• Who is affected by it?

• Which decisions have already been made?

• Where does the team still have room to change direction?

• What evidence do we already have from our own past research, foundational secondary research, usage analytics, support tickets, sales feedback, or previous usability issues?

• Which assumptions are being carried forward mainly because the project needs momentum?

This work is easy to undervalue because it does not always look like a formal research method. But, in my experience it can be one of the most important research contributions in the project.

Pairing Closely With Other UXers

Once a project moves from shaping into design, the UX researcher’s role should change in its nature. More close pairing with UX design and strategy professionals can be especially useful at this stage because they are usually among the first people who have to make the project real.

This is often where bad assumptions start showing up inside the actual experience. Many of those assumptions are reasonable, but reasonable assumptions can still be wrong.

You can see this translation happen in small ways:

• A vague idea becomes a workflow.

• A strategic goal turns into a screen.

• A loose requirement becomes decisions about hierarchy, sequence, content, layout, and interaction behavior.

A researcher paired closely with other UXers can help make those assumptions easier to see. The value comes from understanding the design direction, the tradeoffs being considered, and the moments where user evidence would actually change the work.

The researcher’s contribution might include:

• Running a formal study when the risk is high enough to justify one.

• Running a lightweight study that will still help the team make a better decision without changing the current pace of work.

• Reviewing prior findings when the team already has relevant evidence.

• Holding a quick working session to pressure-test a workflow.

• Reframing a preference debate around usability, user behavior, or product risk.

• Recommending that the team move forward because the existing evidence is already strong enough.

This is an example of the service mindset in practice. The researcher is not trying to own the UX decision. The researcher is helping other UXers make the decision with better information and fewer hidden assumptions.

See Uncertainty Earlier

Research can also serve the project by helping Product see uncertainty more clearly. A lot of software planning treats uncertainty as if it is mostly a scope or timeline issue. Those things matter, but user uncertainty can be just as disruptive.

User uncertainty shows up in practical ways:

• Unclear primary users can distort scope.

• Competing workflows can create sequencing problems.

• Weak concept understanding can affect both design and development.

• Low confidence in whether the new experience improves the old one can affect launch decisions.

A researcher can help sort these questions into practical categories. Some need to be answered before design moves too far. Others can be explored while UX is working through options. For example, prototype testing may be the right place for interaction-level concerns, while post-launch measurement may be better for adoption or behavior questions. There will also be questions that are interesting, but not important enough to slow the project.

For a product manager, this can mean clearer decision-making around priority and release confidence. For a project manager, it can mean a better understanding of which unknowns may affect sequencing, dependencies, or rework. For the broader team, it means uncertainty becomes something visible and manageable instead of something everyone quietly works around.

And if you do this enough times in a row, you will be seen as a go-to person who needs to be in the room to ensure quality work goes out the door.

I hope you are starting to see how this service model works in a show-not-tell way? Once you see your stakeholders reaching out to you to contribute, you will understand the true power you hold when it is their idea to bring you to the table.

Prototype, Testing, and Build

As the work moves into formal prototyping, the user researcher can help make sure the prototype can answer the team’s real questions. For example, A prototype built for stakeholder review is not always the same as a prototype built for user testing. For research, the prototype needs enough realism around the task, the context, and the important decision points. It does not need to be perfect. It needs to support the learning the team actually needs.

Close pairing with UX designers matters here as well because the UX researcher can help define the tasks, identify the moments that need attention, and keep the study focused on decisions instead of general reactions. (You know, the data-based decisions every org raves about but few actually use. hahahaha.) That focus becomes much easier when the researcher has been involved since project shaping and understands what the team is trying to learn.

When evaluative research happens, the study should be stronger because the researcher already understands the project. They know the constraints, (technical and strategic) the reason certain decisions were made, the parts of the design that can still change, and the findings that would affect the next decision.

That context changes the output. Instead of producing a generic list of findings, the researcher can connect findings to the project’s actual choices:

• A primary workflow issue may need to be fixed before release.

• A language issue may need a content change rather than a structural redesign.

• A mismatch between the team’s model and the user’s model may require rethinking the flow.

• A real usability issue may still be small enough to monitor after launch instead of delaying the release.

After the study, the researcher should stay close enough to help the team use what was learned. This does not mean the researcher becomes the designer, strategist, product manager, or project manager. It means the researcher stays in the work long enough for the findings to influence the decisions they were meant to inform.

Post-Launch

And of course, your service should not end when the design is handed off or when the readout is delivered. After launch, the team still needs to learn whether the work actually helped.

Post-launch learning can focus on practical questions:

• Are users completing the workflow more easily?

• Have support tickets changed?

• Are people even using the feature? Hahahaha

• Did a problem move somewhere else?

• Was the original project goal actually addressed?

Post-launch learning closes the loop and gives the next project a stronger starting point. Research becomes part of how the organization remembers what happened, what worked, what did not, and what should be watched next time.

Conclusion

My broader appeal here is quite simple. I’m convinced that UX research can be a service that helps software teams move through uncertainty with more honesty and less waste. That user research as a service starts in project shaping, pairs closely with UX design and strategy, supports Product, informs testing, stays connected through build, and helps the team learn after launch.

This is not the version of an all-too-typical UX research department where researchers either become order takers, stay too far removed from the work until a formal study is requested, or have little to no autonomy. It is a more embedded version of UX research where the goal is to help the team through your own expertise, judgment, and understanding of user behavior.

That is why I keep coming back to service as the frame. Insights and influence matter, but they are incomplete without usefulness. If UX research wants to matter more inside software orgs, it needs to be close enough to serve the work as it is taking shape. That might be the most strategic version of research we can possibly have. Thanks for reading my rant!

I was recently asked whether we have any data on how best to design an AI chatbot’s history by a team that was adding one to its website. Aren’t we all doing this these days? Hahahaha. This week’s article goes through what I learned while diving into the topic and is formatted somewhat like the documentation I originally wrote for the team that asked.

Of course, we’ll cover the basics, like how AI chatbots commonly generate looooooong responses and use multiple data sources, both of which pose a bunch of challenges. We’ll also look at how, over time, the standard back-and-forth chat format has proven to be confusing for users. But then we’ll talk about how these facts help us understand the best way to design the interface, the navigation system, the model’s context, the activity log, and the place where finished work is stored in a strategically designed AI chatbot.

The Recommended History Model

My default recommendation is to design AI chatbot history as three connected layers. Each layer supports a different part of the user’s work and prevents the standard chat transcript from carrying the full weight of the experience.

• Layer 1: Task history - This is the highest-level history view. It contains separate conversations, tasks, or background jobs that users can reopen independently.

• Layer 2: Interaction history - This is the sequence of instructions, clarifications, system actions, and responses within a task. Older interactions should become easier to scan as the history grows.

• Layer 3: Output workspace - This is where substantial results live. Reports, drafts, tables, analyses, and other detailed outputs should remain accessible without forcing users to find them inside the conversation timeline.

These layers may appear together in a single interface, but they should not be treated as the same thing. A list of tasks helps users decide where to go, the interaction history explains what happened, and the output workspace contains the work they actually need.

Layer 1: Task History

Define Task Boundaries

Before deciding how chat history should look, the team needs to define what counts as a separate task. This is the most important design decision because it determines when the product should reuse existing context and when it should begin with a clean context.

In the context of a chatbot, a task is a self-contained request that a user asks the system to complete. It has a specific goal and a clear point at which it can be considered finished. The user should also be able to return to the task later and review its result without needing to revisit unrelated conversations.

An example of this could be if a user asks an account-support chatbot to reset their account-access privileges, or something like that. Then, imagine after resolving that issue, the user asks to update their personal info in that account. Both requests relate to the same account, but they involve different goals, system actions, and completion states.

Use the following questions when deciding whether a new instruction belongs in the current task:

• Is the user still working toward the same outcome?

• Does the new instruction modify or refine the existing output?

• Will the same source material and context remain relevant within this task?

• Would the user reasonably want to reopen both parts of the work together later?

• Does the current task need to remain active for the new request to make sense?

A new task is usually appropriate when the user changes the desired outcome, introduces substantially different source material, or starts work that will be reviewed separately. A new task may also be safer when earlier context could distort the system’s response.

Use Task-Separated Threads

Once you have defined what counts as a separate task, you need to decide how those tasks will appear within the chat experience.

This structure turns the task boundaries you defined earlier into something users can see and navigate. It gives each task its own context, outputs, and completion state while preventing unrelated work from accumulating inside one enormous conversation.

Users often remain inside an existing chat because it is already open. Starting a new thread requires an additional decision, and the consequences of staying in the current thread are usually invisible to the end user. The chat may appear to work normally even as irrelevant context continues to accumulate. We know that long conversations increase the amount of context the system must process. Depending on the base LLM’s architecture, this will almost certainly increase response time and cost. It can also make it harder for the system to determine which earlier information still matters.

Long sessions = more context = more tokens = more time, more cost, degraded results

Task separation gives the system a cleaner working context. It also gives users a more useful history because each item represents a recognizable piece of work instead of an arbitrary portion of one enormous transcript.

Recommended Baseline

Begin with the following structure:

• Create one top-level thread for each distinct user goal.

• Keep refinements and follow-up questions inside the original task thread.

• Start a separate thread when the user begins an independent task.

• Preserve a clear relationship between related tasks when users may need to move between them.

• Allow users to rename task threads using language that reflects their work.

The interface may still use the familiar language of “chats” or “conversations.” Behind that label, the product should treat each history item as a task container with its own context, outputs, and status.

The chatbot should not rely entirely on users to understand context management. Research has proven that many users will continue typing into whichever conversation is open, even after the goal has changed. This is when the system itself can suggest starting a new task when it detects a substantial shift. That suggestion should explain the practical benefit in user-centered language.

For example:

This looks like a separate account task. Start a new task so the information and progress for each request stay separate?

And if possible, the chatbot should let the user decline that suggestion. Automatically splitting a conversation without explanation may cause users to lose track of where their work went.

Isolate Things with Background Instances

Users tend to think about AI chatting differently from a normal back-and-forth conversation. When they ask the system to search multiple sources, run an analysis, use several tools, or produce a detailed report, they often think of it as a task they have started and can return to later. The design should support this mental model by separating substantial background work from the main conversation and giving each task its own clear context, status, and result.

For these tasks, consider creating a dedicated task instance. Depending on the work involved, this may be a separate chat session or a background agent that can complete the task independently.

This approach helps keep unrelated instructions from entering the same working context. It also creates a natural task-history model because users start work, leave it running, and return to review the result.

When a Background Instance Is Appropriate

A dedicated background instance is most useful when:

• The task can run independently after the user provides the initial instructions.

• The task uses a defined set of files, account data, or external sources.

• The output has its own completion state.

• The user may want to review or revise the result later.

• Multiple tasks may run at the same time.

• The system needs to preserve a record of the tools or sources used.

The useful distinction is whether the system is continuing a conversation or beginning a separately reviewable piece of work. When a task will have its own status and output, a dedicated instance often makes sense.

Design the Task List Around Review

A background-task history should help users understand what requires attention. A chronological list alone may not be enough because recently completed work can be more important than a task that is still running.

Useful task states may include:

• Needs clarification

• In progress

• Ready for review

• Completed

• Failed or partially completed

• Archived

These states should be visible without opening every task. Users also need a clear path back into the work when the system requires clarification or when the result is ready to review.

Layer 2: Interaction History

Transcript is Not History

The typical chat layout presents each user message followed by a system response. This pattern is familiar and easy to build, but it becomes difficult to use when responses are long or the conversation contains repeated revisions.

Imagine a user who asks an AI chatbot to analyze several documents and draft a report. The user then asks for a different structure, requests a revised table, and changes the intended audience. The latest report may be useful, but it now sits underneath several earlier versions and a large amount of explanatory text. The problem with this is that although the transcript accurately records what happened, it does not help users find or interact with the current result.

Separate Events from Results

Within a task, the history may contain several different types of info:

• User instructions

• Clarifying questions

• Tool activity

• Intermediate findings

• Errors and recovery actions

• Completed outputs

These items should not necessarily receive equal visual weight. The user’s original instruction and the final output may be central to the work, while a routine tool action may only matter when something goes wrong.

The design should preserve enough history to support trust and recovery. It should also reduce the amount of low-value detail that users must scan during normal use.

Surface Meaningful Milestones

This is the last frame of reference you’ll need to consider when matching your users’ default mental models for chatbots. Consider turning important moments in the interaction into recognizable milestones. A milestone could represent the initial request, a major change in direction, a user-approved decision, or a completed output.

Milestones make the history easier to navigate because they reflect the structure of the work. They also help users understand why two outputs differ when the conversation contains several rounds of revision.

Collapse and Expand

Collapsing older question-and-answer pairs can make long histories easier to scan. Instead of displaying every previous exchange at full length, the interface shows each earlier interaction as a compact, consistently sized block.

This pattern is especially useful when individual system responses are long. It reduces the amount of scrolling required to move between earlier interactions and makes the overall structure of the task more visible.

What a Collapsed Item Should Show

A collapsed interaction needs enough information to help the user decide whether opening it will be useful.

Depending on the product, that may include:

• A concise version of the user’s instruction

• A short description of the resulting action or output

• The interaction time

• The output type

• A status or warning indicator

• A clear expand control

Avoid generating vague labels such as “Follow-up question” or “Updated response.” Those labels describe the mechanics of the conversation without helping the user recognize the content.

A useful collapsed item might read:

Compare current account activity with the previous quarter
or
Completed analysis with 4 flagged changes

That description gives the user enough context to recognize the work without displaying the full response.

Preserve Important Context While Collapsing

Collapsing content introduces its own usability risks. Users may miss information when the system hides too much, and repeatedly opening several sections increases interaction cost.

To account for this, keep the active interaction expanded by default. Important warnings, unresolved questions, and actions that require the user’s attention should also remain visible.

Visible and Verifiable

Users need to understand what information influenced an AI-generated response. This becomes especially important when the chatbot uses account data, uploaded documents, previous instructions, or external market sources.

A response can sound confident even when it relies on outdated, incomplete, or irrelevant context. Showing the underlying sources gives users a better chance of noticing these problems before they act on the output.

Context transparency should answer practical questions:

• Which files or data sources did the system use?

• Did the answer use current account data?

• Which user-provided goals or constraints influenced the result?

• Did the system use information from an earlier task?

• Can the user inspect the specific source supporting an important claim?

The interface should make these answers available without requiring the user to understand the system architecture.

Add Useful Context Automatically

A chatbot can improve an instruction by retrieving relevant context before generating the response. For example, an account-support chatbot may add the user’s current plan, permissions, and recent activity to a request when those details are necessary.

Automatic retrieval reduces the amount of information users must repeatedly provide. It also introduces a responsibility to show what the system added, especially when that context changes the answer.

The interface may show a small set of context indicators near the instruction or response. A more detailed context view can be available when the number of sources becomes too large for a few compact indicators.

Make Citations Part of the Interaction

For source-based responses, citations should appear near the claims they support. Users should be able to open a citation and reach the relevant part of the source instead of landing at the beginning of a long document.

Streaming citations can help users see where information is coming from as the response develops. They also make retrieval visible, which may help people understand the value and boundaries of the chatbot.

Citations still require careful design and validation. A citation creates the appearance of evidence, but it is only useful when the source exists, supports the claim, and is presented with enough context for the user to verify it.

Layer 3: Output Workspace

Chatbot outputs should not live only inside chat messages. Reports, editable drafts, tables, plans, and data-heavy analyses need a persistent place where users can review and act on them.

This layout lets users keep the instruction and relevant process visible while working with the result. It also prevents the output from disappearing inside the timeline as the conversation continues.

What Belongs in the Output Workspace

Use a dedicated output workspace when the result:

• Is long enough that reading it inside a message bubble would be difficult.

• Needs editing, filtering, sorting, or another form of interaction.

• Has multiple versions that users may compare.

• Will be exported, shared, or used outside the chatbot.

• Remains useful after the conversation that produced it has ended.

• Represents the main value of the task.

Keep the Latest Output Easy to Find

The output workspace should make the current version obvious. Earlier versions can remain available when they support review, comparison, or recovery.

A useful pattern is to keep the current artifact open in the output panel while the interaction timeline records the instructions that changed it. This allows the conversation to support the work without becoming the work itself.

Evaluate the History Design

As UX researchers, we should always evaluate the designs we create. Chat history, as a task-management and retrieval system, is no exception. A standard chatbot usability study focused only on prompting and response quality will miss many of the problems described in this article. I’ve seen this mistake made dozens of times already.

Study Design

Give participants histories that already contain several tasks, long responses, revised outputs, and mixed source types. Then observe whether they can understand the structure without receiving an explanation first.

Ask participants to complete these tasks:

• Return to a task they started several days earlier.

• Locate the final output from a conversation with several revisions.

• Determine which sources informed a specific response.

• Start an unrelated task without contaminating the current one.

• Find a task that is waiting for their review.

• Compare the current output with an earlier version.

• Explain what information the system is currently using as context.

These tasks reveal whether the history supports real use over time. They also uncover whether the interface matches the user’s mental model of tasks, conversations, and outputs.

Useful Measures and Observations

During evaluative research, consider tracking:

• Time required to locate a previous output

• First-click accuracy when reopening work

• Frequency of users continuing unrelated work in an existing task

• Ability to identify the active context correctly

• Missed items requiring review or clarification

• Unnecessary creation of new task threads

• User confidence in the sources supporting an answer

Qualitative follow-up is especially important. A participant may complete the task successfully while still feeling unsure about what the system remembered or why a particular output appeared.

Chat-History Design Checklist

Use this checklist during design reviews, requirements gathering, or concept evaluation.

• Define what counts as a discrete task for the product.

• Identify when the system should recommend starting a new task.

• Decide which tasks require dedicated background instances.

• Keep each task’s context and outputs separate where possible.

• Design the top-level history around recognizable user goals.

• Make long interaction histories scannable.

• Keep unresolved questions and important warnings visible.

• Show which data, files, and constraints informed each important response.

• Ensure citations lead users to relevant supporting content.

• Give substantial outputs a persistent workspace outside the transcript.

• Make the current version of an output obvious.

• Test the design using populated histories and returning-user scenarios.

Conclusion

Well, there you have it. Designing an AI chatbot’s history properly requires more than saving old messages and displaying them in chronological order.

The main challenge is deciding how the product should represent work. Teams need to define where one task ends, when a new context should begin, how earlier interactions become scannable, and where substantial outputs should live.

To design a good chatbot, I would begin with task-separated histories and dedicated background instances for independently reviewable work. I would then use collapse-and-expand patterns to manage long interaction histories, make important context verifiable, and move detailed outputs into a persistent workspace.

The traditional chat transcript can remain part of the experience. It just should not be responsible for the entire experience. I hope some of you find these documentation posts helpful!

I’ve written a lot about how to roll out Pendo, or platforms like Pendo, across an entire tech org with many different lines of business. But I’ve spent less time writing about what happens after the rollout, when teams start using those tools to communicate with users and collect feedback inside the product experience itself.

Pendo can support all kinds of in-app pop-ups, from onboarding messages and product announcements to feature prompts, guides, and feedback requests. Today, though, I want to focus on one specific use case that can be especially valuable when it is handled carefully. That use case is short, task-based in-app feedback. I see these wayyyyy overused and grossly misunderstood. So, lucky for you, I recently wrote this guide on best practices for single-question, 1–5 rating pop-up surveys, so I thought I should share a version.

In‑App Task Survey Recommendations

This document provides guidance for deploying single question, in‑app surveys within digital products. It is intended for technology, product, UX, and engineering teams who want to collect timely, contextual, actionable user feedback while minimizing disruption and survey fatigue.

The guidance focuses on:

• When task‑level in‑app surveys should be used

• How and when surveys should be triggered

• Recommended sample sizes and stop rules

• Exposure and frequency limits

• Prioritization when multiple surveys are active

• Best practices for comparing results over time

1. When to Use In‑App Task Surveys

Short, single‑question in‑app pop‑up surveys (such as CSAT, ASAT, effort or ease‑style questions) are best used as task‑level measurement tools, not as broad indicators of overall product satisfaction.

Strong Use Cases

2. Triggering In‑App Surveys

In‑app surveys are often triggered only after successful completion of a workflow. While useful, completion‑only triggers provide an incomplete picture of the experience.

A completion-only trigger tells us how easy the workflow felt to users who made it all the way to the end. It does not fully capture users who:

• Started but abandoned the workflow

• Hit errors

• Could not find the next step

• Needed help

• Took an unusually long time

• Left and returned later

• Gave up before reaching the completion event

For this reason, surveys should target meaningful workflow attempts, not just completions.

Recommended Trigger Strategy

3. Sample Size and Stop Rules

In-app surveys should be removed once the target sample size has been reached or once additional responses are unlikely to change the decision the team needs to make.

Recommended Sample Size Guidelines

• Baseline target: ~55 completed responses per audience segment

• Hard cutoff: 100+ responses per segment (This is usually more than most teams need unless they are doing formal benchmarking, deep segment comparisons, or trying to detect small changes over time.)

Practical Sample Size Tiers

4. Frequency and Exposure Limits

In‑app surveys should be rare enough that they do not feel like part of normal product usage.

Recommended Exposure Limits

5. Survey Instance Prioritization

When multiple survey instances are active, users should not become eligible for all of them at once. If a user qualifies for more than one instance in the same session, only show the highest-priority survey.

If multiple instances qualify, show only the highest‑priority survey.

Recommended Priority Order

6. Comparing Results Over Time

Comparisons are only valid when the underlying measurement conditions are similar.

Before comparing results, confirm:

• Same workflow: Do not compare scores across different workflows

• Same user types: Do not mix audience type, user segments, or different persona’s scores

• Similar trigger logic: Do not compare completion-only data with broader attempt-based data

• Similar workflow version: Changes to the UI or flow may reset the benchmark

• Sample size ≥ ~30: Small samples can make normal variation look meaningful

Conclusion

Well, there you have it. In-app surveys can be useful, but only when they are treated like a research tool instead of a messaging feature someone happens to turn on. The fact that a platform makes it easy to launch a pop-up does not mean every workflow needs one, every user should see one, or every score should be treated like a clean benchmark.

The real world needs guardrails. Teams need to decide what is worth measuring, when the user has meaningfully attempted the task, how many responses are enough, how often the same user should be exposed, and when the survey should come back down. None of that is especially glamorous, but it is the difference between collecting feedback people can actually use and slowly teaching users to dismiss everything that appears in the product.

My general rule is pretty simple. Ask fewer questions, ask them at better moments, and stop asking once the data is good enough to support the decision in front of you.

That might not give you a perfect measurement system, but in most real-world product environments, it will give you a much better one. Thanks all!

A few weeks ago, I wrote about how to make A/B tests more useful from a UX research perspective. That article was mostly about an overarching A/B test strategy. I focused on things like writing a clear hypothesis, designing a meaningful variation, choosing better metrics, and not overstating what the result proves.

This week’s article is about the next practical question teams usually ask:

What sample size do we need?

That question sounds simple, but it can get confusing quickly. Some people expect there to be one standard A/B test sample size. Others start with the number of users they think they can get and work backward from there. “We always A/B with 200.” Something like that sound familiar? Others use whatever default number their experimentation platform gives them without understanding what went into the calculation.

I do not think UX researchers need to become statisticians to have this conversation. But I do think we need to understand the basic inputs well enough to explain what the number means. This basic calculation needs contextual inputs that only you, as the UX researcher, and your stakeholders can provide because of your knowledge of your org. That includes the design improvements that would matter most to the business, the quality of the hypothesis, the strength of the variation, and the behavior being measured.

The goal of this article is to show the basic inputs that go into an A/B test sample-size calculation, explain what they mean in plain language, and walk through a real example.

The Basic Inputs You Need

Before you calculate anything, you need to know what you are measuring and what kind of change would matter. To calculate sample size, you need a few inputs.

These inputs matter because the calculation is only as useful as the numbers you put into it. If the inputs are unrealistic, the sample-size recommendation will be unrealistic too. This is where teams often get into trouble. They want the equation to poop out the perfect number, even though they have not made any decisions based on their own real-world context. Lazy, lazy, lazy. But in the real world, the sample size should come from the decision the team is trying to make.

Start With Current Rate

The two most important inputs are the baseline rate and the smallest improvement the team cares about detecting.

1. Baseline rate = What is happening today. For example, if 18 out of 100 users complete the action, the baseline is 18%. If 5 out of 100 users click the call to action, the baseline is 5%. You get the idea.

   The baseline tells the calculation what normally happens before you change anything.

   Sometimes you will have a clean baseline from usage analytics. That is ideal. Other times, you may have to make a reasonable estimate. That is not perfect, but it is better than not taking baseline into account at all. (Unfortunately, I see this misunderstood and dismissed most of the time in the real world.)

   After that, you need to decide on the smallest improvement worth detecting. This is called the Minimum Detectable Effect, or MDE.

2. MDE = The smallest performance improvement the team would actually care about. For example, let’s say 18% of users complete the task today. The team probably would not want to redesign the workflow, run an A/B test, and change the product just to move that number from 18% to 19%. That improvement may be real, but it may not be big enough to justify the work.

   So the team has to ask a practical question:
   “How much better would the new version need to perform for this change to be worth making?”

   Maybe 20% is not enough because that is only a small improvement over the current 18%. Maybe 25% feels close, but the team wants the test to be planned around a slightly larger, cleaner target.

In this example, the team decides that moving from 18% to about 26% would be a large enough improvement to justify the change.

3 main points here:

• That 26% is strategic and comes from adding the improvement the team cares about to the current baseline:

• 18% current completion rate + 8 percentage point improvement = 26% target completion rate

• In sample-size terms, that means the team is planning the test around an 8 percentage point improvement.

This is why I usually want stakeholders involved in the MDE conversation. The researcher can explain the tradeoffs, but the product team needs to be honest about what size improvement would actually change the decision.

And as a last resort, if all this MDE talk is still confusing, try asking an LLM to help you estimate a reasonable starting point. Do not ask it to “calculate the perfect MDE.” Ask it to help you think through what size improvement would be meaningful based on the product context, the type of change, the current baseline, and the decision the team is trying to make.

Here is a prompt template you could use:

I would still treat the LLM’s answer as a starting point and not the final decision. The team should review the recommendation and decide whether the improvement is actually meaningful in the real-world context of the product. This is one of the safer and more targeted ways Gen AI can be useful in our work, and the human in the loop is built into the approach. Try it for yourself and tell me how it goes!

Small Improvements Need Larger Samples

The smaller the improvement you want to detect, the more users you need.

A small performance improvement can get lost in normal variation. (signal vs. noise.) A larger improvement is easier to detect because the difference between Version A and Version B is bigger.

Here is the basic tradeoff:

If the team does not have enough traffic to detect a small improvement, they have a few options. They can run the test longer, accept a larger MDE, choose a different method, or admit that the A/B test is not practical.

For a tiny button copy change, expecting an 8 percentage point improvement may be unrealistic. For a larger workflow change, an 8 percentage point improvement may be a reasonable threshold.

The simple version is this: if the team wants to detect small behavioral differences in a design change, they need more data. If the team only cares about larger behavioral differences in a design change, the test can usually have a smaller sample size.

Test Settings

After the baseline and MDE, you need to choose a few test settings.

These are usually confidence level, power, and split.

• Confidence level is about how much risk you are willing to accept that the result is just noise. A 95% confidence level is more cautious and usually requires more users. A 90% confidence level accepts a little more uncertainty and usually requires fewer users. I usually consider 90% confidence for lower-risk, reversible product decisions. I would use 95% for higher-risk decisions, especially if the decision is expensive, hard to reverse, or related to high-risk user interactions.

• Power is the chance that the test will detect the improvement if the improvement is really there. I usually use 80% power because it is a common default.

• Split is how users are divided between the two versions. Most of the time, I assume a 50/50 split. Half of users see Version A. Half of users see Version B.

Here’s how I think of it:

My default planning assumptions are almost always:

Real-World Example

Scenario - A product team wants to A/B test a changed workflow. The current workflow starts from a dashboard card. The card includes a call to action that sends users into a multi-step flow. The team wants to know whether the revised experience gets more users to complete the intended action.

The team does not have a perfect baseline, but based on what we know, we estimate that about 18% of users currently complete the action.

Because this is a larger workflow change, not a tiny copy tweak, the team decides that an 8 percentage point improvement would be meaningful. In other words, the new version would need to move the completion rate from 18% to about 26%.

We decide to use 90% confidence because this is a lower-risk product decision and the team does not need 95% confidence to make the call.

We use 80% power because that is the standard default I usually use.

We use a 50/50 split because there is no strong reason to send more users to one version than the other.

Here are the calculation inputs:

In the real world, I just use this calculator:

https://samplesizecalculator.net/ab-test-sample-size-calculator/

For most UX-related A/B tests, the outcome is usually a yes/no behavior. The user completed the workflow or did not. The user clicked the call to action or did not. The user submitted the form or did not.

Assuming yes/no behavior, and with the inputs shown above, the sample size is about 331 users per version. So that means I would plan for about 331 users in Version A and 331 users in Version B.

And remember, because today, about 18% of users complete the task. We want to know if the new version can raise that to about 26%. That’s how we get to 331 users per version.

Base formula:

For this example, the calculation looks like this:

You do not need to memorize that formula. The important thing to understand is that the calculation uses the current behavior, the amount of change you want to detect, and the amount of uncertainty you are willing to accept.

The sample-size calculation does not mean the test is automatically good. It just means that, given these assumptions, this is roughly how many users the team needs to compare the two versions.

I would also not treat that number as perfect. If the baseline estimate changes, the sample size can change. If the team wants to detect a smaller improvement, the sample size will go up. If the team wants 95% confidence instead of 90%, the sample size will also go up.

Sample-Size Checklist

Use this checklist before you calculate the sample size.

• Define the behavior you are measuring. Be specific. “Users completed the four-step workflow” is clearer than “engagement.”

• Get the current baseline rate. Use analytics if you have them. If not, use the best estimate you have and say clearly that it is an estimate.

• Decide the smallest improvement that would matter. This is the Minimum Detectable Effect, or MDE. Decide whether the team cares about a 2 percentage point improvement, a 5 percentage point improvement, an 8 percentage point improvement, or something else.

• Choose the confidence level. I usually consider 90% confidence for lower-risk, reversible product decisions and 95% confidence for higher-risk decisions.

• Use 80% power unless there is a strong reason not to. This is the standard I typically use.

• Use a 50/50 split unless there is a reason to do something else. Half of users see Version A. Half of users see Version B.

• Calculate the sample size per version. Do not only report the total sample size. Teams need to know how many users are needed in each version.

• Round up. If the calculator says 330.31, do not plan for 330. Plan for 331 or more.

• Decide the stopping point before the test starts. Do not wait until the results look good and then stop the test. Set the target sample size ahead of time.

• Do not overstate what the result proves. The test can tell you whether one version performed differently from another version under the conditions of the test. It does not automatically explain why the difference happened. Read more about A/B Testing strategy here.

Conclusion

The big takeaway here is that A/B test sample sizing is mostly about making the team’s assumptions visible.

That is why UX researchers should lead these conversations within our orgs. We must help others to think in terms of sample size being connected to study design by default. It depends on the hypothesis, the variation, the outcome metric, and the decision the team is trying to make.

A correctly sized A/B test can still be interpreted badly. It can tell you whether one version performed differently from another, but it does not automatically explain why. So use the calculator. Get the number. Round up. Then make sure the test is designed clearly, sized realistically, and interpreted carefully. Thanks for reading!

Over the last 10+ years, I’ve written essentially the same series of documents for different orgs around interaction design and accessibility best practices for primary navigation bars.

I finally sat down and combined all those documents into this master best-practice guide. This is intended to be more of a reference guide, and I’m happy to share it because, if you’re anything like me, you hate wasting time going back and forth with your front-end devs on all the little nuances that go into doing something as foundational as a nav bar.

My hope is that many of you bookmark this article, just like I will, and send the link anytime you need to share guidance with others.

This guide details the following topics:

• Navigation bar interaction behavior

• Dropdowns, mega menus, and mobile menus

• How to think about hover

• How to handle parent-child relationships

• All the different UI component interaction states

• Keyboard and screen-reader navigation

• Proper semantic HTML and ARIA execution

The Purpose of This Guide

This document defines interaction design standards for the UI pattern primary navigation bar. Primary navigation bars must help users move between major feature areas, understand where they are, and predict what will happen when they activate an item. The interaction model should remain consistent across desktop and mobile, even when the layout changes across breakpoints.

2. Core Principles

2.1 Use clear information scent

Items must help users predict what they will find or what will happen after activating them.

A primary navigation item should clearly communicate one of the following:

• A destination

• A major feature area

• A user task

• A meaningful content category

• A parent category that exposes related child destinations

Item labels should match the language users expect within the web application. They should not rely on internal team names, implementation structure, roadmap language, or terminology that only makes sense to the organization.

A user should be able to scan the primary navigation and make a confident choice without needing to pause and interpret what the item might contain.

2.2 Support recognition over recall

Primary navigation should not require users to remember where something lives from prior experience.

Users should be able to recognize available paths by scanning:

• Visible labels

• Group headings

• Dropdown indicators

• Parent-child relationships

• Current page indicators

• Visual grouping inside dropdown menus or mega menus

Do not rely on hidden meaning, icon familiarity, or prior training. If an item opens a child menu, the UI should make that behavior visible. If an item sends the user directly to a page, the UI should make that behavior feel different from a Menu item.

2.3 Maintain consistent interaction behavior

Items with the same function must behave the same way throughout the application.

Primary navigation items should follow one of two interaction models:

1. Destination behavior: The item takes the user directly to a page or route.

2. Disclosure behavior: The item opens or closes a child menu.

✔️ Requirements:

• Destination items use link behavior.

• Menu items use button behavior.

• Menu items include a dropdown indicator.

• Destination items do not include a dropdown indicator.

• Parent items that expose child menus should not also act as direct destination links.

• Parent landing pages should be represented as child overview links when needed.

• Menus must open only through intentional activation.

• Menus must close only through intentional dismissal behavior.

• Hover must not open, close, reveal, or dismiss a menu in this navigation pattern.

2.4 Keep the hierarchy as flat as practical

Primary navigation should reduce unnecessary hierarchy.

Each additional level creates more interaction cost and more abstraction for the user to understand. A flatter structure makes it easier for users to scan available destinations and make a confident choice.

A good primary navigation structure should:

• Make major feature areas visible.

• Group related destinations under meaningful parent items.

• Expose terminal nodes inside dropdown menus or mega menus.

• Avoid forcing users through multiple nested levels before they can reach a destination.

3. Primary Navigation Anatomy

3.1 Navigation landmark

Primary navigation must be represented as a semantic navigation landmark.

From an interaction design perspective, this matters because users of assistive technology need a predictable way to identify and move to the primary navigation region.

✔️ Recommended default:

<nav>
  <!-- Primary navigation items -->
</nav>

If the page includes only one navigation landmark, an accessible name is optional. If the page includes more than one navigation landmark, each landmark should have a clear accessible name.

Use distinct names when navigation landmarks serve different purposes, such as Primary, Utility, Breadcrumb, or Footer.

If the same navigation is repeated in more than one place with identical links, the repeated landmarks may use the same accessible name.

✔️ Preferred naming order:

1. Use a visible or visually hidden heading with aria-labelledby.

2. Use aria-label only when no visible or referenceable label is available.

Example when a referenceable heading exists:

<h2 id="primary-navigation-heading" class="visually-hidden">Primary</h2>

<nav aria-labelledby="primary-navigation-heading">
  <!-- Primary navigation items -->
</nav>

Example when no visible or referenceable label exists:

<nav aria-label="Primary">
  <!-- Primary navigation items -->
</nav>

Do not use aria-label=”Primary navigation” as the default. This is usually redundant rather than a severe accessibility failure, because screen readers already announce the landmark as navigation. The accessible name should usually identify which navigation it is, not repeat the word navigation.

3.2 Navigation list

Primary navigation should use a list structure when presenting a set of related items.

✔️ Recommended structure:

• <nav> for the navigation landmark

• <ul> and <li> for the item collection

• <a> for destination items

• <button type=”button”> for Menu items

The interaction design rule is straightforward:

• If the item moves the user to a destination, it is a link.

• If the item opens or closes a child menu, it is a button.

This distinction should be reflected in both the interaction behavior and the visual design.

3.3 Item types

Primary navigation items must be one of two types.

1. Destination item

A destination item takes the user directly to a page or route.

✔️ Interaction requirements:

• Activating the item navigates to the destination.

• The item does not open a child menu.

• The item does not include a dropdown indicator.

• The item may show current page state if it represents the currently displayed page or route.

• Hover may show an over state, but it must not trigger navigation.

✔️ Implementation requirement:

• Use an <a> element.

Example:

<a href="/path">Item label</a>

2. Menu item

A Menu item opens or closes a child menu.

✔️ Interaction requirements:

• Activating the item opens the child menu.

• Activating the same item again closes the child menu.

• Hovering over the item must not open the child menu in this pattern.

• Hovering out of the item must not close the child menu in this pattern.

• The item includes a dropdown indicator.

• The item has distinct expanded and collapsed states.

• The child menu appears in a predictable spatial relationship to the item.

• The child menu can be dismissed through expected dismissal behaviors.

✔️ Implementation requirements:

• Use a <button type=”button”> element.

• Use aria-expanded.

• Use aria-controls only when the control-to-menu relationship needs to be explicit. aria-expanded is the essential ARIA state for the Menu item.

Example:

<button type="button" aria-expanded="false">
  Item label
</button>

3.4 Stronger code example

The following example shows the preferred structure for a primary navigation landmark, one current destination item, and one Menu item with a child menu.

<h2 id="primary-navigation-heading" class="visually-hidden">Primary</h2>

<nav aria-labelledby="primary-navigation-heading">
  <ul class="primary-nav">
    <li>
      <a href="/dashboard" aria-current="page">Dashboard</a>
    </li>

    <li>
      <button
        type="button"
        aria-expanded="false"
        aria-controls="reports-menu"
      >
        Reports
        <span aria-hidden="true">▾</span>
      </button>

      <ul id="reports-menu" hidden>
        <li><a href="/reports/overview">Reports overview</a></li>
        <li><a href="/reports/activity">Activity reports</a></li>
        <li><a href="/reports/compliance">Compliance reports</a></li>
      </ul>
    </li>
  </ul>
</nav>

This example demonstrates the expected implementation model:

• The navigation region uses a semantic <nav> landmark.

• The landmark is named by a referenceable heading.

• The item collection uses a list structure.

• Destination items use native links.

• Menu items use native buttons.

• The disclosure button uses type=”button”.

• The disclosure button exposes its open or closed state with aria-expanded.

• This example includes aria-controls because the controlled submenu has a stable id. aria-expanded remains the essential ARIA state for the disclosure button. If aria-controls is used, it must reference the menu controlled by the disclosure button.

• The closed child menu uses hidden so it is removed from the visual interface and the accessibility tree.

• When toggling a menu, update hidden, keyboard focusability, and aria-expanded together so the visual state, keyboard state, and accessibility tree stay synchronized.

• If the menu is animated, the transition must not leave focusable content hidden from sighted users or visible content hidden from assistive technologies. Animation cannot break synchronization between visual state, focusability, hidden, and aria-expanded.

• The dropdown indicator is decorative and hidden from assistive technology with aria-hidden=”true”.

4. Information Architecture Standards

4.1 Parent-child relationships

A parent item must represent a meaningful category. Child items must belong logically under that parent.

The parent-child relationship should be clear from the user’s point of view, not just from the organization’s point of view.

Good parent-child relationships are based on:

• User tasks

• User goals

• Feature areas users recognize

• Common workflows

• Expected content groupings

4.2 Terminal nodes

A terminal node is an item that takes the user to a page or route.

In a dropdown menu or mega menu, the primary activatable child items should usually be terminal nodes. This lets users move from the menu to a destination without needing to understand additional hidden layers.

Terminal nodes may include:

• Landing pages

• Overview pages

• Feature areas

• Filtered results pages

• Task-specific pages

• Settings pages

• Reports or dashboard views

✔️ Preferred pattern:

Primary navigation item
  Child destination
  Child destination
  Child destination

❌ Avoid:

Primary navigation item
  Category
    Subcategory
      Sub-subcategory
        Destination

4.3 Parent landing pages

If a parent category has a landing page, do not use split-button behavior.

The parent item should not behave as both:

• A link to the parent landing page

• A Menu item that opens a child menu

That interaction model creates ambiguity because the same row appears to have two different functions.

For this navigation system, hybrid parent link plus disclosure behavior is not allowed. The restriction is a design-system rule intended to preserve one clear interaction purpose per row.

✔️ Required pattern:

• Make the parent item a Menu item.

• Add a child destination item labeled [Parent Category] Overview.

Example structure:

Parent Category
  Parent Category Overview
  Child Destination
  Child Destination

In this pattern:

• Activating the parent item expands or collapses the child section.

• Activating the overview item navigates to the parent landing page.

• The row has one interaction target.

• The dropdown indicator is not a separate interaction target.

• The user does not need to understand a split-button interaction.

This pattern should apply consistently across levels of the primary navigation hierarchy. Do not use the split-button UI control.

4.4 Duplicate destinations

A destination should generally appear only once in primary navigation.

Duplicating the same destination in multiple navigation areas can make users question whether the items lead to the same place or different places.

When a destination could reasonably belong in more than one place, choose the location that best matches the user’s mental model.

4.5 Child item grouping

Child items should be grouped by user expectations.

❌ Do not group items by:

• Internal team ownership

• Feature architecture

• System architecture

• Department names

• Release structure

• Roadmap structure

✔️ Groups should be:

• Descriptive

• Mutually distinct

• Medium-granularity

• Ordered intentionally

Group order may be based on:

• Task frequency

• Workflow sequence

• User importance

• Natural conceptual order

If none of those apply, place the most commonly used or most important group in the top-left area of the mega menu.

5. Navigation Labeling Standards

5.1 Label clarity

Item labels must clearly describe the destination, task, or category.

A strong label should answer the user’s basic question:

What will I find if I activate this?

A label is too vague when it could plausibly contain several unrelated destinations or when users would need to open it just to understand what it means.

5.2 Label consistency

Use the same term for the same concept across the application.

Do not alternate between similar labels unless each term has a distinct user-facing meaning.

If multiple related terms are needed, each term should represent a distinct concept and appear in the navigation only where that distinction is meaningful to users.

5.3 Label differentiation

Items in the same set should be easy to distinguish from one another.

Each label should create a distinct information scent trail.

5.4 Icon labels

Icons in primary navigation must include visible text labels.

Do not use icon-only primary navigation items. (buttons/links)

Icons may support recognition, but they should not carry the meaning on their own. This applies to desktop and mobile navigation.

For mobile primary navigation, the menu control should include a visible text label such as Menu. The menu icon may appear with the label, but the icon should not be the only visible cue.

6. Desktop Primary Navigation Behavior

6.1 Opening menus

Dropdown menus and mega menus must open only by intentional activation.

For this primary navigation pattern, hover must not open, close, reveal, or dismiss dropdown menus, mega menus, or child navigation items.

✔️ Required behavior:

• Activate a Menu item: open the child menu.

• Activate the same Menu item again: close the child menu.

• Activate another Menu item: close the current menu and open the activated menu.

• Activate a destination item: navigate to the destination.

• Hover over an item: show only the visual over state.

• Hover out of an item: remove only the visual over state.

This interaction model gives users control because menus open based on intentional activation, not cursor movement.

6.2 Hover behavior

Hover state may indicate that an item is interactive.

✔️ Hover may change:

• Background

• Text decoration

• Border

• Color

• Elevation

• Dropdown indicator styling

❌ In this pattern, hover must not:

• Open a menu

• Close a menu

• Reveal child items

• Dismiss child items

• Replace focused state

• Replace current page state

• Trigger behavior that changes the available navigation structure

Hover is visual feedback only. It should communicate that an item is interactive, but it must not change menu visibility.

6.3 Hover out behavior

Hover out removes the hover state.

Hover out must not close a dropdown menu or mega menu in this pattern.

Users must be able to dismiss an open menu through deliberate actions, not through cursor movement. This prevents fragile interaction paths where a menu closes unexpectedly because the cursor moved slightly outside the menu boundary.

✔️ Required rule:

• If a menu was opened by activation, it remains open until the user intentionally dismisses it.

6.4 Menu dismissal

Users must be able to close an open dropdown menu or mega menu by using deliberate dismissal methods.

✔️ Required dismissal methods:

• Activate the Menu item again.

• Activate outside the menu.

• Press the Esc key.

• Move focus out of the open navigation region, when this dismissal behavior is intentionally defined. Moving focus from the Menu item into its child menu must not dismiss the menu.

• Activate another Menu item.

• Activate a destination item.

When the Esc key closes the menu, focus must return to the Menu item that opened it.

This keeps the user oriented and prevents keyboard users from losing their place in the navigation.

6.5 Animation timing

Dropdown and mega menu animations should be brief.

✔️ Recommended duration:

• 100 to 150 milliseconds

Animation may help users understand that a menu has opened or closed, but it should not slow down navigation.

❌ Avoid animations that:

• Delay access to child items

• Make the interface feel less responsive

• Create unnecessary motion

• Interfere with focus visibility

• Make it harder to track the current state

• Ignore user motion preferences

• Desynchronize hidden, keyboard focusability, and aria-expanded during open or close transitions

6.6 Motion preferences and reduced motion

Navigation animation must not create unnecessary motion or make the interface harder to use.

If menu animation is used, implementation should respect user motion preferences, including prefers-reduced-motion.

✔️ Requirements:

• Animations must not delay access to child items.

• Animations must not interfere with visible focus indicators.

• Animations must not be required for users to understand whether a menu is open or closed.

• When users request reduced motion, non-essential menu animation should be reduced or removed.

• Open, closed, expanded, collapsed, focused, and current page states must remain clear even when animation is reduced or removed.

• When animation is used, visual visibility, keyboard focusability, hidden, and aria-expanded must remain synchronized throughout the open and close behavior.

Reduced motion should not change the interaction model. Menus still open by activation and close through defined dismissal behavior.

7. Dropdown Menu and Mega Menu Behavior

7.1 Dropdown indicators

Menus must include a dropdown indicator.

Acceptable indicators include:

• Down chevron

• Disclosure arrow

• Caret

The dropdown indicator communicates that activating the item will expose child navigation items.

Destination items must not include dropdown indicators.

This distinction helps users tell the difference between:

• Items that navigate directly

• Items that open child menus

Dropdown indicators should be visually consistent across the primary navigation system.

The dropdown indicator should not behave as a separate interaction target. Activating the label area or the indicator area should perform the same disclosure action.

7.2 Open state

When a dropdown menu or mega menu is open:

• The Menu item must show an open visual state.

• The Menu item must use aria-expanded=”true”.

• The child menu must be visible.

• The child menu must appear in a predictable spatial relationship to the Menu item.

• The dropdown indicator should visually reflect the expanded state.

• The menu should remain open until intentionally dismissed.

The open state should make it clear which top-level item controls the visible menu.

Opening a menu must be triggered by activation, not hover.

7.3 Closed state

When a dropdown menu or mega menu is closed:

• The Menu item must show a collapsed visual state.

• The Menu item must use aria-expanded=”false”.

• The child menu must be hidden visually.

• The child menu should be hidden from assistive technology unless there is an intentional reason not to do so.

• The dropdown indicator should visually reflect the collapsed state.

The closed state should communicate that child items are available, but not currently visible.

Closing a menu must be triggered by intentional dismissal behavior, not hover out.

7.4 Esc key behavior

When the user presses the Esc key while a menu is open:

• The menu closes.

• Focus returns to the Menu item that opened the menu.

• The Menu item updates to aria-expanded=”false”.

• The visual state updates from expanded to collapsed.

This behavior should be consistent for all dropdown menus and mega menus in the primary navigation.

7.5 Focus behavior

When a menu opens, focus behavior must be predictable and consistent.

The design team should choose one focus model and apply it consistently across primary navigation menus.

Acceptable patterns:

1. Focus remains on the Menu item: The menu opens, and the user can continue forward through the tab order into the child items.

2. Focus moves into the opened menu: Focus moves to the first focusable item inside the opened menu.

In either pattern:

• Open menu items must be reachable by keyboard.

• Users must be able to move focus out of the menu.

• Focus should not be trapped inside standard desktop dropdown menus or mega menus.

• Focus order must follow the visual and logical order of the menu.

• Esc must close the menu and return focus to the Menu item.

• Focus movement may dismiss the menu only when that dismissal behavior is intentionally defined. Moving focus from the Menu item into the opened child menu must not dismiss the menu.

Focus behavior should not depend on hover.

7.6 Mega menu layout

Mega menus should organize terminal nodes into visible groups.

Each group should include:

• Descriptive group heading

• Related destination items

• Clear visual separation from other groups

• Logical reading order

✔️ Preferred structure:

Primary navigation item
  Mega menu
    Group heading
      Destination item
      Destination item
      Destination item

❌ Avoid deep nested structures inside mega menus:

Primary navigation item
  Mega menu
    Category
      Subcategory
        Sub-subcategory
          Destination item

Mega menus work best when they expose meaningful destination choices in a scannable layout. They should not become a complete site map or a visual representation of the full feature architecture.

7.7 Current page inside a dropdown or mega menu

If the currently displayed page is represented by a child item inside a dropdown or mega menu, the child item should receive the current page indicator.

Example structure:

Parent Item
  Child Destination
  Child Destination ← current page
  Child Destination

In this pattern:

• The current child destination receives the current page indicator.

• The current child destination uses aria-current=”page”.

• The parent item may receive a visual parent-active treatment.

• The parent item should not use aria-current=”page” unless the parent item itself is the current page.

The parent-active treatment helps users understand which primary navigation section contains the current page. It should be visually distinct from the child item’s current page indicator.

8. Mobile Primary Navigation Behavior

8.1 Menu control

The mobile primary navigation control must include a visible text label.

✔️ Preferred label:

• Menu

The menu icon may appear with the label, but the visible text label must remain present.

The control should clearly communicate that activating it opens the primary navigation.

The Menu control and any close control must provide a target area large enough for accurate cursor and tap activation. Use at least 24 by 24 CSS pixels unless a valid exception applies. This is a minimum, not the ideal target size. In practice, mobile menu controls, close controls, and accordion rows should be larger than the minimum whenever layout allows.

8.2 Opening and closing mobile navigation

Mobile navigation must open and close by intentional activation or defined keyboard dismissal.

✔️ Required behavior:

• Activate Menu: open primary navigation.

• Activate Menu again or activate a close control: close primary navigation.

• Press Esc: close primary navigation when keyboard focus is inside the menu.

• Focus returns to the menu control when the menu is dismissed.

Hover must not open, close, reveal, or dismiss mobile navigation or child navigation sections in this pattern.

The open and closed states should be visually clear.

When the mobile menu is open:

• The menu control should show an open state or be replaced by a clear close control.

• The navigation panel or menu region should be visible.

• The first available item should be reachable through the expected tab order.

• The user should be able to dismiss the menu without activating a destination.

If the mobile navigation panel blocks interaction with the rest of the page, treat it as a modal-style experience: prevent background interaction, provide a clear close control, support Esc, and restore focus to the Menu control when dismissed. If the panel does not block interaction with the rest of the page, focus should remain free to move through the page and should not be trapped inside the navigation.

8.3 No split-button behavior

Mobile navigation rows must not use split-button behavior.

❌ Avoid this pattern:

• Text label navigates to a parent landing page.

• Adjacent chevron opens a child menu.

This creates two interaction targets within what appears to be one navigation row. Users may not understand that the label and chevron perform different actions.

✔️ Required pattern:

• Each row has one interaction target.

• A row either navigates to a destination or opens a child section.

• A row must not do both.

If the row opens a child section, the whole row should operate as the Menu item. The dropdown indicator may remain visible, but it should not be a separate interaction target.

8.4 Parent category with landing page

If a parent category has a landing page, include that landing page as a child destination item.

✔️ Required pattern:

• The parent row opens or closes the child section.

• The parent landing page appears as the first child destination item.

• The parent landing page uses an explicit overview label.

Example structure:

Parent Category
  Parent Category Overview
  Child Destination
  Child Destination

This keeps the parent row’s interaction simple and gives users a clear way to reach the parent landing page.

8.5 Accordion behavior

Mobile child navigation should use disclosure or accordion behavior.

✔️ Required behavior:

• Activate parent row: expand child section.

• Activate the same parent row again: collapse child section.

• Expanded state is visually indicated.

• Expanded state is programmatically exposed with aria-expanded.

• Child items appear directly under the parent row.

• The dropdown indicator visually reflects expanded or collapsed state.

Accordion behavior should preserve the parent-child relationship visually. Child items should appear close enough to the parent row that users can understand the relationship without relying on memory.

Hover must not expand or collapse accordion sections in this pattern.

8.6 Mobile current page indication

The currently displayed page, section, or route must have a visible current page indicator in mobile navigation.

✔️ Requirements:

• The current page indicator must be visible when the relevant navigation section is displayed.

• The indicator must be persistent and not depend on hover or focus.

• The current destination item must use aria-current=”page”.

• If the current page is inside a collapsed parent section, the parent row may receive a visual parent-active treatment.

• The parent row should not use aria-current=”page” unless the parent row itself represents the current page.

This allows users to reopen the mobile navigation and understand where they are within the application structure.

9. Keyboard Interaction Requirements

9.1 Tab order

Tab order must follow the visual and logical order of the primary navigation.

The goal is for keyboard users to experience the navigation in the same order that sighted cursor users visually scan it.

✔️ Recommended order:

1. Logo or home link, if present

2. Top-level primary navigation items

3. Open child menu items

4. Utility items, if present

5. Main page content

When a dropdown menu or mega menu is closed, its child items should not be included in the tab order.

When a dropdown menu or mega menu is open, its child items must be reachable through the keyboard.

Tab order should not jump unexpectedly between unrelated areas of the page. Users should be able to move through the navigation predictably and understand where they are at each step.

Hover must not affect tab order. Child items should enter or leave the tab order based on the menu’s activated open or closed state, not cursor position.

9.2 Keyboard commands

Primary navigation must support standard keyboard behavior.

Keyboard behavior should match the item type.

Destination items behave like links:

• Enter activates the item.

• The user is taken to the destination.

Menu items behave like buttons:

• Enter activates the item.

• Space activates the item.

• The child menu opens or closes.

Keyboard activation is intentional activation. A menu may open or close through Enter or Space on a Menu item. A menu must not open or close because the cursor is hovering over the item while it has focus.

9.3 Focused state

Every interactive item must have a visible focused state.

The focused state tells keyboard users where they are in the navigation. It should be treated as a required interaction state, not a visual enhancement.

Focused state must be:

• Clearly visible

• Persistent while the item has focus

• Distinct from hover state

• Distinct from current page state

• Distinct from expanded and collapsed states

• Distinct from active / pressed state

• Not dependent on color alone

Do not remove browser focus styling unless it is replaced with an equally clear or stronger focused state.

The focused state should work across:

• Top-level items

• Child items

• Dropdown indicators when they are part of the same disclosure control

• Menu controls

• Close controls, if present

Focused state must not automatically open a dropdown menu or mega menu. If a Menu item receives focus, it should show focused state only. The menu opens only when the user activates the Menu item.

9.4 Focus management

Focus behavior should help users stay oriented.

When a menu is opened by activation, the team should choose one focus model and apply it consistently.

Acceptable patterns:

1. Focus remains on the Menu item: The menu opens, and the user can continue forward through the tab order into the child items.

2. Focus moves into the opened menu: Focus moves to the first focusable child item inside the opened menu.

In both patterns:

• Child items must be keyboard reachable.

• Users must be able to move focus out of the menu.

• Focus should not be trapped inside standard dropdown menus or mega menus.

• Esc must close the menu and return focus to the Menu item that opened it.

• Focus order must match the visual and logical order of the menu.

• Focus behavior must not depend on hover.

Focus should only be trapped when the navigation is intentionally presented as a modal experience, such as a full-screen mobile menu that blocks interaction with the rest of the page. In that case, prevent background interaction, provide a clear close control, support Esc, and return focus to the Menu control when dismissed. If the mobile menu does not block the page, do not trap focus inside the navigation.

If focus movement is used as a dismissal method, it must be intentionally defined and consistent. Moving focus out of the open navigation region may dismiss the menu; moving focus from the Menu item into its child menu must not dismiss it.

10. Cursor Interaction Requirements

10.1 Hover / over state

Hover state may indicate that an item is interactive.

Hover state may change:

• Background

• Text decoration

• Border

• Color

• Elevation

• Indicator styling

In this pattern, hover must not:

• Open a menu

• Close a menu

• Reveal child items

• Dismiss child items

• Change expanded or collapsed state

• Change tab order

• Replace focused state

• Replace current page state

• Trigger behavior that changes the available navigation structure

The user may hover over an item and see an over state, but the navigation structure must not change until the user intentionally activates an item.

10.2 Hover out

Hover out removes the hover state.

Hover out must not close a dropdown menu or mega menu in this pattern.

Users should not have to move the cursor through a narrow path or maintain exact cursor placement to keep a menu open. That creates fragile navigation behavior and increases accidental dismissal.

✔️ Required rule:

• If a menu was opened by activation, it remains open until the user intentionally dismisses it.

Open menus should close through deliberate dismissal methods, such as:

• Activating the Menu item again

• Activating outside the menu

• Pressing the Esc key

• Moving focus out of the menu, when defined

• Activating another Menu item

• Activating a destination item

10.3 Activate outside

Activating outside an open menu should dismiss the menu.

This applies to:

• Dropdown menus

• Mega menus

• Mobile navigation panels, when appropriate

Activate outside behavior should not interfere with destination activation. If the user activates a valid destination item, the destination action should take priority.

Activating outside is intentional dismissal. Hovering outside the menu is not dismissal.

10.4 Cursor movement

Menus must not depend on precise cursor movement.

This is especially important when:

• A child menu appears below or beside the parent item

• The menu has multiple columns

• The user must move the cursor across spacing between the parent item and child menu

• The menu contains visual groups

The user should be able to move from the parent item into the child menu without fighting the interaction model.

Cursor movement may change hover / over state. Cursor movement must not change whether the menu is open or closed.

10.5 Target size

Interactive targets in primary navigation must be large enough to activate accurately with cursor and touch input.

Minimum target guidance:

• Primary navigation links, disclosure buttons, mobile menu controls, close controls, and accordion rows should provide a target area of at least 24 by 24 CSS pixels unless a valid exception applies. Treat this as the minimum threshold, not the preferred size. For mobile menu controls, close controls, and accordion rows, aim larger than the minimum whenever layout allows.

• Adjacent targets should have enough spacing that users are not likely to activate the wrong item by accident.

• The visible label, icon, or dropdown indicator may be smaller than the target area, but the interactive target area itself must be large enough.

• Do not rely on tiny chevrons or icons as the only practical target for expanding or collapsing navigation sections.

This requirement supports cursor, tap, motor accessibility, and general usability. It is especially important for mobile navigation, dense desktop navigation bars, and any close or menu control.

11. Interaction States

Each interactive item should define the following states as applicable.

The visual system must also define how states combine. A single item can be focused and hovered, focused and active, focused and current, hovered and current, focused and expanded, or focused and disabled. Combined states should not be treated as new semantic roles. They are visual combinations of existing states.

For example, focused + active means the item has keyboard focus and is currently being activated. It does not mean the item has a special hybrid role. focused + current page means the item represents the current page and currently has focus. It does not mean the current page state and focused state are the same thing.

11.1 Default state

Default state is the base visual treatment for an available item before temporary interaction states such as hover, focus, active, or pressed are applied.

For Menu items, the default visual treatment may appear together with collapsed state. A collapsed Menu item can still be in its default visual presentation.

✔️ Requirements:

• Default state should make the item readable.

• Default state should make the item appear available.

• Default state should not visually compete with current page state.

• Default state should not be confused with disabled state.

• Default state should not imply that a child menu is open.

11.2 Hover / over state

The cursor is over the item.

✔️ Requirements:

• Hover state must visually indicate interactivity.

• Hover state must be visually distinct from focused, current page, expanded, and active / pressed states.

• Hover state must be subtle enough that it does not compete with the current page indicator.

• Hover state must not open, close, reveal, or dismiss a menu in this pattern.

• Hover state must not change expanded or collapsed state.

Hover state is visual feedback only.

11.3 Focused state

The item has keyboard focus.

✔️ Requirements:

• Every interactive item must have a visible focused state.

• Focused state must be clearly visible.

• Focused state must not rely on color alone.

• Focused state must be visually distinct from hover, current page, expanded, and active / pressed states.

• Focus must follow the defined tab order.

• Focused state must not automatically open a menu.

Focused state should always answer the user’s question:

Where am I right now?

A focused Menu item opens or closes its child menu only when the user activates it with Enter or Space.

11.4 Active / pressed state

The item is actively being activated.

This state occurs during intentional activation, such as when the user presses the mouse buttons, Enter, or Space.

✔️ Requirements:

• Active or pressed state should provide immediate visual feedback.

• Active or pressed state should be temporary.

• Active or pressed state should not be used to represent the current page.

• Active or pressed state should not be confused with focused state.

• Active or pressed state should not be confused with current page state.

This state communicates that the user’s input has been received.

Hover does not create active or pressed state. The item enters active or pressed state only during intentional activation.

11.5 Current page state

The item representing the currently displayed page, section, or route must have a visible current page indicator.

✔️ Requirements:

• The current page indicator must be visually persistent.

• The current page indicator must be visible without hover or focus.

• The current page indicator must be visually distinct from hover, focused, expanded, and active / pressed states.

• The current page indicator must not rely on color alone.

• The current destination item must use aria-current=”page”.

• Only one item in the same navigation set should use aria-current=”page”.

Example:

<a href="/current-path" aria-current="page">Current item</a>

If the current page is represented by a child item inside a dropdown menu or mega menu, the child item should receive the current page indicator.

Example structure:

Parent Item
  Child Destination
  Child Destination ← current page
  Child Destination

In this pattern:

• The current child destination receives the current page indicator.

• The current child destination uses aria-current=”page”.

• The parent item may receive a visual parent-active treatment.

• The parent item should not use aria-current=”page” unless the parent item itself is the current page.

The parent-active treatment helps users understand which top-level section contains the current page. It should be visually distinct from the child item’s current page indicator.

Current page state must not depend on hover, focus, or menu open state.

11.6 Selected state

Selected state should not be used as the normal way to represent the current page in primary navigation.

Use selected state only when the component supports selection as a state separate from navigation.

Examples of appropriate selected state:

• A selected item inside a picker

• A selected item inside a switcher

• A selected item inside an application-style component

• A selected tab in a tablist

✔️ Requirements:

• Selected state must be visually distinct from hover, focused, current page, expanded, and active / pressed states.

• Selected state must not be used as a substitute for current page state.

• Do not use aria-selected for standard primary navigation links.

• Use aria-selected only for components with true selection semantics, such as tabs, listboxes, grids, or tree views.

For standard primary navigation, the user is usually not selecting an item into a persistent selection set. They are navigating to a location. In that case, use current page state instead.

If a visual state guide uses the phrase Selected (Current Page), the implementation should still treat that state as current page state and expose it with aria-current=”page”, not aria-selected.

11.7 Expanded state

The Menu item’s child menu is open.

✔️ Requirements:

• Expanded state must be visually indicated.

• The Menu item must use aria-expanded=”true”.

• The dropdown indicator should visually reflect the expanded state.

• Expanded state must be visually distinct from hover, focused, current page, and active / pressed states.

• Expanded state must result from intentional activation, not hover.

Expanded state should answer the user’s question:

Which item opened this menu?

11.8 Collapsed state

The Menu item’s child menu is closed.

✔️ Requirements:

• Collapsed state must be visually indicated.

• The Menu item must use aria-expanded=”false”.

• The dropdown indicator should visually reflect the collapsed state.

Collapsed state should still communicate that child items are available, usually through the dropdown indicator.

Collapsed state must result from initial page state or intentional dismissal behavior, not hover out.

11.9 Disabled state

The item is visible but not currently available.

Use disabled navigation items sparingly.

✔️ Requirements:

• Disabled items must not be activatable.

• Disabled state must be visually distinct from default state.

• Disabled state must not rely on color alone.

• Disabled items should not receive keyboard focus unless there is a specific accessibility reason.

• Disabled items must not open, close, reveal, or dismiss menus.

• Hover over a disabled item may show a disabled visual treatment, but it must not trigger behavior.

When possible, avoid showing disabled items unless the unavailable option helps users understand status, permissions, or workflow constraints.

If users cannot do anything with the item and do not need to know it exists, consider removing it from the navigation instead of showing it as disabled.

11.10 Combined interaction states

Interaction states can combine. The design system should document these combinations visually so implementation teams know which cue takes priority.

Common combinations include:

✔️ Requirements:

• Combined states must preserve focus visibility.

• Combined states must preserve the current page indicator.

• Combined states must not imply that focus, current page, active, or selected are the same concept.

• Combined states must not create a hybrid interaction model.

• Combined states must not change whether an item behaves as a destination item or Menu item.

• If the dropdown indicator is present for a Menu item, it should remain visible across applicable states.

The attached primary navigation item state guide should be treated as a visual reference for documenting state combinations. The language in this pattern should describe these as combinations, such as Focused + Hover or Focused + Active + Current Page, not as single blended states.

12. Accessibility Requirements

12.1 Semantic HTML first

Use semantic HTML before ARIA.

From the interaction design perspective, semantic HTML matters because it gives users expected behavior for free. Links behave like links. Buttons behave like buttons. Navigation landmarks behave like navigation landmarks.

✔️ Required:

• Use <nav> for the primary navigation landmark.

• Use <a> for destination items.

• Use <button type=”button”> for Menu items.

• Use <ul> and <li> for item collections when appropriate.

• Use visible text labels wherever possible.

❌ Avoid:

• Styling <div> or <span> elements as links or buttons.

• Adding role=”button” to non-button elements unless there is no native option.

• Adding redundant ARIA to native HTML elements.

• Using ARIA as a substitute for correct HTML structure.

The interaction rule should come first:

• If the item navigates, use a link.

• If the item opens or closes something, use a button.

Menu items must be activated intentionally. Do not create custom hover-triggered disclosure behavior for this pattern.

12.2 ARIA use

ARIA should only be used when semantic HTML does not provide the required name, state, or relationship.

Use ARIA to communicate missing information, not to recreate native behavior that already exists.

ARIA must stay synchronized with the visual state.

For example:

• If the menu is visually open, the Menu item should use aria-expanded=”true”.

• If the menu is visually closed, the Menu item should use aria-expanded=”false”.

Hover must not change ARIA state in this pattern. aria-expanded should change only when the menu is opened or closed through activation or defined dismissal behavior.

When a menu opens or closes, hidden, keyboard focusability, and aria-expanded should be updated together so users do not encounter hidden focusable content or visible content that is missing the correct expanded state.

12.3 Accessible names

Every interactive item must have an accessible name.

✔️ Preferred order:

1. Visible text label

2. Visible text referenced with aria-labelledby

3. aria-label only when no visible label or referenceable text is available

Do not use aria-label as the default labeling method.

Visible labels are preferred because they create the same cue for sighted users and screen reader users. This keeps the interaction model consistent across users.

12.4 Menu roles

Do not use role=”menu” or role=”menuitem” for standard primary navigation unless the component intentionally follows application menu behavior.

Most primary navigation should use native links, buttons, lists, and disclosure behavior.

Use application menu roles only when the component behaves like an application menu and supports the expected keyboard model for that pattern.

For standard web application primary navigation, application menu roles usually add unnecessary complexity.

Menu roles must not be used to justify hover-triggered behavior. The interaction model must still support intentional activation, keyboard operation, clear focus behavior, and predictable dismissal.

12.5 Target size

Frontend implementation must provide cursor and tap targets that are large enough to activate accurately.

✔️ Requirements:

• Interactive navigation targets should be at least 24 by 24 CSS pixels unless a valid exception applies. This is the minimum threshold, not the preferred size. Mobile menu controls, close controls, and accordion rows should be larger than the minimum whenever layout allows.

• Adjacent interactive targets should include enough spacing to reduce accidental activation.

• The whole disclosure row should usually be the target when a row expands or collapses a child section.

• Dropdown indicators should communicate state, but they should not become the only small target users must hit.

• Mobile menu controls, close controls, accordion rows, top-level links, and disclosure buttons must all be reviewed for target size.

12.6 Reduced motion

Navigation must remain understandable and usable when motion is reduced or removed.

✔️ Requirements:

• Non-essential animation should respect prefers-reduced-motion.

• Reduced motion should reduce or remove menu, panel, accordion, and indicator animations when appropriate.

• State changes must still be clear without relying on animation.

• Focus visibility must not be weakened by animation or by reduced-motion behavior.

• Reduced-motion behavior must not change the underlying keyboard, cursor, or tap interaction model.

13. Responsive Behavior

13.1 Preserve interaction logic

Interaction logic should remain consistent across breakpoints.

The layout may change from a horizontal desktop navigation bar to a mobile menu, but the meaning of each item should not change.

Across desktop and mobile:

• Destination items navigate directly.

• Menu items open child menus.

• Menu items include indicators.

• Destination items do not include indicators.

• Parent landing pages appear as overview child items.

• Split-button behavior is avoided.

• Menus and child sections open by activation only.

• Hover does not open, close, reveal, or dismiss navigation content in this pattern.

A user should not need to relearn what an item does because the viewport changed.

13.2 Preserve navigation meaning

Responsive layouts must preserve:

• Label clarity

• Parent-child relationships

• Destination availability

• Current page indication

• Keyboard access

• Screen reader access

When primary navigation collapses into a mobile menu, users should still be able to understand:

• Which section they are in

• Which parent item contains the current page

• Which items navigate directly

• Which items expose child items

• How to open and close child sections

Do not simplify the mobile version by removing important destinations unless there is a clear, user-centered reason.

13.3 Preserve current page indication

The current page indicator must work across breakpoints.

On desktop:

• The current destination item should show the current page indicator.

• If the current item is inside a dropdown or mega menu, the parent item may receive a parent-active treatment.

On mobile:

• The current destination item should show the current page indicator when visible.

• If the current item is inside a collapsed section, the parent row may receive a parent-active treatment.

• Opening the parent section should reveal the child item with the current page indicator.

The user should be able to open primary navigation and understand where they are in the application.

Current page indication must be persistent. It must not depend on hover, focus, or whether the user has recently interacted with the menu.

13.4 Preserve dismissal behavior

Menu dismissal behavior should remain predictable across breakpoints.

Users should be able to dismiss open navigation UI by:

• Activating the menu control again, when applicable

• Activating a close control, when provided

• Activating outside the navigation panel, when appropriate

• Pressing the Esc key

• Activating a destination item

When dismissal occurs through the Esc key, focus should return to the item that opened the menu or panel.

Hover out must not dismiss open navigation UI on any breakpoint in this pattern.

14. Common Anti-Patterns

14.1 Hover-triggered dropdown menus

❌ Problem:

• Do not design patterns where the menu opens, closes, reveals, or dismisses based on hover interactions.

Why this fails in this pattern:

• The menu can open unintentionally.

• The menu can close unintentionally.

• Touch users cannot rely on hover.

• Keyboard users may not receive equivalent access.

• Users have less control over when the menu opens or closes.

• Cursor movement becomes part of the interaction logic instead of simple visual feedback.

✔️ Correction:

• Use activation-triggered disclosure behavior for this navigation pattern.

• Keep hover as visual feedback only.

• Do not use hover to change menu visibility in this pattern.

14.2 Icon-only mobile menu control

❌ Problem:

• Do not design patterns where the primary navigation on mobile is hidden behind an unlabeled icon.

Why this fails in this pattern:

• The icon may not be recognized by all users.

• The control depends on recall or learned convention.

• The interaction has weaker information scent.

• The accessible name may not match the visible UI.

✔️ Correction:

• Use a visible Menu label.

• The icon may remain as a supporting visual cue.

14.3 Split-button navigation row

❌ Problem:

• Do not design patterns where one row contains separate targets for navigation and submenu expansion.

Why this fails:

• Users may not understand that the label and indicator perform different actions.

• The row appears to have one function but actually has two.

• The smaller target is easier to miss.

• Behavior becomes less predictable on mobile.

• The pattern makes users determine whether the text, icon, or surrounding row is the correct target.

✔️ Correction:

• Use one interaction target per row.

• If the parent category has a landing page, include it as an overview child item.

• Activating the parent row should either navigate or disclose child items, not both.

• The dropdown indicator should communicate disclosure state, not act as a separate control.

14.4 Vague item labels

❌ Problem:

• Do not design patterns where the item label does not clearly communicate the destination, task, or category.

Why this fails:

• Users have to interpret the label before activating it.

• The item may appear to overlap with other items.

• Users may open the item only to understand what it means.

• The navigation becomes harder to scan.

✔️ Correction:

• Use plain, descriptive, user-recognizable labels.

• Use labels that describe what the user will find or do after activating the item.

• Avoid labels that depend on internal terminology or feature team language.

14.5 Duplicated destinations

❌ Problem:

• Do not design patterns where the same destination appears in multiple primary navigation areas.

Why this fails:

• Users may wonder whether the items lead to the same place or different places.

• The structure becomes harder to learn.

• The navigation may appear larger and more complex than it is.

✔️ Correction:

• Place each destination in the most user-expected location.

• Duplicate destinations only when research shows users strongly expect the same destination in more than one category.

14.6 Deep nested navigation

❌ Problem:

• Do not design patterns where users must move through multiple levels before reaching a destination.

Why this fails:

• Each level adds interaction cost.

• Each level requires users to interpret another abstraction.

• The path becomes easier to abandon or misunderstand.

✔️ Correction:

• Flatten the hierarchy where practical.

• Expose terminal nodes more directly.

• Use grouped mega menu layouts when many destinations need to be visible.

14.7 Non-semantic controls

❌ Problem:

• Do not design patterns where non-interactive element is styled to behave like a link or button.

Why this fails:

• Keyboard behavior may not work correctly.

• Screen reader behavior may be incorrect or incomplete.

• Focus behavior may be missing.

• Users may receive false cues about what the item does.

✔️ Correction:

• Use native <a> elements for destination items.

• Use native <button type=”button”> elements for Menu items.

• Use ARIA only when semantic HTML does not provide the required name, state, or relationship.

14.8 Missing expanded state

❌ Problem:

• Do not design patterns where the menu opens visually but does not expose expanded or collapsed state.

Why this fails:

• Screen reader users may not know whether the child menu is open.

• The state of the item becomes unclear.

• The visual UI and programmatic UI are out of sync.

✔️ Correction:

• Use aria-expanded on the Menu item.

• Keep aria-expanded synchronized with the visual state.

• Update expanded state only when the menu is opened or closed through intentional activation or dismissal.

14.9 Defaulting to ARIA instead of semantic HTML

❌ Problem:

• Do not design patterns where ARIA is used to recreate behavior that native HTML already provides.

Why this fails:

• The implementation becomes more fragile.

• Native keyboard behavior may be lost.

• Assistive technology behavior may become less predictable.

• The team may accidentally create a custom component with incomplete interaction support.

✔️ Correction:

• Use semantic HTML first.

• Use ARIA only to communicate missing names, states, or relationships.

14.10 Misusing selected state

❌ Problem:

• Do not design patterns where selected state is used to represent the current page in standard primary navigation.

Why this fails:

• Selection and current location are different concepts.

• aria-selected communicates selection semantics, not current page location.

• Users of assistive technology may receive the wrong cue.

✔️ Correction:

• Use current page state for the current route.

• Use aria-current=”page” on the current destination item.

• Reserve selected state and aria-selected for components with true selection semantics.

14.11 Treating combined states as new interaction types

❌ Problem:

• Do not design patterns where a combined visual state is documented as though it were a new standalone interaction type.

Why this fails:

• It can blur the difference between focus, hover, active, current page, expanded, collapsed, selected, and disabled.

• It can make implementation teams think the component needs additional semantic states.

• It can make accessibility state management more complicated than it needs to be.

✔️ Correction:

• Document combined states as combinations of existing states.

• Preserve the semantic meaning of each state.

• Use aria-current=”page” for current page state.

• Use aria-expanded for disclosure state.

• Do not use aria-selected for standard primary navigation links.

• Do not treat focus combinations as hybrid interaction patterns.

15. Validation Checklist

15.1 Information architecture

• Parent-child relationships are clear.

• Child items belong logically under their parent items.

• Terminal nodes are easy to identify.

• The hierarchy is as flat as practical.

• Child items are grouped by user expectations.

• Duplicate destinations are avoided.

• Parent landing pages are represented as overview child items.

15.2 Desktop behavior

• Menus open by activation only.

• Hover does not open menus in this pattern.

• Hover does not close menus in this pattern.

• Hover does not reveal child items in this pattern.

• Hover does not dismiss child items in this pattern.

• Hover state is visual feedback only.

• Menu items include dropdown indicators.

• Destination items do not include dropdown indicators.

• Menus can be dismissed by Esc, activate outside, moving focus out of the open navigation region when defined, trigger toggle, or activating another item.

• Animations are brief and do not delay access to child items.

• Non-essential animation respects reduced-motion preferences.

• Esc returns focus to the item that opened the menu.

15.3 Mobile behavior

• Mobile menu control has a visible Menu label.

• Icon-only primary navigation controls are not used.

• Mobile navigation opens by activation only.

• Mobile navigation closes by activation or defined dismissal behavior.

• Split-button behavior is not used.

• Each row has one interaction target.

• Parent landing pages are included as overview child items.

• Accordion sections expand and collapse by activation only.

• Accordion state is visually and programmatically exposed.

• Current page indication is preserved in the mobile menu.

• Menu and close controls meet target-size requirements and are larger than the minimum whenever layout allows.

• Accordion rows provide a target area large enough for touch input and are larger than the minimum whenever layout allows.

• Panel and accordion animation respects reduced-motion preferences.

• If a mobile navigation panel blocks background interaction, it is treated as a modal-style experience with intentional focus management, Esc support, background-interaction prevention, and focus restoration.

15.4 Keyboard behavior

• Tab order follows the visual and logical structure.

• Every interactive item has a visible focused state.

• Enter activates links and buttons.

• Space activates buttons.

• Esc dismisses open menus.

• Focus returns to the triggering item after dismissal.

• Focus is not trapped inside standard dropdown menus or mega menus.

• Focused state alone does not open a menu.

15.5 Cursor behavior

• Hover / over state is visual feedback only.

• Cursor movement does not open menus.

• Cursor movement does not close menus.

• Cursor movement does not reveal child items.

• Cursor movement does not dismiss child items.

• Hover out removes only the hover state.

• Activate outside dismisses open menus when appropriate.

• Interactive targets are at least 24 by 24 CSS pixels unless a valid exception applies, with larger mobile targets used whenever layout allows. The 24 by 24 size is the minimum threshold, not the preferred target size.

• Adjacent targets include enough spacing to reduce accidental activation.

15.6 Interaction states

• Default state is defined as the base visual treatment for an available item.

• Hover / over state is defined as visual feedback only.

• Focused state is defined.

• Active / pressed state is defined.

• Current page state is defined.

• Selected state is defined only where selection semantics apply.

• Expanded state is defined.

• Collapsed state is defined.

• Disabled state is defined only where needed.

• Common combined states are documented visually.

• Combined states preserve focus visibility.

• Combined states preserve the current page indicator.

• Combined states do not create new semantic roles.

• Combined states do not imply hybrid interaction behavior.

• Current page state is visually distinct from hover, focused, selected, expanded, and active / pressed states.

• The currently displayed page, section, or route has a visible current page indicator.

• The current page indicator is persistent and does not depend on hover or focus.

• Parent items may receive visual parent-active treatment when the current page belongs to their child menu.

• Expanded and collapsed states change only through activation or defined dismissal behavior.

15.7 Accessibility

• Primary navigation uses semantic HTML by default.

• Destination items use native <a> elements.

• Menu items use native <button type=”button”> elements.

• Visible labels are used wherever possible.

• aria-labelledby is preferred when a visible or visually hidden label needs to be referenced.

• aria-label is used only when no visible or referenceable text is available.

• aria-label=”Primary” is preferred over aria-label=”Primary navigation” when labeling the primary navigation landmark directly.

• Current page state uses aria-current=”page”.

• Menu items use aria-expanded.

• aria-controls is optional and used only when an explicit control-to-menu relationship is needed.

• aria-expanded changes only when the visual menu state changes.

• Hover does not change ARIA state in this pattern.

• aria-selected is not used for standard primary navigation links.

• ARIA does not replace semantic HTML.

• Visual state and programmatic state stay synchronized.

• When menus open or close, hidden, keyboard focusability, and aria-expanded stay synchronized.

• Animated menu transitions do not leave focusable content hidden from sighted users or visible content hidden from assistive technologies.

• Disabled links are not implemented with a native disabled attribute. Unavailable destinations are removed, explained, or intentionally implemented with clear disabled behavior.

• Interactive navigation controls meet target-size requirements, with larger mobile controls used whenever layout allows.

• Non-essential navigation animation respects reduced-motion preferences.

• Navigation remains understandable when animation is reduced or removed.

16. Final Acceptance Criteria List

The primary navigation bar meets these specifications when:

1. Users can distinguish destination items from Menu items.

2. Dropdown menus and mega menus open only by intentional activation.

3. Hover does not open, close, reveal, or dismiss dropdown menus, mega menus, or child navigation items in this pattern.

4. Hover / over state is visual feedback only.

5. Menu items include dropdown indicators.

6. Destination items do not include dropdown indicators.

7. The dropdown indicator is not a separate interaction target.

8. Open menus can be dismissed through Esc, activate outside, trigger toggle, moving focus out of the open navigation region when defined, or activating another item.

9. Pressing Esc closes the menu and returns focus to the item that opened it.

10. The currently displayed page, section, or route has a persistent visible current page indicator.

11. The current page indicator is visually distinct from hover, focused, selected, expanded, and active / pressed states.

12. The current destination item uses aria-current=”page”.

13. Parent Menu items may receive visual parent-active treatment when the current page belongs to their child menu.

14. Parent Menu items do not use aria-current=”page” unless the parent item itself is the current page.

15. The mobile menu control includes a visible Menu label.

16. Mobile navigation opens by activation only.

17. Mobile navigation does not use split-button behavior.

18. If a mobile navigation panel blocks background interaction, it is treated as a modal-style experience with intentional focus management and focus restoration.

19. Parent landing pages are available as overview child items.

20. Accordion sections expand and collapse by activation only.

21. All interactive items are keyboard operable.

22. All interactive items have visible focused states.

23. All interactive navigation targets provide at least 24 by 24 CSS pixels of target area unless a valid exception applies, with larger mobile targets used whenever layout allows. The 24 by 24 size is treated as the minimum threshold, not the preferred target size.

24. Adjacent interactive targets include enough spacing to reduce accidental activation.

25. Focused state alone does not open a menu.

26. Disclosure state is exposed with aria-expanded.

27. aria-controls is optional and used only when an explicit control-to-menu relationship is needed.

28. aria-expanded stays synchronized with the visual open or closed state.

29. When menus open or close, hidden, keyboard focusability, and aria-expanded stay synchronized.

30. Animated menu transitions do not leave focusable content hidden from sighted users or visible content hidden from assistive technologies.

31. Hover does not change aria-expanded in this pattern.

32. Item labels are clear, consistent, and user-recognizable.

33. The navigation hierarchy avoids unnecessary nesting.

34. Terminal nodes are clearly exposed within child menus.

35. Primary navigation uses semantic HTML by default.

36. aria-label is used only when no visible or referenceable label is available.

37. aria-label=”Primary” is preferred over aria-label=”Primary navigation” when directly labeling the primary navigation landmark.

38. ARIA is only used to communicate missing names, states, or relationships that semantic HTML does not provide on its own.

39. Non-essential navigation animation respects prefers-reduced-motion.

40. Navigation state remains clear when animation is reduced or removed.

41. Selected state is not used as a substitute for current page state.

42. aria-selected is not used for standard primary navigation links.

43. Combined visual states are documented as combinations of existing states, not as new semantic states.

44. Focus combinations do not imply a hybrid interaction pattern.

45. Disabled navigation items are used sparingly and only when they help users understand status, permissions, or workflow constraints.

46. Disabled links are not implemented with a native disabled attribute. Unavailable destinations are removed, explained, or intentionally implemented with clear disabled behavior.

    ---

Have you ever been in a meeting where everyone on the team starts debating a call to action, a dashboard card, an onboarding step, standard form field behavior, or some other part of a design, and everyone seems to have a strong opinion? One person thinks the button label should be more direct. Someone else thinks the page needs more explanatory text. Someone else wants to move the action higher on the screen because they are worried users will miss it. Then someone says that dreaded question, “Why don’t we just A/B test it?”

To all the non-UX researchers in the room, that sounds like a valid question. It feels objective and data-driven. I get the appeal. Instead of arguing, the team can let users decide through behavior. Easy peasy. Done and done.

We all know that in the right context, that can be a good instinct. A/B testing can be useful because it gives teams a way to compare real behavior in a live product. It can reduce opinion-driven decision-making, help teams avoid endless internal debates, and sometimes reveal that the version everyone loved in a meeting is not the version users actually respond to in practice.

The problem starts when the team treats the test itself as the thing that creates rigor.

A/B testing only compares the variations the team puts into the test. It does not make those variations are any good. If the variation is shallow, the learning will probably be shallow too. If the variation is based on a misunderstanding of the user problem, the result may only tell the team which version of the misunderstanding performed better.

In this article, I want to help reframe the way UX professionals respond to the question, “Why don’t we just A/B test it?” Instead of shutting the idea down outright, a better response is to slow the team down just enough to ask what the variation is actually testing. That is where UX research can add a lot of value before the test ever goes live.

What A/B Data Leaves Out

To run good A/B tests, you need to focus on the work that happens before the traffic split. The team has to clearly define what problem they believe exists, what user behavior they are trying to learn about, and what design variation would best test that belief. You guessed it: this is called strategy and study design. If your strategy sucks, the test can still produce clean-looking data, but it can also be super misleading.

The example I like to use is what I call the “make it more obvious” variation. You know the type of test. Let’s say Version A has a standard button at the bottom of a screen. Version B has a larger, higher-contrast button with more urgent language. The team runs the test, and Version B gets more clicks, so the result gets interpreted as proof that the new design is better. That may be true in one narrow sense. Version B produced more clicks on the metric the team selected.

Here are several different explanations packed into that result:

• The button may have been easier to find.

• The copy may have been clearer.

• The visual treatment may have pulled attention away from other information.

• The urgent language may have pushed users to act before they fully understood the decision.

Those are not small differences. They point to different design problems and different next steps. If the real issue was discoverability, the team might need a better layout hierarchy. If the real issue was unclear copy, the team might need a better label. If the improvement came from pressure, the team may have increased short-term clicks at the expense of trust or downstream success.

In UX research, we are usually trying to understand how people make sense of a system, where they struggle, what they need to know before acting, and whether the design helps them complete the task with confidence. A/B testing can support that work, but it does not replace the thinking needed to create a meaningful variation. This is where I see the method misused the most. In the overuse and overinterpretation of the data.

Strong Variations

I see too many of these misleading and weak A/B test strategies in the real world. A weak A/B test often starts with a super vague hypothesis. “We think this version will perform better” is technically a prediction, but it does not give the team much to go off of. It does not explain what user problem the team thinks it is solving, what part of the design is expected to change behavior, or what the team will learn if the variation wins or loses.

A stronger UX hypothesis is much more specific. For example, “We believe users are missing the primary action because the current button label is vague and visually disconnected from the task. If we make the label more specific while keeping the button placement, size, and color the same, more users will continue to the next step.” The value in this reframing is that it forces the team to be clear about the link between the user problem, the design change, and the expected behavior change.

This is where we can make our A/B testing strategies much stronger. Before the team creates a variation, research can help explain what is actually happening in the current experience:

• Usability testing can show where users hesitate, misread the interface, or misunderstand the next step.

• Interviews can reveal concerns users have before committing to an action.

• Behavioral analytics can show repeated backtracking, abandonment, or drop-off patterns.

• Session replay can reveal behaviors like misclicks or repeated attempts to use unavailable or hidden controls.

Those findings lead to better variations because they ground the design change in observed behavior. The team is no longer guessing if the button needs to be a brighter color because someone in the room thinks bright colors buttons perform better. Instead, they are responding to a specific problem within the experience, in the real world.

This is an overly simplistic example, but I hope you can see how this changes the quality of the test. Instead of comparing two versions of a screen because the team has two opinions, the team is testing a specific explanation for user behavior. The A/B test becomes a way to measure whether a research-informed design change improves the outcome the team cares about.

To be clear, that does not guarantee the variation will win. In some cases, the research-informed version will underperform. That data is still useful if the original hypothesis was clear. Losing variations can teach us as much as winning ones, and in many cases, they are more insightful because they unearth institutionalized bias.

How to Think About Claims

The phrase “A/B test” is powerful in many orgs, especially product-led ones that have taken a page out of the debunked Silicon Valley “move fast and break things” playbook. The power comes for the misunderstanding I’ve been talking about.

In the real world, teams often use A/B testing to compare two designs that differ in several ways at once. That can be useful, but the team needs to be honest about what kind of claim the test can support, and we, as UX researchers, should be there to help them with that.

Imagine Version A has a button labeled “Continue” at the bottom of the page. Help text is hidden behind an icon. The page has three sections, and users have to scroll to review everything before moving forward.

Version B changes the button label to “Review and Submit,” makes the button sticky, adds visible help text, reduces the page to two sections, and introduces a progress indicator.

If Version B performs better, the team has learned something. The new version, as a full design package, performed better than the old version. That may be enough if the team’s goal is simply to decide whether to ship the redesigned flow.

The problem comes when the team tries to make a narrower claim than the test design supports. If five things changed, the result does not prove that the button label caused the improvement. Any of those changes may have helped. Several may have worked together. One may have helped while another made the experience worse.

Here are some examples:

• One button label changes, while placement, size, color, and flow stay the same - Reasonable claim: evidence about the effect of the label change.

• A full-page redesign changes layout, copy, hierarchy, and interaction patterns - Reasonable claim: comparison between two design packages, not proof that one specific element caused the result.

• Several elements change at once without a clear hypothesis - Reasonable claim: the team may know which version won, but interpretation will be weak.

• Multiple variables need to be studied separately - Better fit: a multivariate test or staged testing plan that can isolate the contribution of each variable.

This is why the learning goal has to match the variation. If the team wants to know whether clearer button copy reduces hesitation, the variation should isolate the button copy as much as possible. Keep the placement, visual treatment, surrounding content, and flow stable. That gives the result a cleaner interpretation.

If the team wants to compare two broader design directions, a bundled variation may be fine. A larger redesign often changes layout, information hierarchy, interaction patterns, content, and visual emphasis together because those pieces are connected. In that case, the test is answering whether this complete version performs better than that complete version.

Both types of tests can be useful. They just answer different questions. A focused variation supports a focused claim. A bundled variation supports a broader comparison. Trouble starts when a team runs the second kind of test and talks about the result as if it came from the first.

For UX researchers, this is one of the most important places to push for clarity. The goal is to help the team understand what they will actually be able to learn. Sometimes that means isolating one variable. Other times it means testing a larger design direction. Either way, the variation should match the decision the team needs to make.

These Metrics in Context

A/B testing gets especially tricky when the success metric is too narrow. A variation can win on the primary metric and still make the overall experience worse. That is one of the easiest ways teams accidentally turn A/B testing into a machine for optimizing local behavior instead of improving the actual user experience.

This happens a lot with clicks. Clicks are easy to measure, so they often become the thing the team optimizes for. If Version B gets more clicks than Version A, the dashboard looks good. The test has a winner. The team can point to a number and say the variation worked.

But more clicks are not always better. A user can click because the next step is clearer. A user can also click because the design pressured them, distracted them, interrupted them, or made one action more visually dominant than everything else on the screen. Those are very different outcomes, even if they produce the same short-term metric.

This is where guardrail metrics matter. A team should know what the variation is supposed to improve, but they should also know what they do not want to make worse. Otherwise, the team may optimize one behavior while quietly damaging another part of the experience.

Some common examples:

• More clicks, but lower downstream completion

• More signups, but more support contacts

• More feature activation, but more abandonment later

• More engagement, but lower trust

• More interaction, but worse accessibility

The point is not that every A/B test needs a giant measurement framework. That would be overkill for many product decisions. The point is that the team should think beyond the one number that declares the winner.

A modal that increases feature activation might also increase dismissals, frustration, and support questions. A more aggressive confirmation message might push more users through a flow while making them less confident in what they just agreed to. A redesigned call to action might improve one user segment while creating problems for another segment with different goals, permissions, or prior knowledge.

In complicated software, this gets even messier because the thing being tested is often part of a larger workflow. A design change can make one screen perform better while creating confusion two steps later. The test result may look positive if the team stops measuring at the first click, but the user experience may not actually be better.

This is another place where UX researchers can add useful context. The question is not only, “Did the number go up?” The better question is, “What else might this variation affect?” That one question can change the quality of the test. It can push the team to look at completion, error rates, backtracking, support clicks, time on task, accessibility concerns, or downstream abandonment. It can also prevent the team from declaring victory too early, just because the easiest metric moved in the right direction.

Don’t Skip Step 1

UX researchers do not need to own the experimentation platform to improve the quality of A/B testing. In many orgs, the platform, instrumentation, traffic allocation, and statistical setup will sit with product analytics, engineering, data science, or growth teams. That does not mean UX research has nothing to contribute.

The most valuable UX contribution often happens before the test launches. That is when the team is still deciding what problem they are solving, what variation should be created, and what the result will actually mean. Once the test is live, a lot of the important decisions have already been made.

A researcher can help the team slow down just enough to make the test more meaningful. That does not have to mean a long research project. It might mean a quick usability check, a review of prior findings, a look at session behavior, or a conversation about whether the proposed variation actually matches the problem the team says they are trying to solve.

The practical UXR role often looks like this:

• Clarify the user problem before the variation is designed.

• Turn a vague idea into a testable hypothesis.

• Identify whether the proposed change matches the observed problem.

• Flag when too many things are changing at once.

• Recommend guardrail metrics.

• Run a quick prototype or usability check before the test goes live.

• Help the team interpret the result without overstating what it proves.

This is a playbook for how UX research can help keep the interpretation grounded. The team can say, “In this flow, for this user population, under these conditions, this variation improved this metric.” That may sound less exciting, but it is usually more accurate. It also makes the finding more useful because future teams can understand when the result might apply and when it probably does not.

Also, in a perfect world, A/B testing should not be the first time users encounter a variation, especially in complicated enterprise software. If the design has obvious usability issues, the live experiment will measure those issues along with everything else. A quick round of UX evaluation before launch can remove problems that would otherwise muddy the test. Literally, try anything to help put context around a problem before running an A/B test. That additional evaluation does not need to be elaborate. For some changes, a short prototype review with a few users may be enough. The goal is to make sure the variation is coherent before it gets measured at scale.

Better Variation Design Examples

Let’s say your team started with the old “let’s test a more prominent button” hypothesis. A better version of the conversation starts with questions that you, as the researcher, should be asking.

Questions like:

• Why does the button need to be more prominent?

• Are users missing it altogether?

• Are they unsure what it does?

• Are they afraid of committing to the next step?

• Are they distracted by competing actions on the page?

The answers to these questions lead to a different variation. That is why the design change should come after the problem is understood, not before. Otherwise, the team is just picking a visual change and hoping the test makes it meaningful.

Example questions worth asking before the test:

• What user behavior are we trying to change? - Keeps the test focused on a real behavior, not just a design preference.

• What do we think is causing the current behavior? - Forces the team to name the assumption behind the variation.

• What exactly are we changing? - Makes the variation easier to interpret later.

• What are we keeping the same? - Helps isolate the design variable when possible.

• What is the primary metric? - Defines how the team will judge the result.

• What are the guardrail metrics? - Helps catch harm outside the primary metric.

• What will we know after the test? - Keeps the test connected to a learning goal.

That structure does not need to become a formal template every team fills out before every test. It is more useful as a thinking tool. If the team cannot answer those questions in plain language, the variation probably is not ready.

Weak variation vs. stronger variation examples:

• ❌ Weak: “Make the button more noticeable.”

• ✔️ Stronger: “Increase visual prominence while keeping the label, placement, and flow unchanged to test whether discoverability is the issue.”

• ❌ Weak: “Try different copy.”

• ✔️ Stronger: “Change the label to clarify what happens next because users are hesitating before a commitment-heavy action.”

• ❌ Weak: “Test the new design against the old one.”

• ✔️ Stronger: “Test whether exposing required information earlier reduces abandonment before the confirmation step.”

• ❌ Weak: “Add help text.”

• ✔️ Stronger: “Add one line of contextual help under the field users misinterpret most often.”

Note that the stronger versions are not necessarily more complicated, but they are more specific. They connect the design change to a user problem and a learning goal. That makes the result easier to interpret, even when the result is not what the team expected.

This also helps with stakeholder conversations. When the variation is specific, the result is easier to explain. The team can say what changed, why it changed, what metric moved, and what the result suggests. That is much more useful than saying, “The new version won,” especially when the new version changed several things at once.

Better variation design does not mean every test has to be small. Some tests should compare broader design directions. The important thing is to be honest about the level of learning. A focused variation can tell the team something focused. A broader variation can help the team choose between broader directions. The test design should match the decision the team needs to make.

Conclusion

So, when a product team says, “Let’s just A/B test it,” the instinct to shut them down is reasonable, but we need to do better. As UX researchers, part of our job is to help shape how our orgs use user data, and A/B testing is a perfect example of where that responsibility shows up in practice. We need to lead A/B testing conversations as sense makers, not blockers. We have the opportunity to show how real users respond to real changes in the real world, and we should take those opportunities when they come.

But A/B tests are only as good as the thing being tested. If the variation is shallow, messy, or disconnected from the actual user problem, the result will inherit those weaknesses. It may still produce a winner, but winning is not the same thing as learning. That is the part I wish more teams took seriously. Better A/B testing does not start in the experimentation platform. It starts when the team asks a better question about the variation.

How do you help with A/B testing strategy in your org? Any tactics I missed? If so, comment here or DM me with the details. I’m always happy to add more things to my toolkit, especially when it comes to this misused and misunderstood method. Thanks for reading!

A while back, an earlier-career UX researcher came to me with a common time-on-task question. They had just run a moderated usability study with 8 participants. The task was fairly straightforward, and all participants completed the task successfully. Most participants moved through the flow in about 45 to 70 seconds, which lined up with what the researcher saw in the usability study sessions. The task took a little effort, but most folks experienced very little friction. Except for one participant. (Cue the ominous music. 🎵 Dun Dun Dun 🎵) You know the one. The participant who flat-out got stuck had to reread the content, misclicked a lot, ended up backtracking, and eventually recovered to complete the task, but barely. In this case, they completed the task, but it took them 4 minutes.

After the study, the researcher did what a lot of people are taught to do. They put the task times into a spreadsheet and calculated the average. The spreadsheet gave them a number. The number looked official enough to go into a report. But this earlier-career UX researcher knew that the final calculation didn’t represent what they had seen with their own eyes. At the same time, she knew the long task time from that one participant who got stuck clearly mattered. It was not random noise. It was one of the most useful moments in the study because it showed a real failure path in the design.

These misguided UX pros believe that if an issue only shows up once or twice, it holds less weight. Even worse, I have seen instances where an outlier like this would be removed from the dataset to make the report look more “clean.” This is a huge problem. It is risky and flat-out unethical.

Back to the story. So the earlier-career UX researcher came to me with a question that was basically:

I know this average is technically right, but I don’t think it represents what happened in the study.

That question is why I chose to write about this topic today, specifically, time-on-task calculations with small sample sizes. In this case, the average was pulled upward by that one participant who experienced a serious usability issue. That is the problem with using the mean as the default calculation method here.

For this kind of study, I would usually start with the median. The median gives a clearer read on the typical observed experience. Then I would report the slower case separately and explain what happened, because that struggle is still meaningful evidence.

But I know what you are thinking. All that leads us to an even more foundational question. Why would we want to measure time-on-task with a small sample size anyway? Let’s talk about that a little bit first.

Time-on-Task with Small Samples

Some UX professionals hear “small sample size” and immediately assume time-on-task is not worth calculating. I 100% understand where that comes from.

If we are trying to estimate the exact average task time for an entire user population, 8 participants is a very small number. That kind of claim needs a larger sample, cleaner instrumentation, and a much stronger quantitative design. So yes, there is a real limitation here. The mistake is treating that limitation as a reason to throw out the task time data entirely.

In a small moderated usability study, time-on-task can still tell us something very very useful. It can show whether most participants moved through the task in roughly the same amount of time. It can reveal whether one or two participants had a very different experience. It can help us separate a generally slow task from a mostly smooth task with one serious failure path. The problem happens when people are confused about the kind of claim we are making.

For example, with a small sample, I would not say:

The average task time for this workflow is 78 seconds.

I would say something more like:

In this study, the median task time was 56 seconds. Most participants completed the task in about a minute. One participant took 4 minutes after misinterpreting the account-selection step, which points to a real usability issue we should address.

That second version is much more honest and will be less likely to be misinterpreted by stakeholders. What it doesn’t do is pretend 8 people can give us a precise population estimate. It also does not ignore useful behavioral evidence just because the sample is small. Win/win. I hope you are starting to see my argument here.

That is the basic argument of this article. When UX researchers calculate time-on-task with small sample sizes, which we should be doing, the median should usually be the default starting point, and the reason starts with the shape of the data.

Skewed Time-on-Task Data

Time-on-task data almost never behaves like the tidy examples we learned in statistics classes.

Lower Limits - A user task has a practical lower limit. The user has to read, scan, decide, click, wait for the interface, and move through the flow. Even in a beautifully designed experience, there is some minimum amount of time the task is going to take.

Upper Limits -The upper limit is much messier. A user can take a long time because they misunderstand the task. They can miss a label. They can hesitate because two options sound too similar. They can reread the instructions three times. They can get interrupted. They can hit a slow-loading page. They can accidentally go down the wrong path and recover. They can also run into a real usability problem that nobody on the team realized was there.

That creates a pattern we see all the time in UX research, where most people cluster somewhere near the lower end, while a few slower cases stretch the distribution upward.

Here is a simple example from an 8 participant usability test:

But what if we only show the task times? Then the table looks like a small dataset. Once we add what happened, it starts looking like UX research again.

Now compare the basic summary statistics:

Both the median and the mean are mathematically correct. The issue is that they answer different questions.

Median - Tells us that the typical participant, in this study, completed the task in about a minute. That lines up with the behavior we saw across most usability test sessions. (Note the words “in this study.”)

Mean - Tells us that the average task time was 78 seconds in this study. That number is higher because one participant had a much slower experience.

That slower experience might be the most important finding in the whole study. We should not ignore it, hide it, or treat it like an inconvenience just because it makes the numbers messy. At the same time, it should not quietly redefine the typical experience for the other seven participants.

This is the core reason the median is usually a better default for time-on-task. It answers the question UX researchers usually care about in a moderated study; what was the middle experience?

The mean makes all observed task times and spreads them evenly across everyone. That can be useful in some contexts, especially when you are working with large-scale behavioral analytics or trying to estimate total time burden across a large population. In a small moderated study, though, it is usually not the best starting point.

Most of the time, we are trying to answer questions like:

• Did most users move through the task smoothly?

• Was the task generally fast but still fragile?

• Did one severe confusion point create a long delay?

• Was the experience workable for most people?

• Was there a risky failure path for users who misunderstood one key label?

But the median gives us a better first read on that typical observed experience. It does not explain everything. It does not replace watching the sessions. It does not tell us why someone struggled. It gives us a less fragile summary of what happened, a nd in small-sample UX research, less fragile is usually a very good thing.

Report Median

The median should usually be the starting point for time-on-task, and it should travel with the surrounding evidence. But a median by itself can still hide too much.

If all we say is “the median task time was 56 seconds,” we have improved the statistic, and we still have not explained the experience. We have a better summary number. We still need the behavior that gives the number meaning.

For small-sample usability studies, I would usually report time-on-task with a few pieces of context:

The goal here is pretty simple. We want the number and the observation to support each other.

A stronger write-up might look like this:

Median task time was 56 seconds. Observed times ranged from 42 seconds to 240 seconds. 6 of 8 participants completed the task in under 70 seconds. The longest task time came from one participant who misunderstood the account-selection label and opened the wrong account panel before recovering.

That gives the reader the summary, the range, and the user behavior. It also helps prevent a common reporting problem. Time-on-task gets treated like a standalone performance score, floating outside the rest of the study. Someone sees the number and assumes the number is the finding. Time-on-task rarely explains itself.

A slow task time tells us that one participant took longer than the others, and that’s it. Analyzing the full usability test session tells us everything else that happened. The research synthesis tells us whether that behavior looks like a minor hesitation, a recoverable issue, or a serious failure path.

This is why I like writing the findings in a way that includes both the statistics and the behavior, like this:

Median task time was 56 seconds. Most participants completed the task in about a minute. One participant took 240 seconds after misinterpreting the account-selection step. This suggests the flow may be efficient for users who understand the account structure, with a clear failure path for users managing multiple accounts.

That version gives stakeholders something they can use in the real world. They can see the typical task time, the slow cases, and the design issue all in one view. This gives you the mechanism needed to ensure your stakeholders understand why the outlier participant matters. Your report should make the shape of the data visible. For very small samples, I often think the simplest option is the best one, show the individual task times.

Something like this is easy to understand:

I hope you can see how that table does a lot of work. It shows the cluster. It shows the extreme case. It shows why the extreme case happened.

The same idea applies when talking through the readout. Stakeholders often ask for averages because averages are familiar. That is fine. We can answer the real question behind the question.

A simple explanation might sound like this:

I’m using the median because time-on-task data is usually skewed, especially with small samples. In this study, most people completed the task in about a minute. One person took much longer because they misunderstood the account-selection step, so I’m reporting that separately as an important usability issue.

In my experience, that explanation is usually enough. It avoids making the conversation about statistics for the sake of statistics. It brings the conversation back to the user behavior. That is where the research value is.

Mean Is Still Useful

If you are working with a large analytics dataset, the mean may help answer questions about the total time burden. If a task is completed hundreds of thousands of times per month, long task times may represent real operational cost.

For example, imagine a repetitive common workflow that gets completed 500,000 times per month. If one version of the flow adds 20 seconds on average, that could represent a meaningful amount of user time across the whole system. It could also increase abandonment, support contacts, frustration, or downstream errors.

In that context, the mean might help answer questions like:

Those are valid questions. They are also different from the questions we usually ask in a small moderated usability study.

In a small study, we are usually trying to understand how the experience behaved for the people we observed. We want to know where users moved smoothly, where they hesitated, where the design created confusion, and what moments created delays. For that work, the median is usually the better starting point.

A helpful sequence is:

1. Start with the median.

2. Show the range.

3. Explain the slowest cases.

4. Decide whether the mean adds anything useful.

5. Use the mean only when it answers a specific research or business question.

It also helps avoid a common readout problem where one average becomes the center of the conversation. Once that happens, people start debating the number instead of the experience. They ask whether the task was “fast enough.” They compare it to some remembered number from another study. They want to know whether 78 seconds is good or bad.

Trust me on this, the better conversation is much more specific. Most participants completed the task in about a minute. One participant took four minutes because the account-selection label sent them down the wrong path. The risk is not general slowness across the whole task. The risk is a specific failure path for users who interpret that label differently than the team intended.

That is a much more useful product conversation. The mean may still be worth including in an appendix or supporting detail. I would just avoid making it the headline metric unless it directly supports the decision being made. For small-sample usability studies, the headline should usually be the median plus the observed behavior.

Match the Data

A lot of UX research gets into trouble when we make the analysis look more precise than the study can actually support. Time-on-task is one of the places I see this error in thinking a lot!

For small-sample time-on-task analysis, the practical approach is pretty simple:

• Use the median as the primary summary.

• Show the range.

• Show the individual task times when the sample is small.

• Explain the slowest cases.

• Be careful with p-values when the sample is tiny.

• Use non-parametric methods when comparing groups and the data is clearly skewed.

• Consider bootstrapping when you need an uncertainty interval around a statistic like the median.

• Consider transformations for highly skewed timing data when you are doing more formal modeling, and explain the results in plain language.

Most UX researchers do not need to turn every time-on-task report into a methods appendix. The practical standard is much simpler.

Use a method that fits what you actually collected. If you have eight task times and one of them is 240 seconds, show that shape. Do not hide it inside a clean-looking average. If you have a small sample and a skewed distribution, do not write as though you have a stable estimate of normal behavior across the entire user population.

That distinction matters because stakeholders often want numbers to carry more certainty than they deserve. A number in a report can feel settled. It can feel objective. It can feel like the debate is over.

UX researchers have to be careful with that power. A good time-on-task write-up should make the strength of the evidence visible. It should help people understand what happened, how often it happened in the study, and how much confidence we should place in the pattern.

Here is a practical reporting pattern I would trust:

Median task time was 56 seconds across eight participants. Seven participants completed the task between 42 and 67 seconds. One participant took 240 seconds after misinterpreting the account-selection step and navigating into the wrong account panel. Because the task-time data was skewed, the median is a better summary of the typical observed experience than the mean. The slowest case is reported separately because it reflects a meaningful usability issue. Additional quantitative research should be done if you want to better understand the actual time-on-task for the entire user population.

That paragraph does several useful things:

• It gives the primary metric.

• It shows the spread.

• It names the sample size.

• It explains the skew.

• It preserves the slow case.

• It connects the slow case to a design issue.

• It clarifies that this does not represent the whole user base.

• It points to the need for additional quant research.

That is the kind of reporting that helps teams make better decisions.

Conclusion

Time-on-task is easy to calculate and easy to misuse. That is probably why this problem shows up so often. Averages feel familiar, fit nicely in slides, and make messy behavior look clean. They give teams a number they can compare, track, and, quite frankly, stakeholders are suckers for them, which is why many PMs weaponize this kind of data for manipulation purposes. We need to be the antidote to this trend. We, as UX researchers, need to be the data experts and truth tellers within our organizations.

For example, when we collect time-on-task in a small usability study, we are measuring people trying to complete a task inside a real interface. Their time reflects comprehension, hesitation, confidence, confusion, recovery, and sometimes failure.

The median helps us summarize the typical observed experience without letting one extreme case dominate the headline number.

• The range helps us show the spread.

• The individual task times help us keep the data honest.

• The observed behavior helps us explain what the numbers mean.

Together, these give teams a better read on the experience than a single average ever could. So the next time someone asks for the average time-on-task, I would still answer the question behind the question. Thanks for reading, and I hope this is helpful!

I recently wrote a document to help define a reusable set of UX best practices for product onboarding and in-application help. This week’s post is a generalized version of that document that you can use in your own day-to-day work.

My goal was to make help content decisions more strategic across products, platforms, and teams, so this guidance is intentionally technology- and domain-agnostic. Everything in this week’s post focuses on widely accepted usability best practices, helping users complete real world tasks, and giving users guidance at the moment they actually need it.

And I think this matters because a lot of UXers and product teams don’t seem to know about these best practices anymore. Products like Pendo make it easy to add UI elements into apps, and I worry that many UXers have started favoring these quick-win methods over foundational user-centered principles.

If you need a starting point for discussion with your team, I encourage you to share this post. My hope is this info is especially useful for UXers trying to create more consistent guidance across teams without turning every onboarding or help experience into a maze of popups, tours, and misunderstood help widgets.

Core Strategy Principles

1. Keep guidance contextual and just-in-time. When possible, information should appear at the moment a user needs it to make the next decision or take the next action. Avoid front-loading instruction that users must remember for later interactions and decisions.

2. Require only what is necessary up front. Use step-by-step onboarding only for information the system genuinely needs in order to function, or for required security, compliance, or activation steps. Everything else should be deferred until it becomes relevant and displayed just-in-time in the workflow.

3. Make the next action explicit. Each step should make it obvious what the user is expected to do next. The goal is not the fewest possible words. The goal is the minimum amount of text needed to make the next action or decision clear and explicit.

4. Show only what is relevant to the current user and context. Where possible, the experience should adapt based on role, permissions, product access, workflow stage, and device context. Users should not be exposed to features, instructions, or decisions that do not apply to them.

5. Prevent errors and make recovery easy. Onboarding and self-serve setup or data management should reduce the chance of mistakes through clear structure, validation, and review processes. When errors do occur, the system should make correction options straightforward, visible, and explicit.

6. Keep help inside the workflow whenever possible. The primary support strategy should be built directly into onboarding, self-service configuration, and the product experience itself. It should not depend on separate documentation systems, long text-heavy help panels, or standalone training content.

Help & Onboarding Specifics

• Use wizards selectively. A wizard is appropriate when the task needs a clear sequence of required setup steps. It should not be used to force optional configuration, introduce broad product education, or present every available feature at once.

• Organize around decisions and tasks, not system structure. Each onboarding step should support one main action, one main decision, or one coherent group of closely related inputs.

• Keep progress visible. Required setup should be broken into clearly named steps so users understand where they are, what remains, and what has already been completed.

• Defer nonessential setup. Optional details, secondary preferences, and lower-priority configuration should not be added to upfront onboarding steps. These items should be introduced just-in-time, in context, when they become relevant to the user’s workflow.

• Use progressive disclosure carefully. Show the information most users need directly on the screen. Following the 80/20 Rule, the default view should include the guidance most users need to move forward without extra clicks. Secondary details, exception paths, and less common cases can be placed behind a clearly labeled expand option or another explicit interaction. Do not hide essential guidance by default.

• Design for exceptions without breaking the primary workflow. Even when the underlying onboarding work is not literally linear, the experience should still impose a clear sequence whenever that creates a more understandable path through setup. That sequence should accommodate all valid scenarios without blocking users whose path differs from the common case. (This could be implemented using progressive disclosure in the frontend, supported by conditional logic in the backend.) The default flow should be optimized for the 80/20 path, while less common cases can be handled through branching, progressive disclosure, or guided exceptions inside the overall sequence.

• Support save and return at all cost. Users should be able to pause and resume onboarding without losing work or losing their place when the process is likely to span multiple decisions or sessions.

• Validate in place. Errors and warnings should appear close to the relevant UI control or user action, using plain, user-centered language with explicit guidance on how to correct the issue. However, first look for all possible methods to avoid system errors in the first place. (Allow the system itself to do the bulk of the work.)

• Provide a review point before commitment. Users should have a chance to confirm important choices before activation, submission, or any downstream consequence that is difficult to undo.

• Confirm irreversible actions. Any choice that is irreversible, or that would require manual support intervention to correct, should trigger an explicit confirmation modal that clearly states the consequence.

End-User Onboarding Best Practices

In general, end-user experiences should focus on fast, confident entry into core product use. The product should not assume that new users want a tour, a lesson, or a lengthy profile setup process before they can begin.

• Minimize first-run friction. Only the steps required for access, security, compliance, or immediate product use should appear up front.

• Avoid forcing completion of nonessential profile or preference fields. If information can be added later, it should be deferred and requested just in time, and only when it becomes useful to the user.

• Introduce features at first meaningful use. If a user is about to encounter a task or feature for the first time, contextual instruction may be appropriate when delivered just-in-time.

• Tie guidance directly to the relevant interface components. When a prompt or instructional message appears, it should point to the exact UI control or choice the user is working with and explicitly explain what to do now in context.

• Use the same standards on mobile. A mobile experience should follow the same just-in-time and contextual model. For accessibility and usability reasons, do not rely on hover behavior, hidden affordances, or dense text patterns that are difficult to use on smaller screens.

System Admin Onboarding Best Practices

System admin onboarding should be treated as a guided setup experience, not a general product introduction. These users are usually configuring important parts of the system, making decisions that affect other users, and managing settings that may have downstream consequences. The experience should help them move through required setup tasks with enough structure, context, validation, and review to feel confident before anything is activated or applied.

• Prioritize structured setup over broad education. The setup experience should lead admin users with setup responsibilities through required decisions and setup activities in a deliberate sequence instead of expecting them to learn the system through large amounts of introductory content. Even when the underlying work is not fully linear, the experience should create a clear sequence if one can support all necessary scenarios. That sequence should be designed to work for all such users, while being optimized for the most common (80/20) paths wherever possible.

• Keep the experience focused on the task at hand. Do not expose unrelated features, settings, or reference material during setup or configuration tasks if possible.

• Make system state clear. Admin users should always be able to tell where they are in the setup process, what has been completed, what is in progress, what still needs review, and where action is required. Also consider including additional push notifications to encourage progress where possible.

• Design for confidence before activation. The experience should favor review, preview, validation, and confirmation over workflow speed, especially where configuration choices have impactful downstream effects.

• Support error prevention and correction as part of the workflow. The experience should assume that some information will need to be revised during setup. Editing, revisiting earlier decisions, correcting errors, and continuing forward should feel like part of the intended process and be explicit.

• Use just-in-time embedded help at decision points. Admin users should receive concise guidance where choices are made, especially during setup moments that require interpretation, review, or error correction. This is especially important for bulk setup actions. Guidance should use language framed from the user’s task perspective. The system should not send them away to search through general documentation to understand a required setup decision if possible.

• Respect role and permission context. Different admin user segments should only see the configuration choices, instructions, and follow-up guidance that apply to their responsibilities where possible.

Help & In-App Guidance Best Practices

Help should be treated as part of the product experience, not as a separate system. The goal is to give people enough information to move forward from the screen they are already on, without requiring them to stop, search, and interpret a large external help system.

• Make embedded guidance the default. The first choice should be inline instructions, targeted helper text, well-written empty states, clear labels, and task-level prompts placed within the workflow.

• Write from the user’s point of view. Help text should be framed around what the user is trying to do, what action to take, and what happens next. It should not be written from the system’s perspective.

• Keep required help visible. If most users need a piece of guidance in order to proceed correctly, that guidance should appear on the face of the interface. It should not be hidden behind a click, tooltip, or icon-only control.

• Use progressive disclosure only for secondary help. Expandable content is appropriate for extra explanation, edge cases, definitions, or deeper detail that only a smaller portion of users will need. All uses of progressive disclosure should be accessed via an explicitly labeled UI component.

• Avoid icon-only help patterns. All help-triggering controls should be explicitly labeled. For accessibility and usability reasons, do not rely on question-mark icons or other symbols alone to communicate that useful information is available.

• Do not put essential help in tooltips alone. Tooltips and similar patterns may support supplemental explanation, but they are not strong enough to carry required instructions or critical decision support. Also, keep in mind that hover-only controls are deeply inaccessible in all UI context and inherently unusable on mobile.

• Use empty states as guidance moments. When a screen or area has no content yet, the empty state should explain what belongs there and what the user can do next.

• Use error states as guidance moments. Messages should explain what went wrong, what the user can do now, and whether the issue can be resolved in the product or requires support.

• Treat external help systems as secondary. A separate documentation system may still exist, but it should act as backup reference material. It should not be the primary method for explaining how to complete onboarding or common in-product tasks.

• Design content for reuse across contexts. Help content should be modular enough to be placed near relevant tasks across the product rather than stored only as long, standalone pages.

Example Patterns to Avoid

• Avoid showing the full complexity of the system too early. Sometimes complex products still need sequencing. The right response to complexity is guided structure and contextual disclosure, not exposure of every option at once. See Wizards: Definition and Design Recommendations, UI: Proper Indicators for Hidden Elements, and Design for Optimal Experience.

• Avoid onboarding carousels or introductory tours. Do not present users with a sequence of pop-up slides or feature-overview screens at first login as the primary onboarding method. These patterns usually ask users to remember abstract guidance before they have a real task or decision in front of them. See Carousel Interaction Stats, Onboarding Tutorials vs. Contextual Help and Mobile Tutorials: Wasted Effort or Efficiency Boost?.

  

• Avoid broad first-run walkthroughs that explain the whole interface. Even when step-by-step guidance points at specific UI elements, this pattern is often difficult for users. It can interrupt task flow, appear out of context, and force users to memorize instructions they can’t easily recall or apply when needed. See Onboarding Tutorials vs. Contextual Help, Instructional Overlays and Coach Marks for Mobile Apps, and Help and Documentation (Usability Heuristic #10).

  

• Avoid global “help” entry points as the main solution. A global help button, floating chatbot, generic right-side help pane, or detached knowledge base should not carry the primary burden of teaching the workflow. These patterns can be useful as secondary support, but they often force users to leave the task, search for the right answer, interpret generic content, and then apply it back to the screen they were already using. See Help and Documentation (Usability Heuristic #10) and Onboarding Tutorials vs. Contextual Help.

  

• Avoid non-contextual help panes. A right-side help panel can be useful when it is tied directly to the current screen, task, role, or decision. It becomes much less useful when it acts like a generic documentation drawer that displays broad reference content without knowing what the user is trying to do. Help should reduce interpretation work, not add another layer of searching inside the product. See Help and Documentation (Usability Heuristic #10) and Onboarding Tutorials vs. Contextual Help.

  

• Avoid unlabeled icons and ambiguous controls. If a control is important enough to influence comprehension or decision-making, it should be labeled clearly. Do not hide required guidance behind a “?” icon, icon-only button, hover-only tooltip, or any other symbol that users have to discover, interpret, and activate before they know useful help exists. See Icon Usability and Tooltip Guidelines.

• Avoid putting essential help in tooltips alone. Tooltips and similar patterns may support supplemental explanation, but they are not strong enough to carry required instructions or critical decision support. They are easy to miss, often depend on hover behavior, and can create serious usability and accessibility problems on mobile, touch devices, and keyboard-only workflows. See Tooltip Guidelines, UI: Proper Indicators for Hidden Elements, and Design for Optimal Experience.

  

• Avoid walls of text. Large blocks of explanatory copy are difficult to scan and are usually a sign that the content is not tightly aligned to the decision the user must make. See Plain Language Is for Everyone, Even Experts and F-Shaped Pattern of Reading on the Web: Misunderstood, But Still Relevant (Even on Mobile).

A Pattern That Works

At a high-level, onboarding and help should make the next step clear at the moment it matters. Required setup should be structured and guided. Optional setup should be deferred. Help should be embedded, explicit, role-aware, and written from the user’s point of view.

This approach is especially important for complex, role-based products. The more the product can tailor onboarding and guidance to the user’s role, permissions, and current task, the more intuitive and self-sufficient the digital experience becomes.

UX Strategy for Key Workflows

This section translates the onboarding and in-application help principles into concrete experience direction for high-value workflows across products and platforms. The consistent approach across workflows is to keep the primary screens task-focused, introduce complexity only when it becomes relevant, make the next action explicit, validate in place, provide review before commitment, and keep required help visible within the flow.

Reusable Building Blocks

1. Guided task flows (wizard only when required): Use step-by-step guidance when there is a strict sequence (e.g., review → confirm → submit). Otherwise, keep users on a single primary screen with clear sectioning and strategic use of conditional logic and progressive disclosure.

2. Workflow checklists + system state: A persistent checklist showing what’s complete, what’s blocked, what’s next, and what requires attention (especially for multi-session work).

3. Strong empty states: When there’s nothing to display yet, explain what belongs here and provide one clear call to action (CTA) to get started.

4. Inline validation and guardrails: Validate as users type or select, keep errors near controls, and explain how to fix issues in plain language. Reserve blocking errors for truly blocking constraints.

5. Review/confirm screens for consequential actions: Provide a preview of impacts (what changes, effective timing, downstream consequences) before final submission or activation.

6. Contextual help at decision points: Short helper text, definitions, examples, and “what happens next” messaging embedded in the flow. Use progressive disclosure for edge cases; avoid icon-only help.

7. Save-and-return + drafts: Support long or multi-stakeholder tasks with drafts, autosave, and clear “resume where you left off” entry points.

8. User role- and permission-aware tailoring: Show only actions and guidance relevant to the user’s role, access level, and workflow stage.

9. Notifications: Use in-app and (where applicable) email notifications to drive completion for time-bound or multi-session workflows (missing items, pending approvals, approaching deadlines).

Conclusion

In practice, a lot of teams do not follow these guidelines. Sometimes it is because the product is complex. Sometimes it is because onboarding gets treated like a communication problem, a training problem, or a feature-adoption problem. And sometimes it is because patterns like tours, popups, floating chatbots, generic help panes, and “?” icons feel like easy solutions when a complex workflow is hard to design for across multiple teams.

But the usability and accessibility data gives us a pretty clear reason to reconsider. Required guidance works better when it is visible, contextual, and tied to the task a real user is trying to complete in the real world. Users should not have to remember instructions from five screens ago, hunt through generic help content, or discover hidden guidance before they can move forward.

So if your team is using this post as a starting point, I hope it helps you push for onboarding and help decisions that are more grounded in evidence, more consistent across products, and more respectful of how people actually use software in the real world. Thanks for reading!

Citations

• Baymard Institute, UI: Proper Indicators for Hidden Elements

• Interaction Design Foundation, Design for Optimal Experience

• University of Notre Dame, Carousel Interaction Stats

• Nielsen Norman Group, Onboarding tutorials vs. contextual help

• Nielsen Norman Group, Wizards: Definition and design recommendations

• Nielsen Norman Group, Instructional overlays and coach marks for mobile apps

• Nielsen Norman Group, Help and documentation (Usability Heuristic #10)

• Nielsen Norman Group, Mobile tutorials: Wasted effort or efficiency boost?

• Nielsen Norman Group, Plain language is for everyone, even experts

• Nielsen Norman Group, Tooltip guidelines

• Nielsen Norman Group, Icon usability

• Nielsen Norman Group, F-Shaped Pattern of Reading on the Web: Misunderstood, But Still Relevant (Even on Mobile)

I've noticed a lot of UX research writing here on Substack focuses on methods, quant specific statistics, and researchops process. I write about all that stuff too. But today I wanted to provide a list of recent articles that focus on old-school interaction design foundation.

So if you are newer to this blog and want a practical place to begin, start here. This list gives you a pretty good sense of how I think about interaction design principles in the real world.

Top 8 UI-Focused Reader Picks

1. Reframing Accessibility

Summary: This post argues that accessibility should not be treated like a separate compliance bucket sitting off to the side of usability. In practice, accessibility is often just usability under less forgiving conditions. That framing makes it easier for teams to see why these issues matter before they ship them.

✏️ Note: I keep coming back to this one because it gives people a cleaner way to talk about accessibility with product teams that still treat it like an optional add-on.

2. Make It Look Like What It Does

Summary: This is my plain-language breakdown of affordances and signifiers, and why so many modern interfaces still get them wrong. If a thing is clickable, it should look clickable. If a control does something important, users should not have to guess what will happen next.

✏️ Note: This is one of those posts that tends to make people re-look at their own UI work immediately, which is usually a good sign.

3. Every Icon Needs a Label

Summary: This post gets into one of the easiest UI shortcuts to defend and one of the easiest to misuse. Icons on their own often assume too much, especially in complex software. Labels reduce ambiguity, improve scanning, and make interfaces easier to learn without forcing users to decode symbols first.

✏️ Note: If you’ve ever had to sit through someone defending an unlabeled icon-only interface with complete confidence, this one may feel familiar.

4. The Anatomy of an Input Field (Part 1)

Summary: This post breaks down the input field as an actual interface component instead of a blank box teams barely think about. Labels, helper text, required indicators, defaults, placeholder misuse, field states, all of that stuff matters more than people think.

✏️ Note: A lot of form problems start here. This is one of the more practical posts I’ve written, and probably one of the more reusable ones too.

5. The Anatomy of an Input Field (Part 2)

Summary: Part 2 keeps going and gets deeper into the decisions teams usually make too casually. Error handling, validation timing, edge cases, and all the little ways forms create unnecessary friction tend to show up once you stop treating field design like decoration.

✏️ Note: If Part 1 is the foundation, this is the part where the cracks start to show.

6. The Problem With Split Buttons

Summary: Split buttons look efficient on the surface, but in practice they often create hesitation, accidental actions, or plain old confusion. This post breaks down why that happens and why combining multiple action models inside one control is usually not as elegant as teams think it is.

✏️ Note: I like this one because it focuses on a very specific pattern, but the logic behind it applies to a lot of interface design decisions.

7. Stop Weaponizing Jakob’s Law

Summary: Jakob’s Law gets cited constantly in UX, but a lot of people use it as a conversation-ending slogan instead of a principle that still requires judgment. This post is about what the law actually helps with, where people misuse it, and why blindly copying familiar patterns is not the same as good design.

✏️ Note: This one tends to resonate with anyone who has watched a decent design critique get flattened by someone saying, “users expect it this way,” without much real thought behind it.

8. How To Run A Cognitive Walkthrough

Summary: This is a practical guide to using a cognitive walkthrough as an actual evaluation method, not just as a vague UX theater exercise. It helps teams inspect flows step by step and catch points where users are likely to get stuck, hesitate, misread the interface, or lose confidence.

✏️ Note: This is probably the most directly useful post on the list if you want something you can apply to design reviews right away.

Conclusion

A lot of my writing ends up circling the same basic truth from different angles: small interface decisions are rarely small once real users have to deal with them. A missing label, a vague icon, a bad default, a poorly timed validation rule, or a control that tries to do too much can quietly turn a simple task into something slower, riskier, and more frustrating than it needed to be.

That’s part of why I wanted to make this list narrower than some of the broader roundup posts I’ve done before. This version is really for the people who care about the craft side of interface design. The folks who still think details matter because they do.

If any of these posts helped you think through a design problem, feel free to share them, quote them, I encourage you to share these links. Thank you for reading.

In my 20+ year career in UX, I’ve noticed that a lot of enterprise/complex software does a decent job designing the moments when users take action, but it seems like far less work goes into designing everything that happens after that. Over time, I’ve seen a lot of the real usability burden shift into that post user action space. Users are trying to understand what the system is doing, what happens next, and whether they can trust what they are seeing in that moment.

That’s the topic I want to rant about this week. I believe that in complex software, a lot of the most important UX work starts after the click, because that is the point where users need clear system feedback to stay oriented, make good decisions, and keep the work moving. Let’s dive in a little.

After the Action

A lot of the worst user experiences I have seen were not caused by a button being hard to find, a form being impossible to complete, or a navigation model that made no sense. They happened after all of that stuff. You fill out the form, upload the file, submit the request, assign the ticket, or approve the change, and for one brief second, it feels like the hard part is over. Then the system gives you some tiny piece of feedback like “Submitted” or “In Progress,” and now you are left trying to figure out whether it actually worked or not.

That is the part I think UX teams underestimate. The action itself may have taken three seconds, but the full experience now stretches across time and whatever mental model the user has constructed. And that guessing becomes the majority of the experience, because the user is no longer thinking about the task at hand. Instead, they are too busy trying to stay oriented inside a process that has suddenly become completely opaque. Not good.

Think about all the extra interaction costs we as UXers have caused users of complex systems over time, forcing them to hover over a nondescript UI element, refresh a table, check their email for some kind of update, ping a coworker, and open a second application just to understand whether the first one actually worked. My point is, it does not really matter that the submit button was clear, accessible, and placed exactly where best practices say it should be. The interaction did not end when they clicked it. It just became wayyyyy less visible.

Worse in Enterprise Software

My main argument is that this is more painful in enterprise environments because enterprise workflows usually unfold over multiple touchpoints over time.

I think of these touchpoints dealing with these 4 areas:

1. People

2. Approvals

3. Dependencies

4. Backend Processes

That kind of workflow creates a lot more room for uncertainty if the system does a poor job communicating state.

That is why I do not think this is really a complexity problem, at least not in the way UXers usually talk about it. In my experience, most enterprise users can handle a fair amount of complexity. They can learn terminology, navigate layered workflows, and deal with not-so-standard processes. But what I see wearing users down the most is system status ambiguity. They can work inside a complicated system if that system is explicit about what just happened, what is happening now, and what comes next. What really drains users is a system that looks simple on the surface but keeps forcing them to infer its actual state from vague labels, partial updates, and a lot of silence.

This is also why the good old visibility of system status heuristic still matters. The principle sounds basic until you apply it to a real enterprise workflow, where the “system” is no longer just one screen responding to one click. It is a chain of events that may involve timing, queueing, routing, review, exception handling, and another person picking up the work later. Once you look at it that way, it becomes obvious that a spinner or a tiny success message is often nowhere near enough.

Showing Don’t Tell Status

A lot of interfaces technically show status, and that is exactly why this problem gets missed so often. Anyone can point to a badge, banner, icon, or label and claim the state is visible, therefore it’s communicated to the user. In the narrowest possible sense, that may be technically true, but the leap from “it’s visible” to “it’s communicated well” is a wild one. (It’s also counter to any basic logic.)

For example, a status like “Pending” might be accurate from the system’s point of view and still nearly useless from the user’s point of view, because that single label can hide several completely different realities. It might mean waiting on review, waiting on a sync, waiting on another team, waiting on a nightly process, or sitting in a broken state that nobody has noticed yet. Do any of these hidden ambiguities sound familiar to you?

That is where this starts to become a language problem as much as a status problem.

Enterprise systems often expose backend categories, workflow abstractions, and generic state labels, then act as if the user has been meaningfully informed. I’m looking at you, Workday, hahaha! In reality, the user has often just been handed a term that still requires interpretation, and interpretation is work.

The real question is whether the state it displays helps the user answer what they actually need to know in that moment:

• Did my action work?

• What is happening right now?

• Do I need to do anything else?

• Can I safely leave this screen?

• Is another person responsible now?

• Should I expect this to update in ten seconds, an hour, or tomorrow morning?

Those are the questions users are carrying around, and a lot of enterprise interfaces answer them very poorly, even when they appear to be full of status signals.

A status indicator can satisfy the design team because something is present on the screen, while still failing the user because it does not reduce uncertainty in any meaningful way. In my experience, that is where a lot of enterprise friction actually lives. The interface has not gone fully silent. It has done something even worse. It has spoken just enough to sound informative while still leaving the user with the actual cognitive burden of figuring out what the information means.

The Hidden Cost

When a system does a bad job of communicating system status, the most obvious cost is user confusion. But the bigger cost is the amount of extra user effort that gets layered on top of the workflow just to make the workflow usable. You’ve seen this before, users start refreshing screens, checking neighboring tabs, reopening records, scanning their email for confirmation messages, and asking coworkers whether something is normal. None of that work is the real job. It is orientation work that only exists because the interface failed to make the process legible.

Think about how this plays out in the real world. If a user is unsure whether a request actually went through, they are likely to submit it again. If someone can’t tell whether another team has the next step, they might start following up too early or too often. If a user can’t make sense of an item’s status, they might just create their own spreadsheet to track it manually in parallel. (You get the idea.) Over time, those behaviors stop being exceptions and start becoming the actual workflow.

There is also a trust cost that is harder to measure but very easy to overlook. For example, once users learn that a system’s status indicators are unreliable, they’ll likely stop taking them at face value. And in my experience, that kind of thing will totally change how users interact with the product over time. That is a bad place for enterprise software to land, especially when the work inside that software is high stakes.

Why UX Teams Miss This

Part of the reason this problem survives so often is that it hides in plain sight. UX teams spend a lot of time reviewing entry points into workflows because those are the parts that show up cleanly in mocks, prototypes, and design critiques. Team members can comment on layout, labels, form fields, hierarchy, and button placement with a lot of confidence. But the moment after submission is much messier. It involves time, dependencies, handoffs, conditional logic, and states that may only become visible hours later or in another part of the product. My theory is that this kind of design work is harder to capture in a static review, so it simply gets less consideration and attention.

I also think product teams regularly underestimate how much of their own understanding comes from being too deep in the weeds. The people building the system usually know what happens after the click because they know the architecture and business rules that drove the original workflow. These folks know that a request enters a queue, gets validated overnight, routes to a specific team, or changes state only after another process runs. Because they already know the story behind the workflow, they can look at a status label like “Pending” and mentally fill in the missing detail without noticing that the interface never actually communicated it. And of course, as user advocates, we all know that the user does not have that advantage.

There is a more structural issue underneath this too. I see a lot of orgs treat post-action states like an afterthought, then try to solve the problem later by blindly throwing a handful of UI components at it and calling it done.

The most common culprits I see in the real world are:

• a badge

• a toast

• a status column in a table

• a filter that lets users sort by state

• a tooltip

• a color-coded icon or dot

All of these can be useful, but are often added after the core workflow has already been defined. By that point, the team is not really designing state communication. The result is usually a collection of indicators that reflect technical states or fragments of the process, but do not add up to a coherent explanation of how the work is actually moving. (noise vs. signal.)

The 3 Problem Children

Here are 3 of the most common UI patterns I see in the real world. All are often used as stand-ins for better status design, even though they usually leave users doing far too much interpretation on their own.

1. Progress Indicator Spinners

The spinner is probably the most familiar example of this whole issue because it gives the appearance of feedback without always providing much explanation. For very short waits, that is usually fine. People do not need a full narrative when something takes two seconds. The problem starts when teams use the same pattern for situations that are no longer short, simple, or predictable. At that point, the spinner stops functioning like useful feedback and starts functioning more like a placeholder for missing design. Motion is still happening on the screen, but the user has learned very little about what the system is doing, how long it might take, or what they should do if the wait continues.

2. Toasts

The toast creates a slightly different problem. It is often meant to reassure the user that their action was successful, but in practice, it tends to confirm only the thinnest possible version of success. A message like “Saved” or “Submitted” can be technically true while still leaving the most important questions unanswered. Saved where? Submitted to who? Finished for now or sent into a longer process. Then, of course, the message disappears, the user is left staring at the same screen or a slightly updated one, and the real burden of interpretation begins. That pattern is especially weak in enterprise systems because the action that just occurred is often not the end of the story. It is the beginning of a larger sequence, and a fleeting little confirmation message is nowhere near enough to orient someone inside that sequence. And of course, we have decades of usage data showing that temporary UI elements like toasts are often completely missed by users, so that’s still a thing too.

3. Dashboards

Then there is the dreaded dashboard, the junk drawer for the info your org thinks users need. The logic seems reasonable on the surface. If users need to understand status across time, give them a centralized place to monitor status. Sometimes that does help.

A table full of badges, timestamps, and vague labels does not automatically do that. In a lot of cases, it just relocates the confusion. The user can now see ten things marked “Pending” instead of one, but still has very little help understanding which of those items are healthy, which are stuck, which need intervention, and which are simply waiting their turn.

Better Status Design

Despite what many UXers seem to think, better status design does not require turning every workflow into a verbose tutorial. (And it also does not mean adding a Pendo-style pop-up walkthrough, you lazy bums. hahaha.)

That usually means the interface needs to communicate more than a bare state label. It needs to communicate where the work is, what that state means in practical terms, whether anything is expected from the user right now, and what kind of change they should be watching for next.

This all sounds obvious when stated plainly, but a lot of enterprise products still stop at the thinnest possible layer of feedback. They tell the user that something exists in a state, but not what that state actually means for the work. A better system closes that gap. If something is under review, say that. If another team has the next step, say that. If processing typically takes until the next business day, say that. If the user is done for now and can safely leave, that should be explicit too. The goal is not to stuff the interface with more words. The goal is to reduce interpretation work.

This is also where good status design starts to feel more usable. When the system is clear about all of this, users stop floundering. You know the status design is working when users can move on and come back later because the interface has given them a usable explanation of the current situation. That is wayyyyyy more valuable than any generic success message.

A lot of the strongest examples I have seen do this by clearly separating the current state from the next-step meaning. That distinction matters because it is what turns status from a technical label into something a user can actually work with.

❌ Bad: “Submitted”
✔️ Better: “Submitted for manager review”

❌ Bad: “Processing”
✔️ Better: “Processing. No action needed. Expected completion by tomorrow morning.”

❌ Bad: “Pending”
✔️ Better: “Pending legal approval”

The better versions do more than report a state. They help the user understand what is happening, who or what owns the next step, and whether they need to do anything right now.

The other thing better systems do well is make status legible across time instead of only in the moment.

❌ Bad: “Saved” shown only in a disappearing toast
✔️ Better: “Changes saved at 2:14 PM” shown in a stable, visible location

❌ Bad: a vague status badge with no history
✔️ Better: a clear status label paired with a timestamp or status history

❌ Bad: a table update that gives no clue what changed or when
✔️ Better: a visible update that shows what changed, when it changed, and what happens next

A user should not get one brief chance to understand what happened and then lose the thread as soon as the screen refreshes, the toast disappears, or the table updates. If the work continues later, the interface should help them come back and quickly figure out where things stand.

The Bigger HCI Point

What makes this topic interesting to me is that it looks like a small interface issue until you sit with it for a while. On the surface, it can seem like a complaint about wording, spinners, or whether a status badge should be blue or green. But the deeper issue is that enterprise software often fails to help users build an accurate mental model of work that is unfolding across time. That is a much bigger design problem than most teams acknowledge.

A lot of UX conversations still treat interaction as something that happens in a neat little burst between the user and the screen. The person clicks, the interface responds, and the experience is more or less contained inside that exchange. That model breaks down pretty quickly in enterprise environments because the work is often distributed. A request can move through systems, rules, queues, reviewers, exception states, and downstream teams long after the user has stopped actively touching the interface. If the design only accounts for the visible moment of interaction, then it leaves the user in the dark during the part of the workflow where understanding matters most.

That is why I think poor status communication creates so much unnecessary cognitive load. It forces users to simulate the system in their own heads. They have to infer what happened from partial clues, remember how similar cases behaved last time, and guess which delays are normal and which ones signal a problem.

In complex environments, that compensation can start to look like expertise. (This is the power-user fallacy.) But in reality, it is often just an adaptation to a system that never explained itself clearly enough in the first place.

This is also one of those places where usability and organizational clarity run directly into each other. When software communicates status and state poorly, it does not just confuse an individual user. It makes it harder to tell whether work is moving, waiting, stuck, or owned by someone else. In other words, the interface stops being a useful representation of the process. And that is where I have seen a huge amount of waste happen, all while UX teams are still debating small UI details in the hope that they will fix the core problem. (Spoiler, they won’t, no matter how good the UI is.)

Conclusion

To wrap things up, I’ve noticed for a long time that in complex and enterprise software, a lot of the most important UX work starts after the user takes action. That is the point where people need the system to clearly communicate what is happening, what comes next, and whether the work is moving the way it should.

Here’s an idea. Instead of leaving users to compensate, UX teams can design system feedback that does more real work. We can use clearer status labels, make ownership and next steps explicit, show when no action is needed, add timestamps or status history that help users reorient later, and explain what the system is actually waiting on. That kind of status design reduces interpretation work, makes workflows easier to follow across time, and gives users something they can actually trust in the real world. (e.g., basic old-school UX principles.)

Have you seen this kind of system-status confusion play out in the complex tools your team uses or builds? If you have, comment here or send me a DM. I’d be interested to hear where you’ve seen ambiguous system feedback create extra work and workaround behavior. Thanks for the continued support and for reading this week’s article.

I’m in the middle of facilitating a service blueprint creation workshop at work, and it has had me thinking about a pattern I have seen in more than one past role. Does this sound familiar to you? A product has grown large enough that people across the org no longer share the same understanding of how the experience is actually delivered to end users because teams are somewhat siloed. One team understands the interface. Another understands the policy constraints. Another knows the manual steps happening behind the scenes. The Support team may know where users get stuck, while engineering knows which systems are patched together just well enough to keep the whole thing moving. At that point, the org does not just need design ideas. It needs a reliable picture of how the service currently works holistically.

In several past orgs, I watched that exact need get answered with a journey-mapping exercise. Stakeholders wanted the workshop version of user understanding. They wanted a Post-it wall full of stages, pain points, emotional states, and sticky notes that translated internal beliefs into something that looked user-centered. The result often felt productive in the moment. It gave people something visible to react to and something easy to circulate afterward. What it usually did not give them was a more accurate look into the root causes behind a disjointed user experience.

Service blueprints and journey maps are completely different tools, yet they are often conflated in UX circles. Nielsen Norman Group has a useful way of defining service blueprints. (NOTE: I’ll be using this definition and framework throughout this article.) They describe them as tools for mapping the relationships between service components, including people, processes, and touchpoints, and note that they are especially useful for cross-functional, multi touchpoint experiences. That is often the kind of understanding enterprise teams are missing. They do not need a workshop that helps everyone imagine what the user is thinking. They need something much more simple and straightforward. They need a way to document what the org is currently doing to create the experience users are having.

Journey maps can give teams the feeling that they have done that work when they have not. That gap between the feeling of learning and the presence of actual learning is a big part of why I have grown more skeptical of them over time.

The Problem

This is not a general comparison article about journey maps and service blueprints. (Many, many people have already written that article.) It is also not an article about how to run a better journey-mapping session. (That horse has already been beaten enough as well. hahaha.) I am not trying to rescue any methods or explain the ideal version of how they are supposed to work. Instead, I want to tell the real-world truth about journey maps and service blueprints.

I want to stay focused on how journey maps are often used in real orgs. In practice, they are frequently done incorrectly, and they are easier to mess up than many UX pros seem comfortable admitting. That becomes even more important in larger enterprise environments, where the system itself is already complicated and where polished internal artifacts can carry more authority than they deserve.

Service blueprints do answer a different question. Nielsen Norman Group makes that distinction clearly. Journey maps focus on understanding the end user’s journey, including actions, thoughts, and emotions, while service blueprints document how the org creates that experience through frontstage actions, backstage actions, and support processes. I agree with that distinction. I also think it helps explain why service blueprints are often the more useful strategic move in practice.

In my experience, orgs are not facing a shortage of made-up user emotions, but they are facing a shortage of shared operational understanding. Most orgs need to know who is doing what, where the process breaks down, which teams influence the experience, and where additional research is actually needed. A service blueprint gives them a more grounded way to start that work.

That is the argument I want to make in this week’s post. Journey maps are often easier to misuse. Service blueprints are more objective in nature. In large orgs with complicated products and layered service models, that difference starts to matter very quickly.

Stop Weaponizing Journey Maps

Journey maps are one of the easiest UX artifacts to turn into “UX Theater” because they let teams simulate user-centeredness without doing the real work of learning from actual users. And of course, the design thinking workshop format helps create that problem. A group gets together, starts filling in what they think users think and feel, and before long, the output begins to look like insight, whether or not insight is actually present. This is the standard critique that I’m sure most of you have heard many times.

That visual credibility carries a lot of weight inside orgs. A journey map tends to look substantial. It is detailed, well-structured, and full of language that sounds user-centered. Once it is on the wall or turned into a polished deliverable, people can start treating it like research simply because it resembles the kinds of artifacts research teams produce. The format does a lot of persuasive work on its own.

The first reason this happens is that journey maps invite workshop participants to make up how they think the typical user behaves without actual user data. Once a workshop starts assigning motivations, frustrations, and expectations to users who were never studied, the group is no longer synthesizing evidence. It turns into a baseless projection exercise filled with bias.

A second problem is that the artifact creates confidence too easily. Stakeholders leave the session feeling aligned. Teams feel like progress was made. Leadership sees something concrete that appears to capture the user experience. Those reactions are understandable, but they are not proof that the artifact is accurate. A workshop can create momentum and still produce a distorted view of reality.

Journey maps also tend to mix different categories of information together until the boundaries disappear. A real process fact can sit next to a stakeholder’s guess. A present-state observation can sit next to a future-state aspiration. An imagined user pain point can get written in the same format as a finding that actually came from research. Once all of that is arranged into a single visual system, the weaker claims borrow credibility from the stronger ones.

This becomes more dangerous in enterprise settings because the experience is shaped by a lot of things that do not fit neatly into a simplified user story. Internal policies shape what can happen. Support workflows affect resolution time. Permissions, exceptions, legacy systems, and manual workarounds all influence what the user experiences. When a team tries to compress that kind of environment into a neat narrative about what the user is thinking or feeling, the result can be very misleading while still sounding strategic.

That is why I use the word weaponized. I do not mean that every journey map is made in bad faith. I mean that the artifact can be used to create the appearance of user understanding, reassure stakeholders that the team is being user-centered, and move decisions forward without doing the harder work of gathering real evidence. At that point, the journey map is no longer functioning as a research artifact. It is functioning as a persuasion artifact.

Service Blueprints Are Honest

The distinction between service blueprints and journey maps matters a lot because the organizational side of the experience is usually where the end user confusion is derived from.

A service blueprint gives you a way to document those handoff thresholds without pretending to know more than you know about the user.

Nielsen Norman Group also points out that service blueprints are especially useful for experiences that are omnichannel, involve multiple touchpoints, or require cross-functional coordination. That description fits a huge amount of enterprise work. It also helps explain why service blueprints tend to hold up better in those environments. They are built to make organizational complexity visible.

That does not make service blueprints perfect. They can still be incomplete and executed in a sloppy way. But, they often still unearth organizational blind spots even if the wrong people are in the room or the wrong inputs are used. And unlike journey mapping, what they are less likely to do is drift into fake empathy-land. The fundamental structure of service blueprints keeps the team closer to observable reality. They ask what happens, who does it, what supports it, and where things break down. That is a much safer place to start than a workshop built around simulated user feelings.

This is why I keep coming back to the idea that service blueprints are often the more honest strategic move. They do not claim to give you a deep understanding of the user. They just thoroughly document how the end-to-end service actually works in the real world. In a lot of orgs, that is the thing people need before they are ready to make bigger claims about the user experience.

Document Then Interpret

A lot of teams skip past the documentation step. They start interpreting the experience before they have documented the service that creates it. That mixed-up order of operation causes problems because user experiences do not emerge out of thin air.

They are shaped by how the org operates in very specific ways, including things like:

• users getting inconsistent information from different teams or channels

• processes feeling disjointed because work moves from one department to another without a clean transition

• tasks taking longer than they should because approvals, reviews, or manual steps slow them down

• duplicated work happening across teams

• unclear ownership over who is responsible for fixing a problem

• workarounds people rely on because the official process does not work well

• approval flows that add friction or delay

• support processes that never show up in a simplified user story

Service blueprints are useful because they force teams to look at those underlying structures directly. Nielsen Norman Group describes blueprinting as a way to discover weaknesses, eliminate redundancy, and coordinate future changes by helping orgs visualize who is doing what and when. That is often the work that needs to happen first. Before a team starts speculating about emotion, it needs to understand the machinery behind the experience.

That is also why service blueprints tend to be more productive, especially in enterprise environments. They give teams a shared picture of the service as it currently operates. They show where departments intersect, where internal work supports user-facing work, and where breakdowns are happening across the system.

That kind of clarity is a big deal. A lot of enterprise problems survive because each team understands only its own slice of the work. The blueprint does not magically solve that, but it gives people a more grounded way to talk about it. It puts the service itself on the table instead of letting the conversation drift into assumptions about the user.

I think that is one reason service blueprints feel less theatrical to me. They are still workshops. They still involve collaboration. They still depend on facilitation. The difference is that the output is tied more directly to what the org is actually doing. That makes the conversation more disciplined. It also makes the next steps more useful.

That is a much cleaner process than gathering stakeholders in a room and asking them to fill in what they imagine users are thinking.

It also respects the limits of the artifact. A service blueprint does not need to overclaim in order to be useful. It can simply show what is happening, where support structures are helping or failing, and where more evidence is needed. That kind of restraint is valuable. In my experience, teams make better decisions when the artifact in front of them is honest about what it knows and honest about what it does not.

Alignment Is Not Evidence

One of the reasons journey maps keep surviving in this form is that they do produce something people genuinely want. They can create stakeholder alignment, a shared language, and make it easier for a room full of people to discuss the same flow together. I understand the appeal. I understand why teams walk out of those sessions feeling like useful work got done.

I think this is where a lot of UX pros get themselves into trouble. They confuse the visible outputs of a workshop with the quality of the knowledge behind it. Don’t get me wrong, a well-facilitated session, a collaborative artifact, and a room full of engaged stakeholders can be useful. Journey mapping workshops are useful as organizational internal alignment tools, but they are not useful for better understanding real users.

That distinction is worth holding onto because it protects the discipline from drifting into performance. If the artifact makes claims about users, those claims should be grounded in user evidence. If the artifact is documenting how the org works, it should be grounded in operational evidence.

This is also why I have less patience now for the softer defenses of these workshops. People will sometimes say that the point is not accuracy. The point is conversation. Or alignment. Or ideation. I understand what they mean, but that defense falls apart pretty quickly once the artifact starts getting circulated as if it says something true about the user experience.

In real orgs, these things rarely stay contained as innocent workshop exercises. They become reference points that influence priorities. They shape how leaders talk about user problems and become part of the decision-making environment, whether they deserve that role or not. A service blueprint is not immune to misuse, but it is less likely to create that same kind of false legitimacy as a journey map because it is anchored in a narrower and more honest claim.

It is trying to accurately document how the service works. That is a much more defensible job for a collaborative workshop artifact. It stays closer to what a room of stakeholders can actually know, especially when the right people are involved.

Sometimes that means doing direct user research. Sometimes that means documenting the service well enough to know where direct user research is needed next. Both are legitimate. Pretending to know what users think because a workshop produced a good-looking map is not.

Conclusion

The confusion between service blueprints and journey maps has been around for a long time, and I do not think it is going away anytime soon. Journey maps look strategic. They look user-centered. They give people something visible to point at. They give teams a fast way to feel aligned. All of that makes them easy to overvalue, especially in environments where the underlying service is complicated and no one fully understands how all the pieces fit together. That is exactly why I trust them less.

Service blueprints are not doing the same job, and I am not claiming they should replace every other artifact in UX. I am saying something much simpler. In a lot of enterprise settings, they are the more honest place to start. They stay closer to observable reality. They make operational complexity visible. They help teams document what is actually happening before they start making bigger claims about what users think or feel.

If a team wants to understand user journeys, it should study its own systems first.

Have you used service blueprints to bring more clarity to a messy product, service, or internal workflow? If so, comment here or DM me. I’d love to hear where you’ve seen service blueprints help teams build shared understanding, uncover operational gaps, or make smarter decisions in complex environments. Thanks for reading!

In the meantime, I found a missing brick in this wall...

Boom, UX!

A while back, I was watching a researcher try to get a pretty normal study out the door. Nothing fancy. A handful of participants, a few stakeholder observers, some straightforward user interviews, and a plan to turn the findings around quickly.

This researcher pulled participant information from one system, cleaned it in Excel, scheduled sessions through a third-party tool, hosted the calls in a different platform, tracked incentives in a spreadsheet, downloaded recordings manually, used an LLM to create transcripts, and then started trying to stitch everything back together for analysis. By the time this person got to synthesis, the study already felt fragmented. Not because the research was bad, but because the workflow around it introduced a lot of noise. Sound familiar to anyone else?

That’s when I started thinking that a lot of research pain is not caused by a lack of methods, talent, or rigor. It comes from broken hand-offs between steps within a process.

The Problem

When I tell others this story, many peeps start solutionizing in their heads, and it is usually about how to reduce the number of tools used in the process. My hot take is that I don’t think most UXR teams are suffering because they use too many tools in some abstract sense. In practice, most orgs are always going to have some sort of tech stack. Their email, calendar, and meeting systems have been chosen for them. General customer data probably lives in some CRM that the research team does not control. The problem is not that the workflow spans multiple systems. The problem is that the systems do not behave like a single workflow. It’s the hand-offs between systems that’s the real problem.

Any well-designed user research process needs to work across several tools and still feel coherent in the transitions. But when every transition creates more admin work, more duplication, or more opportunities for something to break, the researcher ends up doing tool-to-tool busywork instead of research.

Recruitment & Outreach

Recruitment is usually where the cracks in a research process tend to show first. Participant data often lives in CRMs, customer databases, spreadsheets, support systems, or a mix of all of them. That part is normal. What is not normal is how much manual effort researchers are often expected to spend just getting that information into usable shape. Exporting files, cleaning fields, checking for duplicates, verifying preferences, re-uploading lists, and trying not to mishandle sensitive information should not be the hidden cost of running a study.

This feels more like admin work wrapped around research than valuable research work in and of itself. With a more strategic process, participant data can come from where it already lives, stay reasonably up to date, and be managed without someone rebuilding the pool every time a new study starts.

And of course, the same thing shows up in outreach. Participants are much more likely to trust and respond to an invitation when it feels legitimate and connected to a real person. We all know that instinctively because we have all ignored emails that looked vague, generic, or slightly off. Yet research teams still end up using awkward workarounds to make outreach happen, especially when the recruitment system and the communication system do not play nicely together. That creates unnecessary friction on both sides. The participant gets less trustworthy communications, and the researcher gets more coordination work. Neither of those helps the study.

And in a world in which users are less likely to engage than ever, even small signs of friction or illegitimacy in the outreach process can lower response rates before the research even has a chance to start.

Scheduling & Incentives

Another administrative task that eats away at our time is scheduling research. Real research scheduling is rarely as simple as finding an open hour on a calendar.

In my experience, you’ll likely be juggling these types of constraints:

• coordinating a participant, a moderator, a note taker, and multiple observers at the same time

• limiting how many sessions can happen in a single day

• building in buffers between interviews so moderators can reset

• setting notice periods so nobody wakes up to a surprise session the next morning

• protecting stakeholders from booking themselves into parts of the process they should only observe

These things aren’t rare edge cases. This is what real research operations look like. When teams try to force research scheduling into generic workflows that were not built for those realities, they end up compensating with manual effort. That manual effort may look small in isolation, but across multiple studies it adds up fast.

This is the same with incentives. Incentives are one of those things that sound simple until you are the person managing them. Then suddenly you are checking attendance, verifying completion, making sure the right person gets the right amount, tracking who has already been sent something, and following up on exceptions. None of it is intellectually difficult, but all of it takes attention, and it creates yet another opportunity for error in a workflow that already has too many of them.

That matters because research quality is not only about how well you ask questions or how carefully you synthesize patterns. It is also about whether the operational side is stable enough that the researcher can stay focused on the study itself. Every extra tracking step pulls attention away from the work that actually requires judgment.

Method Flexibility

Most mature research teams use a mix of methods. In my opinion, that’s how it should be. Interviews, surveys, usability tests, diary studies, card sorts, and other methods all serve different purposes. The issue is not whether a team uses multiple methods. The issue is whether switching methods breaks the continuity of the study, and that is the part I see even experienced UXR’s miss.

Can the researcher keep the participant details, invites, reminders, session logistics, observers, recordings, and follow-up connected as they move between approaches? Or does every method change force the team to rebuild the operational side from scratch?

I’ve seen a lot of teams quietly accept that rebuilding is normal. In my experience, that is one of the clearest signals that the workflow is not serving the research.

Chains of Evidence Have Momentum

Recording and transcription are other places where this shows up in very obvious ways. A surprising number of teams still move evidence through a chain of disconnected steps. Record here. Download there. Upload somewhere else. Wait for a transcript. Move it again. Tag it later. Clip the useful moments once enough time has passed that half the context is already fading. Technically, that works, but in practice, it slows synthesis down and creates distance between the researcher and the evidence.

When notes, recordings, transcripts, highlights, and clips stay connected, the study remains alive while it is happening. Researchers can start noticing patterns earlier. They can pull supporting moments while the interviews are still fresh. They can move more naturally from observation to synthesis without having to perform three rounds of file management first.

That continuity matters because momentum matters. And this is the thing we researchers bring to the table. The ability to hold all of this user-centered context in our heads as we learn is what will be the key differentiator for our discipline moving forward.

If we concentrate on making these hand-off points a way to carry human context through our development processes, we will have the ultimate level of job security.

A Study is not A Single Deliverable

This is the part I care about most and the main point of this week’s article. Too many UXers still act like the final report or readout is the study. Have you ever heard a team, after they get a polished presentation, say, “Great job, and the work effectively ends there? I know I have, and that has never made much sense to me.

A single study usually creates several layers of material, and those layers serve different purposes:

• the source data

• the notes and transcripts

• the tagged moments that start to show patterns

• the individual clips that make a point visible

• the highlight reel that helps tell the story quickly

• the deeper report for people who need the full thinking

• the presentation deck for stakeholders who need the version that travels well

These are different ways of communicating findings from the same study, because we all know that more than half of our job is telling stories with data.

Different stakeholders need different kinds of evidence. Sometimes the raw data tells the story best. Other times, it is the synthesis wrapped in context. And some stakeholders do not really understand the issue until they see a short clip of a participant struggling through the workflow in real time. This is where strategic communication matters most. Good research communication is not about forcing every finding into one format. It is about preserving the connection between these layers so the study can do more than one job.

When those layers get separated from each other, the evidence weakens and the report becomes less reusable. Those powerful clips lose important context, and your readout deck becomes a snapshot in time that feels dated almost immediately after you deliver it. Then the final nail in the coffin is that the raw data disappears into storage, and the team moves on.

But when those layers stay connected, the opposite happens. Research becomes easier to revisit, easier to share, easier to adapt for different audiences, and much more likely to remain useful after the first presentation is over. This is the big idea. The hand-offs are the critical ops moment for strategic UXR workflows.

What Good Looks Like

Okay, so that was the bad news. Most research departments are not struggling because they lack methods, smart researchers, or one more feature in a tool. They are struggling because the process keeps breaking between steps.

Some of those features do matter a lot, but they are not the real standard we should be focusing on. The real standard is much simpler and much more important: does the workflow actually hold together in practice?

A good ResearchOps system is about connecting the building blocks that make up a cradle-to-grave research study. It keeps recruitment connected to outreach. It keeps outreach connected to scheduling. It keeps scheduling connected to sessions. It keeps sessions connected to evidence. It keeps evidence connected to synthesis. And it keeps synthesis connected to all the different outputs a study may need, instead of letting the work collapse into one static artifact that dies in a folder.

That is why I do not think researchers need more software; we just need less process fragmentation and a workflow that keeps the study connected from start to finish. This means everything from the raw data to the synthesized insight to the final stakeholder story. If those things stay tied together, you will inevitably reduce the administrative and cognitive drag, compromising your researchers.

When research turns into a chain of broken hand-offs, teams start losing the things that make the work useful in the first place: clarity, momentum, and time that should have been spent understanding users and helping the org make better decisions. From what I see in the real world, that is often what separates research that gets reused and remembered from research that gets presented once and then fades out over time.

Checklist: How to Fix Broken Hand-Offs

If you want to improve a research process in the real world, do not start by shopping for a new platform. Start by tracing where the study breaks.

The easiest way to do that is to follow one recent project from beginning to end and look for the moments where the researcher had to stop doing research and start managing the process instead. That is usually where the hand-off problem lives.

Here is the checklist I would use:

• Map the full study lifecycle

  • Write out every step from recruitment to final readout.

  • Include where participant data comes from, how outreach happens, how sessions get scheduled, where recordings live, how transcripts get created, how incentives get tracked, and where findings get turned into deliverables.

  • If the team cannot map the process clearly, that is already a sign the workflow is too fragmented.

• Mark every point where information has to move

  • Highlight every export, upload, copy-paste, spreadsheet update, file rename, manual reminder, and hand-entered status check.

  • These moments may look small, but they are often where time gets wasted and errors get introduced.

• Look for duplicate effort

  • Ask where the same information gets recreated in multiple places.

  • Common examples are participant status, calendar details, attendance tracking, incentive logs, transcript files, and study metadata.

  • If the same fact has to be updated in two or three places, the process is fragile.

• Notice where the researcher has to rely on memory

  • If the workflow depends on someone remembering who got paid, which participant rescheduled, where the latest transcript lives, or which clip matches which finding, the process is carrying too much invisible risk.

  • Good systems do not require researchers to hold the workflow together in their heads.

• Find the steps that slow down momentum

  • Ask where the study loses energy.

  • Does it stall between interview and transcript?

  • Does synthesis get delayed because files need to be cleaned up first?

  • Does reporting take longer because the clips, notes, and evidence are no longer connected?

  • Wherever momentum drops, the hand-off probably needs work.

• Separate high-judgment work from low-value admin

  • Be honest about which steps actually require research skills and which ones are just operational maintenance.

  • Moderating an interview requires judgment.

  • Deciding what evidence supports a finding requires judgment.

  • Re-entering attendance into a tracking sheet does not.

  • Broken hand-offs usually show up where too much researcher time is being spent on low-value coordination work.

• Check whether each output stays connected to the evidence

  • Look at the final deliverables and ask whether they still point back to the source material.

  • Can someone move from the deck to the clip, from the clip to the transcript, and from the transcript to the original study context?

  • If not, the study is already starting to decay the moment it gets shared.

• Fix the highest-friction hand-off first

  • Do not try to redesign the whole process at once.

  • Pick the one transition that causes the most wasted time, confusion, or risk.

  • In some teams, the recruitment will be for outreach.

  • In others, it will be sessions to evidence, or evidence to synthesis.

  • Start where the pain is most visible and most repeated.

• Use process fixes before platform fixes

  • Sometimes the answer is a tool change, but often the first improvement is simpler than that.

  • Standardize naming conventions.

  • Create one source of truth for participant status.

  • Define who owns incentive tracking.

  • Set rules for where recordings, transcripts, and clips live.

  • Add a clear hand-off owner for each step so the process does not become everyone’s responsibility and no one’s responsibility at the same time.

• Measure whether the fix actually worked

  • After you change a hand-off, look for real signs of improvement.

  • Did scheduling take less time?

  • Did researchers spend less effort tracking logistics?

  • Did synthesis start sooner?

  • Were findings easier to reuse later?

  • If the fix did not reduce friction in practice, it was not really a fix.

In my experience, this kind of audit is where a lot of teams uncover the truth about their process.

Conclusion

My main point is that a lot of UX research pain does not come from weak methods, underqualified researchers, or a lack of enough software solutions. It comes from the spaces between steps, where the study gets handed from one system, one format, or one person to the next and loses continuity along the way. That is why I think hand-offs deserve a lot more attention than they usually get. And a more strategic research process is what will make your team’s work easier to trust, easier to reuse, and much more likely to influence decisions over time.

So if your team is feeling buried in admin, struggling to keep momentum, or watching good research disappear after the deck gets presented, I would not start by asking whether you need more tools. I would start by asking where the hand-offs are breaking. From what I have seen, that question usually gets you much closer to the real problem, and a lot closer to a process that actually supports the work.

Have you seen UXR teams treat hand-offs like an afterthought instead of recognizing how much they shape the quality, speed, and usefulness of the work? If so, comment here or DM me. I’d love to hear where you’ve seen broken hand-offs show up in real-world research processes. As always, thanks for reading!