A/B Testing
Top Navigation
Context
I worked at a life insurance tech start up (specific name available upon request). This was a usability test for our core web app to test the top navigation bar. I examined the discoverability of the dashboard/home page with or without an explicit "dashboard" label. And to research if most users would know to use the logo in the absence of a label as a return to the home page/dashboard.
Key Assumptions
Test A: We expect that the label will make it easier for users to find the dashboard.
Test B: We expect that the user will recognize the logo as an area to return to the dashboard.
Key Questions
We will use an A/B test to see which top navigation best helps users find the dashboard. Question for user was : Where will you click to return to the home page?
Key Outcomes/Metrics
Time to complete
Statistically significant difference between designs
The tool used was Usabilityhub and below are heat maps of users click paths. I used MeasuringU's Binomial Test with Confidence Interval to determine statistically significant differences between the design patterns.
Test A
When both the "dashboard" label and logo were included, we can be 80% confident in the assumption that the logo still worked as a return to home page even despite the presence of the label "Dashboard." (Average response time 48.6 seconds).
Test B
When I removed the "Dashboard" label, 100% of the participants selected the logo as a return to the dashboard. (Average response time 40.7 seconds).
Results
When all participants are calculated together we can still be 90% confident in the logo as the primary click path back to the home page. The majority of written responses from participants also supported the users' understanding that the logo worked as a return to the home page/dashboard.
"Usually, when there isn't a distinct home button, I click on the site name, which is often in the upper left hand corner to take me back to the home page."
"I was recently discussing this with a friend. - We both agreed it was VERY NORMAL and almost expected, that you can click the logo to return to the home page."
"There is no explicit home page link, so clicking on the logo of the page is a normal way to return home."
"Most websites use their logo as a way to return to the homepage."
Design Recommendations
Use the logo as a return to the home page/dashboard for users to click the correct part of the top navigation. This will also declutter the top navigation bar.