Problem/Motivation
Our user testing on the HTML mock of the left-aligned, collapsable, vertical sidebar navigation will happen in two phases. In the first phase, our goal is to evaluate the effectiveness of the changes we’ve implemented on the convenience and usability of the admin navigation.
Our objectives are to:
- Identify design and interaction concerns that need to be addressed
- Build understanding on how easy or difficult it is to navigate the admin UI with the redesigned toolbar
- Validation that the new designs make Drupal appear as a more modern CMS
Phase one of the usability testing will include three experienced site builders and three individuals that are newer to Drupal, plus an initial 0 tester. Testers will be given a set of everyday tasks to complete on the current implementation and then they will be asked to complete the same tasks in the new HTML mock-up. Their feedback will be recorded and kept anonymous.
Results
The moderated usability tests were conducted online. Participants joined a video-call where they shared their screens of their work spaces (all desktop). We conducted the tests with 8 people: an initial participant to pilot the test protocol, 3 participants who were not familiar with the Drupal administration interface and 4 long-term Drupal users who were familiar with the Drupal administration interface.
We focused on testing high level hypotheses like moving the toolbar navigation from the top of the viewport to the left, the collapsible toolbar behavior and its button trigger placement, and the menu flyout usability.
The test protocol asked participants to perform the same actions (log-in, edit a content type, create a content type, list content) in a control version with a default 11.x Drupal installation and then to repeat those actions in a 2nd round with the new prototype with
🐛
Page title should contextualize the local navigation
Needs work
.
Note: The prototypes weren’t fully finished. For example, the active-trail behavior wasn’t being fully implemented on the first ones and the mobile designs weren’t finished (which we did not test in this phase).
Findings
Some of the learnings we can use from the feedback about the current Toolbar is that all experienced users mentioned the lack of control when the Toolbar automatically switches to left on smaller screens is annoying, especially when it renders on top of the content on narrow viewports. This means that we will need to find the least impactful sidebar resize when we switch the menu in smaller viewports.
Several users had challenges discovering additional menus beyond the default option "Manage" with the old designs.
Several users missed the button to toggle the toolbar orientation. This could be due to the fact that the button is located too far to the right.This is likely caused by the documented F shape reading pattern, which should be avoided in the new designs.
- The first menu items' hover and focus states weren’t enough for 2 of the 8 participants, who missed the arrow and clicked directly on the label for the 1st item (like Structure) when they wanted to navigate to 2nd level menu item in the submenu. It happened even after knowing and being told they could open the sub-menu. To note this NN Group recommendation to Don’t Use Split Buttons for Navigation Menus shared by @lauriii.
- Users showed higher confidence in their orientation within the admin UI with a strong active menu background indicator for the active section. In the first tests, when it wasn’t implemented yet, it was suggested by the participants as a missing feature. Drupal core is still missing in some situations that will be addressed in
🐛
Page title should contextualize the local navigation
Needs work
.
- The interaction to open and use the flyouts (when the sidebar is collapsed) needs to be refined: the flyout didn’t always stay open for the mouse trail.
- Renaming Shortcuts to Bookmarks made it more clear for the 3 less familiar users what it was for, and created low friction with the experienced ones. Only one asked “Have the Shortcuts been renamed to Bookmarks?”, and had no issues with it when it was confirmed.
- In general all reactions towards the new navigation were positive. All participants preferred the new navigation over the old navigation. Some of the positive feedback included that:
- Less experienced users felt that having the navigation on the left was a familiar pattern closer to other modern applications and products that they use.
- Separation of concepts from the working area or content.
- Users prefer having consistency in the position of the toolbar. They preferred to avoid resizing or changing layout automatically.
- Users were able to find the different submenus.
- The ability to collapse the sidebar by all users because it gives more real estate to work or preview content while still having access to the submenus
- Icons on their own were not sufficient for users to identify the different menu links. Users were able to identify the different menu links using the tooltip.
- For one participant, the visual difference between the sidebar and the front-end/website wasn’t clear enough.
- Flyout background color contrast: one participant was using a screen with low contrast and missed some visual clues to determine where the limits for the flyout were.
- All the participants found useful the “Create” menu and its options below:
- Quicker to add content.
- Liked there’s also Media and other entities. This will need research and discussion to define the final elements.
- Danger: Plus “+” icon might be confused with other actions like collapsing the sidebar.
- All the participants were able to separate between Content items and the other main navigation items. One of the potential conflicts with this approach is the Block content page: it’s a drupalism (
📌
Locate drupalisms that might create confusion among users
Active
) that might confuse new users.
- Concentration of similar tasks (content related tasks) in the same place was helpful.
Other results
All participants showed higher confidence in their orientation with the prototype that used a title that was generated based on the local navigation section.The test consisted in providing a link to a Manage Fields page and asked them to tell us if they could identify where they were: in the actual UI only the breadcrumb shows which content type you are. In the new version, the page title was Article and the Manage fields title was underlined. This signaled that they were managing fields for the Article content type.
But by itself doesn’t solve the problem entirely: users on the first tests (where the active trail in the main menu wasn’t implemented yet) still missed more clues to use as reference to locate themselves in a section.
Participants agreed on wanting to see more meaningful content after login. For example, after logging in and landing on the user page, we asked users if there was any information that was unclear for them. For inexperienced users, the page didn’t provide direction about what to do next. For experienced users, it doesn’t provide quick access to the tools they use the most.
