Project browser September 2023 usability testing results with the University of Minnesota

The current project browser user interface was tested last week with David Rosen of the University of Minnesota. The scenarios were prepared by @leslieg and @chrisfromredfin with the help of @lauriii, @dead_arm, @Gábor Hojtsy and @ckrina.

Goals of the test

We were interested to observe whether users find the Project Browser and understand its basic elements, such as the filters and cards, as well as how they use the functionality to search for specific modules and whether users can perform module installation. We provided precreated environments with Project Browser installed that users accessed remotely. Project installation functionality was also enabled with Package Manager. Some of the key findings were as follows. For problems found, we assigned severity scores from 1 (low) to 5 (high) based on the impact of the problems on doing the tasks.

General user feedback

Overall people were really happy with the functionality. While problems were identified, the testers gave an average 4.25 score (on a scale of 1-5 where 5 is best) that they would like to use it in the future. Easyness got an even better score at 4.75, and people thought others would have an easy time learning to use this functionality (scored a clear 5 on average). That said, there were problems identified, see below.

Issues identified with potential next steps

The difference between List and Browse in Extend was not clear. Also the placement of Browse made it unclear if it may be another view on the same data under List. Many users did not even notice the Browse tab and needed explicit instruction to use it. (Severity 5)

We need to explore whether to make Browse the default functionality in Extend (since installation happens there). However then some functionality such as configuration of installed projects is lost. Either way, List and Browse sound too similar and serve overlapping purposes, so this needs a more holistic solution than merely changing labels or orders.

It was not clear that only compatible and maintained projects are listed. (Severity 2-4)

Some users were looking to gain more information about the maintenance level of projects. Not all of them noticed the default filter of only maintained modules. Multiple users suggested that displaying information about the "most recent update" on the card would be useful to help gauge activity on a project.

Project browser may not install the version that the user expects (Severity: 4)

Currently Package Manager is responsible for picking which version of a project to install, so it may install a version that did not necessarily match the filters selected. This is probably the most complicated technical issue uncovered. We may want to add a verification step when the version is known before doing the installation or do a dry-run of composer first (Package Manager would need to support that in its API).

Different users raised that the cards were a bit chunky or that they contained little information compared to their size. (Severity 1-2

We currently display the 200 character truncated description of the project, but working on hand-crafting better text for the top 100 projects. We also discussed making the card more useful by making the logo image clickable and potentially adding a "Read more" link to lead to the more detailed standalone page. For those that find the card view taking up too much space, the list view is already available but nobody used it or found it. We may want to revisit how we display the toggle visually.

People expected more of the detail page: to configure installed modules, get issue queue stats, etc (severity: 4)

For projects that are already installed, people expected to be able to configure them there. This would help in part with taking over the "List" page from Extend. Also more information was expected of the detail page. We did not yet work much on this page experience, so this was not a surprise. We may also consider opening the detail page in a modal.

Category filters were not used without explicit guidance (severity: 4-5)

Even though the category filter takes up a lot of side space, people went to the free form text search and used that instead. But the free text search does not currently search in the category which was a major bummer. We are already working on reducing the number of categories. We could filter out categories that have zero results for other search terms to make this list easier to use. We could move it closer to the other search widgets or make it a single select as well. Or we could reduce the items to the top 5-10 categories and hide the rest under a "more" link. Either way searching for free form text should also search in the categories attached to projects.

It was not clear where a project is placed when its installed (severity: 4)

There are multiple places where projects may be installed and it was not clear where Project Browser installed them. There was no feedback on it once done. We could add some kind of confirmation once the installation is done that explains what happened and what are the next steps (how to configure the project installed for example). Potentially we could introduce a way for project maintainers to provide this information and encourage them to do so.

It was unclear if the user can leave the page while an installation is underway (severity: 5)

We need to enforce the install function better. Make sure that they can't leave or that they're warned if they try to leave the page during installation. A confirmation prior to "Apply" could solve the issue, so that the user needs to confirm that they truly want to install the module and its related items (as per above, this may be the point when we know which exact version will be installed also).

Some users expected more than 12 results per page and it was not clear how to change this (severity: 3-4)

Pagination hides the setting to change number of projects per page. Consider separating the pager from the number per page functionality.

Capitalisation in search terms changed the results (severity: 5)

This will not be an issue in the final version as the search will happen on the Drupal.org API that will not be case sensitive.

Multiple people noticed that some modules had tags that were not expected. That made the results less useful (severity: 3)

There is ongoing work in the initiative to recategorize projects to clean these up at least for the top modules.

One user noted that the meaning of the pills were not clear on the cards (severity: 2)

We may want to consider adding a "Categories" label to them.