Blacklight User Interface Research

This page is a collection of known reports evaluating aspects of Blacklight’s user interface. Reports using various research methods are appropriate to include here (e.g., usability tests, accessibility evaluations, heuristic inspections, surveys, web analytics, etc.).

Have reports to share? Please add them here directly if you're able. Else, reach out on Code4Lib Slack #blacklight channel or the blacklight-development listserv.

Study: Blacklight Search UX (2024)

https://crkn-rcdr.gitbook.io/user-centered-design/study-blacklight-search-ux

Authors: Brittny Lapierre, Canadian Research Knowledge Network / CRKN-RCDR
Methods: Usability Testing (24 remote/unmoderated tests); Questionnaire
Platform: Blacklight 8 default UI, branded for CRKN's Canadiana collection

Aspects Evaluated

basic keyword searching
results per page element
sort results by element
facets: expanding facets that are collapsed by default
facets: in-modal navigation of all values for a given facet
date range facets; entering From & To values
removing applied filters; Reset Search

Key Findings / Recommendations

Facet Modals. Task success rate decreased the more modals were involved. Scrolling the main window was not disabled when the modal/popup was active (likely bug). Scrolling within the actual modal was confusing/frustrating. Only 50% success rate on a task involving a facet sorted by count, when the target value was not in the first page of values. 33% of users clicked the A-Z sort button during that task to try to find the value. Next/previous links were hard to find and frustrating to have to use. A-Z links were expected especially when sorted alphabetically.
Search Box. When tasks could be completed by either scoping the main search box to a particular field or narrowing by that field's possible values via its facet (find results matching a particular Author or Subject), slightly less than half of users (45.8% for Subject; 37.5% for Author) opted to use the search box. Out of these users, only about half (63.4% Subject; 44.44%) used the field dropdown to scope their search.
Results per Page. 91% success rate, but unanimous confusion about whether the element actually worked, since no loading indicator was presented while the results were re-rendering.
Loading Indicators. Add a loading indicator on the search page that displays when the page sends a request to the back end for search results. This could be triggered by any of the sorting options, the display count, or when entering new search terms or facets.
Removing All Applied Filters. 96% task success rate, but via different paths. Most users clicked X on the individual applied constraints at the top. Some (16.67%) clicked X on the individual applied constraints in the facet sidebar. Only 30% of users who noticed the top constraints section clicked the Reset Search button.
Sorting. 100% success rate for re-sorting results by "newest to oldest." Ensure "oldest to newest" is also an option.
Facet Labels. Ensure facet labels are easy to understand and differentiate. A facet title like "Collections" may be unintuitive.
Date Range Facet. Place the date range inputs directly under the Date Range facet header.

The American Congress Digital Archives Portal Project White Paper (2022)

https://researchrepository.wvu.edu/faculty_publications/3090/ (Appendix D: usability report)

Author: Danielle Emerling (West Virginia University)
Methods: Usability Testing (20 remote/moderated tests); Questionnaires
Platform: American Congress Digital Archives Portal / custom Samvera/non-Hyrax
Blacklight Version: 6.x

Aspects Evaluated

usability of the web interface design, information flow, and information architecture
finding a document about a particular organization
finding files created by or about a particular person
finding files related to a particular subject

Key Findings / Recommendations

Overall. Most participants found the site clean, well organized, and intuitive; search limiters (facets) and search bar most valuable features. Overall satisfaction rating of 87%. The three tasks had 85%, 95%, 100% success rate.
Search Box & Facets. Some users didn't realize that the facets and search bar could be used together to refine their search results.
Sort Options. Participants noted being able to sort results by date or media type would be useful.
Advanced Search. Several users felt it would be useful to add (not present at time of testing).
Search with Quotation Marks. Some users attempted searching using quotations, which yielded zero results. Recommended supporting phrase searching using quotes.
Stacking Facets. Some users found the ability to stack search limiters to be useful, whereas others found this feature to be cumbersome. Some suggested that search limiters should refresh for new searches.
Metadata on Search Results. Some users felt there was too much metadata shown for each search result, decreasing scanablility. Others felt it was adequate.
Facet Label Jargon. Several users were confused by the naming conventions of search limiters (e.g., coverage, spatial). Review labels for clarity.

ArcLight remote usability test (2020)

https://dukespace.lib.duke.edu/items/ce8380b4-757c-4df6-8383-15ce64e6efdc

Authors: Candice Wang, Anh Nguyen, Joyce Chapman (Duke)
Methods: Usability Testing (10 remote/moderated tests); Questionnaire
Platform: ArcLight 0.x with custom layout / Duke University Archives & Manuscripts
Blacklight Version: 7.5.0

Aspects Evaluated

general page layout intuitiveness (search results, collection pages, component pages)
requesting materials from a specific box
“search within collection” vs. “search across collections” scopes
component context / breadcrumb trail
hierarchical nav tree vs. contents list
locating digital content

Key Findings / Recommendations

Page Layout. Mostly intuitive in funneling users to what they need to see; appreciation for having multiple ways to search for and get to the same material. 100% success rate on task finding a given collection using main search box and locating relevant components in it.
Requesting a Box. Only 30% success rate overall. Counterintuitive and confusing workflow: see a container list in arclight UI > click Request (leave arclight UI for separate request system) > read list of collection's requestable containers and remember which one was relevant. Request action should lead directly to the relevant container, else the request system UI should list the contents.
Search Box Scope. Almost all users successfully changed the search scope dropdown from this collection to all collections when beginning on a component page and tasked with searching for materials in other collections.
Component Context. Users all used the breadcrumb trail to locate parent collection context for a component. Some confusion noted over archival terminology ("collection" and "series") even among experienced users.
Sidebar Nav Tree. When tasked with finding a different component within the same collection, 80% of novice users interacted with the tree unprompted. Yet 100% of experienced users opted first for the search box and only used the tree when prompted. Many users struggled to recognize the show more expander in the tree.
Locating Digital Content. 100% task success rate, but via different strategies. 60% used an Advanced Search filter; 40% used the Online Access facet, which was the final facet at the bottom. Recommended giving this facet more prominence.

Usability Analysis of the Big Ten Academic Alliance Geoportal: Findings and Recommendations for Improvement of the User Experience (2017)

https://journal.code4lib.org/articles/12932

Authors: Mara Blake (Johns Hopkins), Karen Majewicz (Univ. of Minnesota), Amanda Tickner (Michigan State), Jason Lam (Univ. of Michigan)
Methods: Usability Testing (16 in-person/moderated); Heuristic Evaluation; Web Analytics
Platform: GeoBlacklight / BTAA Geoportal

Aspects Evaluated

Search strategies: relative usage of main search box, facets, and map inset
Heuristic evaluation categories: features/functionality; homepage; navigation; search; content/text; help; performance

Key Findings / Recommendations

Keyword Search. Most problematic feature as searches often did not return expected results (esp. after typing a place name or phrase that did not exist in any metadata records). Web analytics confirmed the vast majority of text searches were place name keywords. Normalize element values for keywords, place names, and authors/publishers. Add synonym files to Solr to facilitate discovery.
Failed Search / No Results Page. Add more info/suggestions for what else to try.
Facet Order. The first three facets were expanded by default; users rarely clicked on any others. They found Institution and Collection facet labels confusing. Emphasize the most useful facets (Place, Data type, Subject, and Time period) by putting them first, and set them to display as expanded by default.
Year Facet. Users found it difficult to use when sorted by numeric count (default) instead of chronological order. Recommended a Time Period facet instead.
Subject Facet. Users found the facet helpful for exploring content, however values were not normalized for capitalization/consistency so similar terms appeared multiple times.
Applied Constraints. Many users did not notice that they can combine facets to narrow down the items returned; also did not realize that as they clicked on facets, the system "remembered" selections, and this behavior could cause subsequent searches to fail. Many did not notice the nav options for Back to search & Start over. Make options in this section clearer.
Results per Page. Many users expressed desire for more than ten results per page; set default to twenty.
Map Inset. Users did not recognize it could be used as a search tool; low usage confirmed via web analytics.
Item Page. Have a consistent download button/or substitute for download button. Add text that indicates what is being shown in the preview box. Include thumbnails for scanned maps. Add functionality for supporting documentation (attribute table, codebook, data dictionary, etc.).
Search Strategies. Level of GIS experience appeared to have little to no effect on item discovery. Instead, success could be predicted based upon the level of experience in research, libraries, or even online shopping.