The Unseen Value: Cognitive Inclusion in UX Research Reveals 1.8x More Usability Insights

A groundbreaking exploratory user research study, a collaboration between Fable and the University of California, Irvine, has demonstrated that participants with cognitive disabilities identify significantly more usability issues and offer a greater volume of practical recommendations than their general population counterparts. The findings, highlighting the profound benefits of cognitive inclusion in user experience (UX) research, suggest a paradigm shift in how digital products should be evaluated and designed for universal accessibility.

The Growing Imperative for Cognitive Accessibility

The impetus for this research stems from a critical, yet often overlooked, demographic shift. Cognitive disability, an umbrella term encompassing conditions that affect memory, focus, and learning—including neurodevelopmental conditions like Dyslexia, ADHD, and Autism—is the most prevalent disability in the U.S., affecting 13.9% of the population, according to the Centers for Disease Control and Prevention (CDC). Alarmingly, a Yale study indicates this prevalence is increasing rapidly. This demographic reality, coupled with the projected increase in the U.S. population aged 65 and older—from 17% to 25% by 2060, as per the U.S. Census, leading to a natural rise in age-related cognitive decline—underscores an urgent need for digital experiences that are inherently accessible to diverse cognitive profiles. Historically, UX research has predominantly focused on general population users, often inadvertently overlooking crucial friction points that disproportionately impact individuals with cognitive challenges, and by extension, a much broader user base experiencing temporary or situational cognitive load.

A Chronology of Dedicated Research

The journey towards this pivotal study began in the summer of 2024, when the Vice President of Innovation at Fable, spearheaded the co-chairing of a specialized working group. This collective of expert researchers convened with a singular, ambitious goal: to establish optimal methodologies for conducting accessibility testing with people with cognitive disabilities. The initiative was born from Fable’s commitment to advancing inclusive design principles.

The working group embarked on a structured research trajectory. Initially, they developed a precise screener to recruit participants who self-identified with challenges related to memory, focus, and learning. Simultaneously, an extensive review of existing published studies involving cognitive testers was undertaken to glean best practices and refine methodological approaches.

This preparatory phase culminated in a pilot study involving an initial group of 25 testers. This pilot served as a crucial proving ground, allowing the researchers to test and iteratively fine-tune their approach. The insights gained led to the creation of a comprehensive guide for running user interviews with cognitive testers, alongside a specialized survey designed to quantify their experiences with digital products. The detailed learnings from this pilot were meticulously documented, laying the groundwork for the subsequent, more extensive study. It was after observing the pilot’s outcomes that the lead researcher developed a strong hypothesis: participants with cognitive disabilities might uncover more usability insights than traditional general population user research participants. The full-scale "Cognitive Usability Study" was then launched to validate this intuition.

Methodology: A Deeper Dive into the Study Design

To rigorously test the hypothesis, the Fable team partnered with the University of California, Irvine, with significant contributions from Syed Fatiul Huq and Fable researchers Pranav Pidathala, Ali Brown, and Michael Fagan. The study design aimed for a comprehensive evaluation across varied digital contexts.

Three distinct websites were generated using an AI prototyping tool, specifically chosen to represent different user goals, content structures, and levels of complexity:

• Strong Snacks: A simple, brutalist-designed website for three-ingredient high-protein recipes, featuring basic browsing, blog posts, and contact information. Its key functionalities included category filtering and newsletter subscriptions.
• Turning Pages: A moody, classic bookstore website with extensive filtering by genre, a book-swiping feature for profile building, custom book lists, a shopping cart, and checkout. This site presented the most varied functionality.
• Crown & Comb: A bold, clean hair salon website enabling online appointment bookings and consultations, complete with a VIP program and special packages. This site was intentionally designed to be complex, particularly one task involving finding a bridal package.

A single screener, inquiring about memory, focus, and learning challenges, was used to recruit participants. Based on their self-identification, participants were divided into two distinct groups: those with cognitive challenges and those representing the general population.

The study involved 30 user interviews, with 10 participants allocated to each website. Crucially, each website’s participant group maintained an even 5:5 split between cognitive and general population individuals. Each participant completed all assigned tasks for one specific website during a facilitated online user interview. Researchers guided these sessions using a standardized task structure and discussion guide template, although different researchers facilitated different sessions. At the conclusion of their session, all participants completed the Accessible Usability Scale (AUS) survey, a free, Creative Commons-licensed 10-question tool designed to evaluate the usability of websites and mobile applications, providing a quantifiable measure of their experience.

For data analysis, the lead researcher meticulously reviewed all study recordings and transcripts. Every instance where a participant expressed a concern, asked a question, encountered a difficulty, or sought clarification on functionality was logged as an "issue." This also included instances where a participant missed a task element, even if unacknowledged by them. Additionally, every suggestion for improvement offered by participants was recorded. To ensure consistency, issues and suggestions were counted once per participant, even if mentioned multiple times by the same individual. The expectation in UX research is that similar issues will surface across multiple participants, signaling a universal challenge rather than an isolated incident.

Key Findings: Unpacking the Quantitative and Qualitative Data

The study’s results robustly validated the initial hypothesis, revealing a compelling quantitative advantage in involving cognitive participants. Across the three diverse websites tested:

• Cognitive participants identified a total of 197 usability issues, compared to 113 identified by general population participants.
• Cognitive participants offered 93 suggestions for improving the user experience, while general population participants provided 54.

This translates to cognitive participants identifying 1.8 times more issues and making 1.8 times more suggestions than their general population counterparts, a statistic that underscores the profound value of their unique insights.

Delving into the website-specific data further illuminated these trends:

Table 2: Strong Snacks – Simplicity and Hidden Friction
This website, with its minimalist design and straightforward content, predictably yielded the lowest overall issue counts and the highest median AUS scores. However, even in this seemingly simple environment:

• Cognitive participants found 3.4 times more issues (9.8 average vs. 6.4 for gen pop) and made 2.2 times more suggestions (4.8 average vs. 2.6 for gen pop).
• Their average AUS score was 13.7 points lower than gen pop participants, indicating a less favorable overall experience despite the site’s apparent simplicity.

Table 3: Turning Pages – Complexity and Overload
As the site with the most varied functionality and the highest number of tasks (four), "Turning Pages" naturally generated the most issues overall. Here, the disparity was even more pronounced:

• Cognitive participants found 6 more issues on average (17 vs. 11 for gen pop) and made 3.2 more suggestions on average (8.4 vs. 5.2 for gen pop).
• Their average AUS score was 17.2 points lower than gen pop participants, signaling significant frustration with the site’s complexity and interactive elements.

Table 4: Crown & Comb – Counterintuitive Usability Perceptions
This website was intentionally designed with complexity, particularly task 3 (finding the bridal package), which was meant to be extremely difficult. Intriguingly, while cognitive participants found significantly more issues:

• Cognitive participants found 7 more issues on average (12 vs. 5 for gen pop) and made 2.4 more suggestions (5.4 vs. 3 for gen pop).
• Paradoxically, their average AUS score for the overall experience was 14.3 points higher than the gen pop participants (63.8 vs. 49.5). This anomaly suggests that while cognitive participants identified more granular issues, their perception of the overall difficulty might have been tempered by the anticipation of a challenging task, or perhaps they appreciated any small successes more keenly. Conversely, general population participants might have been more surprised and frustrated by the unexpected complexity. This highlights a nuanced difference in how perceived difficulty translates to overall satisfaction between the groups.

Qualitative Findings: Beyond the Numbers
The qualitative analysis revealed distinct patterns in the types of issues identified:

• Cognitive participants consistently surfaced issues related to clarity, predictability, and cognitive load. They struggled with ambiguous instructions, inconsistent interface behavior, and overwhelming information density. Their feedback often revolved around the mental energy required to process information and complete tasks.
• General population participants, while also identifying usability issues, tended to focus on aesthetics, content relevance, and efficiency. Their concerns often pertained to visual appeal, directness of information, and streamlining workflows.

When issues were categorized, cognitive participants consistently identified more problems in specific areas:

• Content: Ambiguous language, lack of clear hierarchy, too much information.
• Buttons and links (affordances and function): Unclear calls to action, unpredictable link behavior, confusing iconography.
• Icons or visual elements: Misleading or non-standard icons, distracting animations.
• Media: Overwhelming or auto-playing videos, complex interactive elements.
  Notably, issues related to navigation were nearly tied between both groups (45 for gen pop vs. 46 for cognitive), suggesting navigation is a universal challenge, but cognitive participants found more issues in the details of content and interaction within those navigation paths.

The profound impact on cognitive participants’ well-being was starkly illustrated by their direct quotes. While a general population participant described a frustrating and unengaging experience due to repetitive and unselectable services, a cognitive participant articulated feeling "drained and less able to focus" due to "too many options to look through" and a barrage of "cookies, and then ads, pop-ups." This distinction underscores that for cognitive users, usability issues translate into a significant expenditure of mental energy, potentially leading to disengagement and exclusion.

Broader Implications: The Universal Benefits of Cognitive Inclusion

The findings transcend mere accessibility compliance; they present a compelling business case for integrating cognitive participants into UX research. By addressing the cognitive load issues surfaced by this demographic, companies don’t just cater to a niche; they enhance the user experience for everyone. These improvements have a ripple effect, benefiting:

• Older adults: As the population ages, age-related cognitive decline becomes more common. Designs that reduce cognitive load are inherently more usable for this growing demographic.
• Individuals with temporary cognitive impairments: Stress, fatigue, illness, post-surgical recovery, or even multitasking in a busy environment can temporarily impair cognitive function. Clearer, more predictable interfaces are a boon to these users.
• Non-native speakers: Simplified language, intuitive layouts, and unambiguous visual cues aid comprehension for those processing information in a second language.
• Users in distracting environments: Whether commuting on a busy train or working in an open-plan office, users often operate with limited attention. Streamlined experiences help maintain focus.

Consider the specific examples from the study:

• On Strong Snacks, cognitive participants highlighted the need for more descriptive titles, clearer content hierarchy, and less continuous animation. These improvements foster greater trust in the content, improve readability, and allow for easier information scanning—benefits for all users, regardless of cognitive profile.
• For Turning Pages, cognitive participants found subtle but critical issues with confusing interactions, such as the "Add to book bag" button, and ambiguities around the source of reviews. Such issues erode user trust and, in an e-commerce context, can directly lead to abandoned carts and lost revenue. A clear, predictable purchase flow is paramount for business success.
• The Crown & Comb website particularly underscored the value of cognitive inclusion. Participants with cognitive disabilities identified problems with inconsistent visual cues, a lack of immediate feedback on actions, and the use of abstract iconography. While general population users might "muddle through," these issues can be a significant barrier for cognitive users, potentially leading to a complete inability to complete tasks. When a complex task, like finding a bridal package, is compounded by abstract imagery, a lack of direct feedback, and inconsistent interaction patterns, the cognitive load becomes overwhelming. It is at this critical juncture that a usability issue transforms into an accessibility barrier, effectively excluding a segment of the user base and impacting customer loyalty and retention.

Expert Perspectives and Industry Outlook

The sentiment within the UX community mirrors these findings. As one UX Manager at Bell Media aptly stated, "2 sessions with cognitive users feel like 200 because of the volume of insights we get." This perspective underscores the efficiency and depth of insights gained from this inclusive approach.

Cognitive inclusion in UX research is no longer merely an ethical imperative or a regulatory checklist item; it is a strategic advantage. It is how UX teams can significantly enhance the efficiency of their research, leading to clearer content, simpler user flows, and ultimately, superior products for everyone. As the digital landscape becomes increasingly complex, designing with cognitive accessibility in mind will future-proof products and services for an evolving user base. It serves as a powerful and accessible "on-ramp" into broader accessibility research, building a foundational understanding of universal design principles that can then be extended to assistive technology users, such as screen readers and screen magnifiers.

Study Limitations and Future Directions

While the findings are compelling, it is important to acknowledge the study’s limitations. The relatively small sample size (30 participants) means the findings are more qualitative and indicative of trends rather than quantitatively validated across a larger population. Furthermore, the use of two different testing platforms—Fable Engage for cognitive participants and UserFeel for general population participants—could introduce subtle variables regarding participant comfort levels and the quality of insights. The facilitation of user interviews by different researchers, though guided by a consistent template, also represents a potential variable. Future research could mitigate these limitations by conducting larger-scale studies on a single platform with consistent researcher facilitation.

Despite these limitations, the study unequivocally demonstrates the immense, untapped value of incorporating cognitive participants into UX research. It signals a critical shift towards a more inclusive and effective approach to product development, ensuring that digital experiences are truly accessible and beneficial for the broadest possible audience.

Resources
For those embarking on their cognitive inclusion journey, resources such as Fable’s accessibility testing platform, the Accessible Usability Scale, and various guides on cognitive accessibility can provide valuable starting points.