A Multimodal Analysis of Online Information Foraging in Health-Related Topics Based on Stimulus-Engagement Alignment: Observational Feasibility Study

Abstract
BackgroundThe recent increase in online health information–seeking has prompted extensive user appraisal of encountered content. Information consumption depends crucially on the quality of encountered information and the user’s ability to evaluate it; yet, within the context of web-based, organic search behavior, few studies take into account both these aspects simultaneously.
ObjectiveWe aimed to explore a method to bridge these two aspects and grant even consideration to both the stimulus (web page content) and the user (ability to appraise encountered content). We examined novices and experts in information retrieval and appraisal to demonstrate a novel approach to studying information foraging theory: stimulus-engagement alignment (SEA).
MethodsWe sampled from experts and novices in information retrieval and assessment, asking participants to conduct a 10-minute search task with a specific information goal. We used an observational and a retrospective think-aloud protocol to collect data within the framework of an interview. Data from 3 streams (think-aloud, human-computer interaction, and screen content) were manually coded in the Reproducible Open Coding Kit standard and subsequently aligned and represented in a tabularized format with the R package {rock}. SEA scores were derived from designated code co-occurrences in specific segments of data within the stimulus data stream versus the think-aloud and human-computer interaction data streams.
ResultsSEA scores represented a meaningful comparison of what participants encountered and what they engaged with. Operationalizing codes as either “present” or “absent” in a particular data stream allowed us to inspect not only which credibility cues participants engaged with with the most frequency, but also whether participants noticed the absence of cues. Code co-occurrence frequencies could thus indicate case-, time-, and context-sensitive information appraisal that also takes into account the quality of information encountered.
ConclusionsUsing SEA allowed us to retain epistemic access to idiosyncratic manifestations of both stimuli and engagement. In addition, by using the same coding scheme and designated co-occurrences across participants, we were able to pinpoint trends within our sample and subsamples. We believe our approach offers a powerful analysis encompassing the breadth and depth of data, both on par with each other in the feat of understanding organic, web-based search behavior.