A Framework for Evaluation of Wheelchair Simulator-based Mobility Technologies

Abstract

Wheelchair simulators offer significant potential to support assessment and training in the power mobility provision. However, their adoption in clinical settings remains limited, largely due to the absence of standardised protocols for meaningfully evaluating user-simulator interaction in a safe and adaptable manner. Given the wide variability in users’ motor and cognitive abilities, there is a clear need for a structured framework that defines how simulator-based sessions are delivered and how user responses are assessed. This thesis presents a multidimensional evaluation framework, grounded in Quality of Experience (QoE) and Cognitive Load Theory (CLT), to assess user interaction with a virtual wheelchair simulator. The framework integrates subjective feedback (e.g., usability, emotion, cognitive workload, sense of presence), physiological signals (e.g., cardiac and electrodermal activity), and behavioural data (e.g., head and wrist movement) to capture users’ quality of experiences and tolerances (e.g., cybersickness) during simulator use. Two lab-based studies and one field-based pilot study were conducted. The first lab study compared immersive (Headset-1, N = 17) and non-immersive (Desktop, N = 24) conditions. Results revealed significant differences in workload (NASA-TLX, p < .01), presence (IPQ, p < .01), and heart rate variability change from baseline to first collision (p < .01, r = 0.43), with immersive use also eliciting higher cybersickness symptoms. These findings illustrate the trade-off between realism and tolerance. The second lab study expanded on the first (Headset-2 group, N=16) tested a smoother motion profile (low jerk). Symptoms decreased (SSQ-oculomotor, p < .05), but not enough to justify continued headset use in clinical settings. The most notable finding was the combined effect of display type and motion dynamics (p < .05), influencing usability (SUS) and sense of presence (IPQ). The final field-based pilot study (N = 20; 10 wheelchair users, 10 controls) examined the framework’s feasibility and clinical relevance in Irish Wheelchair Association (IWA) centres. Simulator metrics distinguished groups, with collisions (p < .01, r = 0.74) and trajectory deviation (RMSE, p < .01, r = 0.85) higher among users. Cognitive functioning (MoCA) correlated negatively with collisions (ρ = –.73, p < .05) and RMSE (ρ = –.74, p = .05). Heart rate analysis showed a significant group difference in HR change (p = .003, r = 0.67), with users maintaining elevated HR while controls showed reductions. User feedback further guided refinement of the evaluation framework. Together, these studies demonstrate the feasibility and potential clinical value of wheelchair simulators for delivering tailored, repeatable, and safe assessments. The resulting QoE-based framework, EMPOWER-SIM, is presented as a set of preliminary, feasibility-informed guidelines for clinical integration, offering a foundation for future pilot and validation studies in power mobility assessment and training.