Modular Integration of Raw NASA-TLX and VisAWI-S in UIX-Probe: An Analytics Dashboard for Supporting UX Evaluation
DOI:
https://doi.org/10.58812/wsist.v4i02.3060Keywords:
UI/UX Evaluation, Raw NASA-TLX, VisAWI-S, Analytics Dashboard, Multi-Method Evaluation PlatformAbstract
Evaluation of User Interface (UI) and User Experience (UX) typically relies on several questionnaire instruments that are administered and scored separately, making score aggregation and cross-method triangulation labor-intensive. UIX-Probe is a web-based multi-method UI/UX assessment platform that already integrated the System Usability Scale (SUS), the Technology Acceptance Model (TAM), A/B testing and the Web Content Accessibility Guidelines (WCAG), but lacked instruments for two constructs central to user experience: subjective mental workload and perceived visual aesthetics. This paper reports the design, implementation and workflow of two new modules integrated into UIX-Probe Raw NASA-TLX and the Short Visual Aesthetics of Websites Inventory (VisAWI-S) with a focus on the analytics dashboard that turns raw responses into interpretable output for developers and interface designers. The dashboard provides a composite score with automatic classification, a per-dimension mean comparison, per-question response distributions, a per-respondent results table, respondent demographics, and automatically generated narrative descriptions, all exportable to Excel. The system was built using the Waterfall SDLC on a Laravel 10 + React.js (Inertia.js) architecture. Functionality was verified under real-world deployment: five independent developer teams created and distributed their own evaluation surveys on the platform, collecting 115 responses across five surveys. The four surveys using the new modules produced Raw NASA-TLX composite scores of 10.99–21.75/100 (all Moderate) and VisAWI-S composite scores of 4.23–5.81/7 (Positive to Very Positive). All three verification criteria were met: 5/5 teams configured a survey successfully, every survey obtained real respondents, and every dashboard rendered without error.
References
[1] International Organization for Standardization, Ergonomics of human-system interaction Part 210: Human-centred design for interactive systems, ISO 9241-210:2019, 2019.
[2] R. Hartson and P. Pyla, The UX Book: Agile UX Design for a Quality User Experience, 2nd ed. Burlington, MA, USA: Morgan Kaufmann, 2018.
[3] J. Brooke, “SUS: A 'quick and dirty' usability scale,” in Usability Evaluation in Industry, P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L. McClelland, Eds. London, U.K.: Taylor & Francis, 1996, pp. 189-194.
[4] J. R. Lewis, “The System Usability Scale: Past, present, and future,” Int. J. Hum.-Comput. Interact., vol. 34, no. 7, pp. 577-590, Mar. 2018, doi: 10.1080/10447318.2018.1455307.
[5] F. D. Davis, “Perceived usefulness, perceived ease of use, and user acceptance of information technology,” MIS Quart., vol. 13, no. 3, pp. 319-340, Sep. 1989, doi: 10.2307/249008.
[6] World Wide Web Consortium, Web Content Accessibility Guidelines (WCAG) 2.1, W3C Recommendation, 2018. [Online]. Available: https://www.w3.org/TR/WCAG21/
[7] S. G. Hart and L. E. Staveland, “Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research,” in Human Mental Workload, P. A. Hancock and N. Meshkati, Eds. Amsterdam, The Netherlands: Elsevier, 1988, pp. 139-183, doi: 10.1016/S0166-4115(08)62386-9.
[8] S. G. Hart, “NASA-Task Load Index (NASA-TLX): 20 years later,” in Proc. Hum. Factors Ergon. Soc. Annu. Meeting, vol. 50, no. 9, 2006, pp. 904-908, doi: 10.1177/154193120605000909.
[9] S. Said et al., “Validation of the Raw National Aeronautics and Space Administration Task Load Index (NASA-TLX) questionnaire to assess perceived workload in patient monitoring tasks: Pooled analysis study using mixed models,” J. Med. Internet Res., vol. 22, no. 9, p. e19472, Sep. 2020, doi: 10.2196/19472.
[10] A. Ramkumar et al., “Using GOMS and NASA-TLX to evaluate human-computer interaction process in interactive segmentation,” Int. J. Hum.-Comput. Interact., vol. 33, no. 2, pp. 123-134, 2017, doi: 10.1080/10447318.2016.1220729.
[11] B. R. Lowndes et al., “NASA-TLX Assessment of surgeon workload variation across specialties,” Ann. Surg., vol. 271, no. 4, pp. 686-692, Apr. 2020, doi: 10.1097/SLA.0000000000003160.
[12] N. Al Madi, S. Peng, and T. Rogers, “Assessing workload perception in introductory computer science projects using NASA-TLX,” in Proc. 53rd ACM Tech. Symp. Comput. Sci. Edu. (SIGCSE), vol. 1, 2022, pp. 668-674, doi: 10.1145/3478431.3499406.
[13] F. N. Azizah, D. Nurwinata Rinaldi, and F. Wafiq Al-Muqaffa, “Assessing mental workload of automotive factory employees using NASA-TLX and RSME methods,” J. Ilm. Tek. Ind., vol. 24, no. 1, pp. 107-116, Jun. 2025, doi: 10.23917/jiti.v24i1.8363.
[14] M. Moshagen and M. T. Thielsch, “Facets of visual aesthetics,” Int. J. Hum.-Comput. Stud., vol. 68, no. 10, pp. 689-709, Oct. 2010, doi: 10.1016/j.ijhcs.2010.05.006.
[15] M. Thielsch and M. Moshagen, “Measuring visual aesthetics with the VisAWI” [originally published in German as “Erfassung visueller Ästhetik mit dem VisAWI”], in Usability Professionals. Stuttgart, Germany: German UPA, 2011, pp. 260-265.
[16] M. Moshagen and M. T. Thielsch, “A short version of the Visual Aesthetics of Websites Inventory,” Behav. Inform. Technol., vol. 32, no. 12, pp. 1305-1311, Dec. 2013, doi: 10.1080/0144929X.2012.694910.
[17] M. Thielsch and M. Moshagen, VisAWI Manual (Visual Aesthetics of Websites Inventory) and the Short Form VisAWI-S. Münster, Germany: Univ. Münster, 2014.
[18] G. Hirschfeld and M. T. Thielsch, “Establishing meaningful cut points for online user ratings,” Ergonomics, vol. 58, no. 2, pp. 310-320, Feb. 2015, doi: 10.1080/00140139.2014.965228.
[19] I. Sommerville, Software Engineering, 10th ed. Boston, MA, USA: Pearson, 2016.
[20] European Parliament and Council, Regulation (EU) 2016/679 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data (General Data Protection Regulation), Official J. Eur. Union, L 119, pp. 1-88, 2016. [Online]. Available: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32016R0679
[21] Republic of Indonesia, Law No. 27 of 2022 on Personal Data Protection [Undang-Undang Nomor 27 Tahun 2022 tentang Pelindungan Data Pribadi], State Gazette of the Republic of Indonesia No. 196, 2022.
[22] Lyssna, “Lyssna: user research and testing platform (formerly UsabilityHub),” [Online]. Available: https://www.lyssna.com. [Accessed: Aug. 2026].
[23] Maze, “Maze: user research platform,” [Online]. Available: https://maze.co. [Accessed: Aug. 2026].
[24] Lookback, “Lookback: user research platform,” [Online]. Available: https://www.lookback.com. [Accessed: Aug. 2026].
[25] R. Budiu and C. Moran, “How many participants for quantitative usability studies: A summary of sample-size recommendations,” Nielsen Norman Group, Aug. 2021. [Online]. Available: https://www.nngroup.com/articles/summary-quant-sample-sizes/
[26] J. Sauro and J. R. Lewis, Quantifying the User Experience: Practical Statistics for User Research, 2nd ed. Amsterdam, The Netherlands: Morgan Kaufmann, 2016.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Anugrah Nur Rahmanto, Retno Damayanti, Muhammad Abel Rif'at Nandito
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Facebook 
Instagram 
Twitter 
Youtube 
westscience.journals@gmail.com