EVALUATION OF USABILITY AND USER INTERFACE DESIGN TO ENHANCE USER EXPERIENCE THROUGH HEURISTIC EVALUATION APPROACH AND LEAN UX METHOD WITH ISO 13407 STANDARDS Case Study : SISTER Kemendikbud Ristek RI
Abstract
Abstract: Sistem Informasi Sumberdaya Terintegrasi (SISTER) is a system that integrates services within the scope of higher education resources and data integration with the Higher Education Database (PDDIKTI). This study aims to evaluate usability using the Heuristic Evaluation questionnaire, Usability Testing in the Lean UX phase, and ISO 13407 Evaluation in the final testing phase. The distribution of the questionnaire resulted in 676 responses, with 397 samples used and involving 3 evaluators in Usability Testing during the Lean UX phase. In the final phase, the evaluation was conducted according to ISO 13407 standards, involving 9 respondents to test the prototype using task scenarios. The evaluation results showed an average score of <1200 and a mean score of <3, indicating a low level of user satisfaction with the system. Additionally, a TCR value of <65% also indicated that the system's interface was considered fair or average by users. On the other hand, the Usability Testing results identified improvements related to redundant pop-up messages and over-engineering of outline buttons in the prototype. This list of issues served as a reference for compiling recommendations and design improvements based on the 10 usability heuristics. After making improvements, the ISO 13407 evaluation results achieved a score of 80.89%, indicating a Good qualification.
Downloads
References
Kemendikbudristek, “Peraturan Menteri Nomor 28 Tahun 2021 Tentang Organisasi dan Tata Kerja Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi.,” Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi., Jakarta, 2021.
T. Prasetyo, “Evaluasi Usability Sistem Informasi Sumberdaya Terintegrasi: Studi Kasus Kementrian Pendidikan, Kebudayaan, Riset, dan Teknologi Republik Indonesia,” Universitas Indonesia, Jakarta, 2022.
Ditjen. Diktiristek, Jakarta. Sistem Informasi Sumberdaya Terintegrasi (SISTER), (2023).
Kemenristekdikti, “Permenristekdikti Nomor 61 Tahun 2016 Tentang Pangkalan Data Pendidikan Tinggi,” Kementerian Riset, Teknologi, dan Pendidikan Tinggi., Jakarta, 2016.
N. E. Jacobsen, Usability Evaluation Methods The Reliability and Usage of Cognitive Walkthrough and Usability Test. Denmark: University of Copenhagen, 1999.
J. Nielsen and R. Molich, Usability Engineering. Academic Press, 1993.
D. A. Norman, The design of everyday things.
H. Himawan and M. Yanu, Interface, User Experience. Yogyakarta: Lembaga Penelitian dan Pengabdian kepada Masyarakat, 2020.
S. Y. Chen and R. D. Macredie, “The assessment of usability of electronic shopping: A heuristic evaluation,” Int J Inf Manage, vol. 25, no. 6, pp. 516–532, Dec. 2005, doi: 10.1016/j.ijinfomgt.2005.08.008.
J. Nielsen, “10 Usability Heuristics for User Interface Design,” 1995.
J. Gothelf, Lean UX. United States of America: O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472., 2013.
ISO, ISO/IEC 13407 - Human Centred Design Process for Interactive Systems. ISO/IEX 13407, 1999.
T. Jokela, N. Iivari, J. Matero, and M. Karukka, “The standard of user-centered design and the standard definition of usability,” in Proceedings of the Latin American conference on Human-computer interaction, New York, NY, USA: ACM, Aug. 2003, pp. 53–60. doi: 10.1145/944519.944525.
R. Likert, A Technique For The Measurement of Attitudes. New York, 1932.
Sugiyono, Metode penelitian kuantitatif, Cet. 1. Bandung : Alfabeta, 2018.
Y. R. Endra, Indonesia. Cara Mencari Sampling Data Skripsi Denga Rumus Slovin, (Feb. 05, 2021). Accessed: Nov. 21, 2023. [Online Video]. Available: https://youtu.be/MfY3mZvrtCs?si=QnZ5dC7azkAUznMD&t=78
T. Fessenden, “Design System 101,” Nielsen Norman Group.
Andi, SPSS 20. Yogyakarta, 2012.
A. Pramudani, “Analisis User Experience (UX) Menggunakan Metode User Questionnaire Dan Heuristic Evaluation Pada Sistem Virtual Class Universitas Lampung,” Universitas Lampung, Bandar Lampung, 2022.
R. Parente Da Costa, E. Di. Canedo, R. T. De Sousa, R. De Oliveira Albuquerque, and L. J. Garcia Villalba, “Set of Usability Heuristics for Quality Assessment of Mobile Applications on Smartphones,” IEEE Access, vol. 7, pp. 116145–116161, 2019, doi: 10.1109/ACCESS.2019.2910778.
R. S. Pradini, R. Kriswibowo, and F. Ramdani, “Usability Evaluation on the SIPR Website Uses the System Usability Scale and Net Promoter Score,” in 2019 International Conference on Sustainable Information Engineering and Technology (SIET), IEEE, Sep. 2019, pp. 280–284. doi: 10.1109/SIET48054.2019.8986098.
N. Anggraini, D. Putra, and N. Hakiem, “Development of an Islamic Higher Education Institution Tracer Study Information System and It’s Performance Analysis using ISO/IEC 25010,” International Conference on Cyber and IT Service Management , 2019.
R. Kalantari and T. C. Lethbridge, “Characterizing UX Evaluation in Software Modeling Tools: A Literature Review,” IEEE Access, vol. 10, pp. 131509–131527, 2022, doi: 10.1109/ACCESS.2022.3227504.
H. M. Salman, W. F. Wan Ahmad, and S. Sulaiman, “Usability Evaluation of the Smartphone User Interface in Supporting Elderly Users from Experts’ Perspective,” IEEE Access, vol. 6, pp. 22578–22591, Apr. 2018, doi: 10.1109/ACCESS.2018.2827358.
H. Noprisson, N. Husin, M. Utami, P. Rahayu, Y. G. Sucahyo, and D. I. Sensuse, “The use of a mixed method approach to evaluate m-Government implementation,” in 2016 International Conference on Information Technology Systems and Innovation, ICITSI 2016 - Proceedings, Institute of Electrical and Electronics Engineers Inc., Feb. 2017. doi: 10.1109/ICITSI.2016.7858195.
R. Mayasari, A. Susilo Yuda Irawan, and U. Singaperbangsa Karawang, “Implementation Of Lean UX Method On UI/UX Design Of Digilib UNSIKA Application In Windows Version,” Journal of Information Technology and Computer Science (INTECOMS), vol. 4, no. 2, 2021.
H. Y. Pratama, T. B. Hanggara, and Y. N. Setiawan, “Evaluasi Usability dengan Menerapkan Metode Heuristic Evaluation pada Website Dinas Pendidikan Kota Batu. ,” Jurnal Pengembangan Teknologi Informasi Dan Ilmu Komputer, 6(3), 1350–1359., 2022.
S. Balafif, “Analisis website menggunakan Heuristic Evaluation berbasis severity ratings dan sistem usability scale,” JINTEKS (Jurnal Informatika Teknologi dan Sains), vol. Vol. 4 No. 3, Aug. 2022.
B. D. R. Kurniawan, H. S. Wijoyo, and H. N. Wardani, “Evaluasi Usability Aplikasi MY JNE Dengan Metode User Exsperience Questionner (UEQ) dan Heuristic Evaluation,” . Jurnal Pengembangan Teknologi Informasi Dan Ilmu Komputer, pp. 5583–5591, 2019.
E. Saputra, ) Sulistiowati, J. Lemantara, ) Program, S. / Jurusan, and S. Informasi, “Perancangan Desain User Interface / User Experience Layanan Informasi Kampus (LIK) Dengan Metode Lean User Experience (Lean UX) Pada Universitas Dr. Soetomo,” Surabaya, 2018. [Online]. Available: http://lik.unitomo.ac.id.
R. N. Wiwesa, “User Interface Dan User Experience Untuk Mengelola Kepuasaan Pelanggan,” Jurnal Sosial Humaniora Terapan, vol. 3, no. 2, 2021.
N. I. Arifin, H. Tolle, and I. R. Rokhmawati, “Evaluasi dan Perancangan User interface untuk Meningkatkan User experience menggunakan Metode Human-Centered Design dan Heuristic Evaluation pada Aplikasi Ezyschool,” urnal Pengembangan Teknologi Informasi Dan Ilmu Komputer, vol. 3, pp. 1725–1732, 2019.
Y. P. Budistria, Sulistiowati, and J. Lemantara, “Perancangan Desain User interface/User experience Sistem Informasi Akademik Mahasiswa (SIAM) dengan Metode Lean User Experience (Lean UX) pada Universitas Wr. Supratman,” Jurnal Sistem Informasi Universitas Dinamika, vol. 9, 2020.
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
0895634518464 
