Exploring the structure of digital literacy competence assessed using authentic software applications

Abstract

Digital literacy competence (DL) is an important capacity for students' learning in a rapidly changing world. However, little is known about the empirical structure of DL. In this paper, we review major DL assessment frameworks and explore the dimensionality of DL from an empirical perspective using assessment data collected using authentic software applications, rather than simulated assessment environments. Secondary analysis on representative data collected from primary and secondary school students in Hong Kong using unidimensional and multidimensional item response theory reveals a general dimension of digital literacy performance and four specific, tool-dependent dimensions. These specific DL dimensions are defined by the software applications that students use and capture commonality among students’ performance that is due to their familiarity with the assessment tools and contexts. The design of DL assessment is discussed in light of these findings, with particular emphasis on the influence of the nature of digital applications and environments used in assessment on the DL achievement scores measured.