Virtual and augmented reality text environments support self-directed multimodal reading Interactive Learning Environments
Virtual (VR) and augmented reality (AR) technologies are two of the fastest-growing technologies anticipated to peak this decade, with a gap in research to understand the new multimodal affordances for teachers to support students’ self-directed interactive reading. This research aimed to understand early adolescents’ experiences of reading multimodal texts containing written and spoken language and 3D imagery in VR and AR text environments. The study applied thematic coding and multimodal interaction analysis to understand how students engaged with reading 3D, multimodal, non-fiction VR and AR texts in educational games. The findings attend to the following reading practices exhibited by learners: (i) connecting new knowledge to the known, (ii) recalling information; (iii) resolving cognitive disequilibrium, (iv) using haptic interactivity to navigate text pathways, and (v) using selective attention to filter information. The students followed varied text pathways while demonstrating cognitive disequilibrium and resolution when responding to multilayered attentional cues and text pathways, with students encountering multiple cuing systems with VR and AR texts that use different navigation aids than those offered in books. Reading multimodal texts across modes involved attending to several modes, while backgrounding others, with opportunities for extensive haptic interactivity, and requiring selective attention in extended reality environments.
Coding and computational thinking across the curriculum: A review of educational outcomes. RER
by Kathy A. Mills, Jen Cope, Luke Rowe
Teaching coding and computational thinking is an emerging educational imperative, now embedded in compulsory curriculum in the United States, Finland, the UK, Germany, Belgium, the Netherlands, New Zealand, and Australia. This meta-synthesis of 49 studies critically reviews recent international research (2009–2022) of coding and computational thinking as core and integrated across the curriculum. It addresses four essential problems: (a) What are the key features of learning environments that successfully develop students’ coding and computational thinking? (b) What is the impact of student engagement in coding and computational thinking on learning outcomes across curriculum areas? (c) What pedagogical constraints are evident for coding and computational thinking, including across curriculum areas? and (d) Which conceptual frameworks support coding and computational thinking, and what has been marginalized or excluded? The review advances knowledge of coding and computational thinking—vital to guide and develop future AI-based solutions to real-world problems that challenge disciplinary boundaries.
Virtual reality games for 3D multimodal designing and knowledge across the curriculum
Immersive virtual reality (VR) is anticipated to peak in development this decade bringing new opportunities for 3D multimodal designing across all levels of education. The need for students to gain capabilities with multimodal texts—texts that combine two or more modes, such as spoken, written, and visual—is emphasised at all levels of education from P-12 in the Australian Curriculum. Likewise, the use of technology-supported pedagogies is increasing worldwide, rendering multimodal texts ubiquitous across all knowledge domains. This original, qualitative classroom research investigated students’ 3D designing of multimodal texts using an immersive VR head-mounted display. Upper primary students (ages 10–12 years, n = 48) transferred their knowledge of ancient Rome through 2D drawing, writing, speaking, and 3D multimodal designing with VR. The application of multimodal analysis to video data, screen recordings, and think-aloud protocols, and the thematic coding of student and teacher interviews yielded four key findings: (i) VR gaming supported 3D multimodal designing through haptic and embodied experience, (ii) VR improved performance through creative redesigning, (iii) VR-supported knowledge application, consolidation, and transfer, and (iv) pedagogical strengths of VR were situated and transformed practice. This research is timely and significant given the increasing accessibility and affordability of VR and the need to connect research and pedagogical practice to support students’ advanced knowledge and capabilities with multimodal learning across the curriculum.
Smart glasses for 3D multimodal composition
by Kathy A. Mills, Alinta Brown
Extended reality technologies – mixed, augmented, and virtual reality, and future-related technologies – are rapidly expanding in many fields, with underexplored potentials for multimodal composition in digital media environments. This research generates new knowledge about the novel wearable technology – smart glasses – to support elementary students’ multimodal story authoring with 3D virtual objects or holograms. The researchers and teachers implemented learning experiences with upper elementary students from three classrooms to compose and illustrate written narratives before retelling the story with Microsoft HoloLens 2 smart glasses, selecting 3D holograms to illustrate the settings, characters, and events from the 3D Viewer software. The findings analyse how smart glasses supported students’ multimodal composition, and relatedly, the new modal resources available to students wearing smart glasses to compose 3D stories. The findings have significance for educators and researchers to understand and utilise the multimodal affordances of augmented and mixed reality environments for composing and storytelling.
Virtual Reality and Embodiment in Multimodal Meaning Making
Immersive virtual reality (VR) technology is becoming widespread in education, yet research of VR technologies for students’ multimodal communication is an emerging area of research in writing and literacies scholarship. Likewise, the significance of new ways of embodied meaning making in VR environments is undertheorized—a gap that requires attention given the potential for broadened use of the sensorium in multimodal language and literacy learning.
Virtual reality street art with L2 learners as multimodal literacy
By Kathy A. Mills, Alinta Brown, Amanda Guiterrez
This chapter explores how immersive virtual reality (VR) technologies, which are one of the highest-growth digital technologies this decade, can be used innovatively with culturally and linguistically diverse learners, enabling students to share and empathise with cultural stories in new virtual landscapes. This study provides an example of how L2 educators can use digitally supported pedagogies to engage students in culturally relevant multimodal literacy practices in ways that take up the multiliteracies’ call to transform pedagogy in digital communication environments. Artwork was created by the students in virtual subways and industrial warehouses using immersive VR technology, with each student contributing an individual element to the shared substrate. Multimodal designing in the VR environment provided L2 students with the opportunity to apply their knowledge of their own and others’ cultures, transforming their real-life experiences and new understandings into virtual street art.
Critical literacy and digital futures: Looking ahead
By Kathy Mills
This chapter offers a panoptic view of the future of critical literacy in digital spaces of the future. While reinforcing the aims of critical literacy in education, it raises five key arguments that anticipate the future of critical literacy for adolescent learners in digital times. First, it highlights the rapidly changing impact of AI-driven social media spaces, such as machine learning algorithms, filters, predictive analytics, and chatbots. Second, it argues for critical media literacy to counter the widened spread of fake news, dis-information, and misinformation on social media and the internet. Third, it draws attention to a new group of extended reality (XR) technologies – virtual (VR), augmented (AR), and mixed reality (MR), and technologies still in development – that bring new forms of interactive hybridity in 3D spaces, and which require hybrid forms of critical literacy. Fourth, it brings into focus a widening digital divide in the future in terms of critical algorithmic literacy that is following socio-economic and gendered fault lines, and the role of education in mitigating these effects. Fifth, it argues for continuing the important role of bringing critical literacy to life in the classroom to embrace popular texts across a growing range of everyday modes and media.
Quality in Big Qualitative Research. In Flick, U. Sage Handbook of Qualitative Research Quality
By Kathy Mills and Luke Rowe
This chapter explores the characteristics of good qualitative research when using big data, providing guidance for researchers to envisage the potentials for utilizing large digital data sets for qualitative research questions. The chapter provides a brief definition and conceptualization of big qualitative research. It then elucidates some of the key epistemic and theoretical assumptions about quality in big qualitative research. The chapter provides a set of essential tips for using big qualitative research with issues of quality at the forefront, giving useful examples from recent studies across a range of fields. Finally, the chapter identifies the limits and ethical principles to be maintained in big qualitative research, concluding with directions for future research.
What Counts as Language learning in a born-digital textual world?
By Kathy Mills
This chapter debates the central question: In an era of born-digital texts, what are the boundaries of language and literacy practices? Born digital texts are defined, following Selfe and Hawisher (2012) as texts that originate and live only in digital spaces, and which do not rely on alphabetic text as the principal mode of meaning. Even if born-digital texts are converted to print, they cannot be fully represented in this format. The chapter provides a rationale for the inclusion of born-digital texts in the language and literacy classroom, extending theories of multiliteracies, multimodality and the New Literacy Studies. It raises provocations regarding the born-digital practices of reading and text production that will count in recreational, educational and community spaces, and in the future literacies of Industry 4.0. The chapter concludes with a discussion of the challenges and implications for language and literacy pedagogy and curriculum, including issues of diversity and inclusion, and the scope and sequencing of born-digital practices for curriculum across levels of schooling. Recommendations are provided to advance language and literacy curriculum and research in the 21st century.
Literacy for Digital Futures. Mind, Body, Text
by Kathy A. Mills, Len Unsworth, and Laura Scholes.
Topics in chapters 1, 5-7, and 11 are related to research conducted through this Critical Pedagogies for AI and Extended Reality (XR) Technology project」, and list these chapter titles, as per below:
- Beyond Education for Industry 4.0: Next-Generation Literacies
- Embodiment and Digital Media Literacies
- Haptics and Motion in Literacy Practices with Digital Media
- Virtual, Augmented and Mixed Reality
- Conclusion: Multimaterial Literacies for Digital Futures
The unprecedented rate of global, technological, and societal change calls for a radical, new understanding of literacy. This book offers a nuanced framework for making sense of literacy by addressing knowledge as contextualized, embodied, multimodal, and digitally mediated.
In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes—Mind and Materiality; Body and Senses; and Texts and Digital Semiotics—to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice.
In today’s world of technological breakthroughs, social shifts, and rapid changes to the educational landscape, literacy can no longer be understood through established curriculum and static text structures. To prepare teachers, scholars, and researchers for the digital future, the book is organised around three themes—Mind and Materiality; Body and Senses; and Texts and Digital Semiotics—to shape readers’ understanding of literacy. Opening up new interdisciplinary themes, Mills, Unsworth, and Scholes confront emerging issues for next-generation digital literacy practices. The volume helps new and established researchers rethink dynamic changes in the materiality of texts and their implications for the mind and body, and features recommendations for educational and professional practice.」
Chapter “Big Data in Qualitative Research Methods” in SAGE Research Methods
by Kathy A. Mills
Big data has emerged in an era of heightened digital change, referred to in the literature and research since the turn of the twenty–first century. Big data refers to rapidly produced, digitally encoded information of variety, velocity, volume, value, and veracity (5 V’s) that is used as evidence for a phenomenon, commonly having relationality within data networks (Mills, 2019). While increasingly relevant to qualitative, quantitative, and mixed methods research, this article discusses big data in qualitative research—a field in which a large variety of nonstatistical data have always been central. Here, the meaning and significance of big data is established, providing examples for qualitative and mixed methods research. A brief history of big data is described, while identifying some of the key potentials and challenges of big data in qualitative research.
Chapter 7 Digital literacies in Literacy Theories for the Digital Age
Forward by David Howes and Chapter 7 Sensory Literacies
by Kathy A. Mills
Literacy Theories for the Digital Age insightfully brings together six essential approaches to literacy research and educational practice. The book provides powerful and accessible theories for readers, including Socio-cultural, Critical, Multimodal, Socio-spatial, Socio-material and Sensory Literacies. The brand new Sensory Literacies approach is an original and visionary contribution to the field, coupled with a provocative foreword from leading sensory anthropologist David Howes. This dynamic collection explores a legacy of literacy research while showing the relationships between each paradigm, highlighting their complementarity and distinctions. This highly relevant compendium will inspire researchers and teachers to explore new frontiers of thought and practice in times of diversity and technological change.
Handbook of Writing, Literacies, and Education in Digital Cultures
by Kathy A. Mills, Amy Stornaiuolo, Anna Smith, Jessica Zacher Pandya
At the forefront of current digital literacy studies in education, this handbook uniquely systematizes emerging interdisciplinary themes, new knowledge, and insightful theoretical contributions to the field. Written by well-known scholars from around the world, it closely attends to the digitalization of writing and literacies that is transforming daily life and education.
Conferences and Keynotes
Transmediation in Language Learning with XR Technology
Professor Mills was recently invited to the University of Munster, Germany to present a keynote on Transmediation in Language Learning with VR from the Senses Together project. The presentation was well-received by a large audience of graduate students staff on Wednesday 15th January.
IWCA Annual Fall Conference
Professor Kathy Mills is one of three keynote speakers at the IWCA Annual Fall Conference 2024 (International Writing Centers Association). She will be sharing new research on generative AI. Friday 25th, 9am-10:30 Brisbane time.
Critical Literacy for Generative AI: Implications for School Reforms
Have you ever wondered where Generative AI, brain-based learning and social media platforms are heading?
Professor Kathy Mills presented an invited keynote “Critical Literacy for Generative AI: Implications for School Reforms”
On September 18th at Madison, Wisconsin University, USA. The presentation was well-received by an appreciative audience of international academics and graduate students.
Future Directions for Critical literacy for CritLit Research Symposium
Mills presented an invited keynote called Future Directions for Critical literacy for CritLit Research Symposium, University of South-Eastern Norway, Sept 23, Tonsberg, Norway.
Mexico, SIELL Conference
For colleagues in Guadalajara, Mexico, Professor Kathy Mills was invited to present from the Senses Together project on The Future of the Digital Turn in the New Literacy Studies at del Instituto Transdisciplinar en Literacidad
At the University Centre of Art, Architecture and Design (CUAAD) on June 12th, 2023. The keynote was followed by a lively discussion about the challenges of being truly literate the digital age
University College London, Knowledge Lab and Research in Media Arts and Play
Professor Mills was invited to present from the Senses Together project on Smart Glasses for 3D Multimodal Design, Knowledge Lab in London, UK on June 28th, 2023. This event was well-attended, both in person and online, and was generously organised by Professor John Potter, the Director of ReMap.
Sheffield Hallam University, UK
In the UK, Professor Kathy Mills also presented from the Senses Together project on Using Mixed Reality for 3D Multimodal Literacy to the Language and Literacy in Education department at Sheffield Hallam University on June 22nd, 2023. The event was followed by a networking dinner and sparked new questions about extended reality technologies in early childhood and primary education in a digital age
ALEA Digital Synergy Symposium: Language, Literature and Literacy in a Digital Age, Queensland
Professor Kathy Mills was invited to present a Keynote on Smart glasses for 3D Multimodal Composition at the ALEA Digital Synergy Symposium: Language, Literature and Literacy in a Digital Age on Saturday 13th May, 2023, University of Southern Queensland, Springfield.
Contemporary Humanism International PhD Program Seminar, Italy
Professor Kathy Mills presented at the Contemporary Humanism International PhD Program Seminar hosted by the University of Rome, LUMSA, Italy, on Wednesday 22nd February, 2022.
Her virtual presentation shared from her research was entitled: The Future of the Digital Turn in the New Literacy Studies.
ALEA/AATE 2022 National Conference, Darwin
Professor Kathy Mills presented at the ALEA 2022 Conference Darwin, on Friday July 8th Her virtual presentation was shared on Day 1 from her ARC Future Fellowship was entitled: Virtual Reality Technology for indigenous Heritage Media.
Digital Reading and Writing Symposium
We hosted the “Digital Reading and Writing Symposium” and think tank to address key issues of digital change for literacy education, held at ILSTE on Thursday 14th July. This whole day event was enthusiastically attended by Department of Education, QLD staff, Big Picture Industries Inc, and academics from multiple ACU campuses nationally.
Digitality and Normativity in Informal and Formal Learning Contexts, Research Exchange Day, Germany.
Mills, K.A. (2022, March 31) Keynote Speaker: What is the Future of the Digital Turn in the New Literacy Studies?
Digital Symposium: Friedrich-Alexander-University, Erlangen-Nurnberg.
Multimodal Research International Conference, Auckland, New Zealand
Mills, K. A. (2019, December). Transmediation: Story writing to multimodal design in in virtual reality. Multimodal Research International Conference, Auckland, New Zealand.
Symposium conducted at the meeting of American Educational Research Association, Toronto, Canada
Mills, K. A. (2019, April). Transmediation in children’s virtual reality (VR) art. In D. Low (Chair). Symposium conducted at the meeting of American Educational Research Association, Toronto, Canada.
Critical Digital Literacies in a Sensory World. USN, Norway
Mills, K.A. Keynote Speaker (2020, Sept). Critical Digital Literacies in a Sensory World. USN, Norway.
Transmediation using Virtual Reality Technology, Cyprus
Mills, K.A. Keynote Speaker (2021, March 12-13). Transmediation using Virtual Reality Technology. 4th International Conference Literacy and Contemporary Society, Cyprus.
AATE/ALEA National Conference, Challenge & Change: Contemporary Literacy & English Teaching, Brisbane.
Mills, K.A. Keynote Speaker (2021, July 7-9) . The Future of the Digital Turn in the New Literacy Studies. AATE/ALEA National Conference, Challenge & Change: Contemporary Literacy & English Teaching, Brisbane.
Australian Literacy Educators Association
Mills, K.A. Keynote Speaker (2020, March). The Role of the Senses in Multimodal Design: Stories from Culturally Diverse Classrooms, Australian Literacy Educators Association, June 29.