Amy Kong
The application of Generative Artificial Intelligence (GenAI) in various writing contexts has transformed the writing process from a sheer manual cognitive task into human-AI collaborative writing. This paradigm shift necessitates a reconceptualization of writing constructs within higher education to ensure that writing assessments accurately reflect the evolving knowledge and skills demanded in the authentic AI-assisted writing context. Grounded in the argument-based validity framework, this qualitative study investigates the domain definition inference of three university English writing courses in Hong Kong. Data collection involved a comprehensive analysis of construct artifacts, including course outlines, teaching materials, and assessment task specifications, supplemented by semi-structured interviews with instructors. The data were mapped against Cardon et al. (2023)’s AI literacy framework comprising four dimensions: application, authenticity, accountability, and agency. Findings reveal significant discrepancies in operationalization of AI literacy across the faculties. All courses exhibited construct under-representation in the application dimension due to a lack of instruction on iterative prompting and operational mechanisms of Large Language Models (LLMs). One case even demonstrated construct irrelevance by utilizing AI detection reports that inhibit human-AI collaboration. New assessment specifications are proposed in the end to bridge the gap between conventional instructional/assessment design and the requirements for effective AI-mediated writing.
