Reconceptualizing the target domain of AI-assisted Writing: A multi-case analysis of AI literacy integration in writing courses in Hong Kong universities&nbsp;

doi:10.65961/AJELT-2026-4-004

PDF(517 KB)

Asian Journal of English Language Teaching ›› 2026, Vol. 35 ›› Issue (4) : 51-79. DOI: 10.65961/AJELT-2026-4-004

Reconceptualizing the target domain of AI-assisted Writing: A multi-case analysis of AI literacy integration in writing courses in Hong Kong universities

Author information +

History +

Abstract

The application of Generative Artificial Intelligence (GenAI) in various writing contexts has transformed the writing process from a sheer manual cognitive task into human-AI collaborative writing. This paradigm shift necessitates a reconceptualization of writing constructs within higher education to ensure that writing assessments accurately reflect the evolving knowledge and skills demanded in the authentic AI-assisted writing context. Grounded in the argument-based validity framework, this qualitative study investigates the domain definition inference of three university English writing courses in Hong Kong. Data collection involved a comprehensive analysis of construct artifacts, including course outlines, teaching materials, and assessment task specifications, supplemented by semi-structured interviews with instructors. The data were mapped against Cardon et al. (2023)’s AI literacy framework comprising four dimensions: application, authenticity, accountability, and agency. Findings reveal significant discrepancies in operationalization of AI literacy across the faculties. All courses exhibited construct under-representation in the application dimension due to a lack of instruction on iterative prompting and operational mechanisms of Large Language Models (LLMs). One case even demonstrated construct irrelevance by utilizing AI detection reports that inhibit human-AI collaboration. New assessment specifications are proposed in the end to bridge the gap between conventional instructional/assessment design and the requirements for effective AI-mediated writing.

Key words

GenAI-assisted writing, AI literacy, writing, argument-based validity framework

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

Amy Kong. (2026). Reconceptualizing the target domain of AI-assisted Writing: A multi-case analysis of AI literacy integration in writing courses in Hong Kong universities .Asian Journal of English Language Teaching , 35(4): 51-79. https://doi.org/10.65961/AJELT-2026-4-004

References

Abdaljaleel, M., Barakat, M., Alsanafi, M., Salim, N. A., Abazid, H., Malaeb, D., ... & Sallam, M. (2024). A multinational study on the factors influencing university students’ attitudes and usage of ChatGPT. Scientific Reports, 14(1), 1983. https://doi.org/10.1038/s41598-024-52549-8
Adel, A., & Alani, N. (2025). Can generative AI reliably synthesise literature? exploring hallucination issues in ChatGPT. AI & Society, 40(8), 6799-6812. https://doi.org/10.1007/s00146-025-02406-7
Ansari, A. N., Ahmad, S., and Bhutta, S. M. (2023). Mapping the global evidence around the use of ChatGPT in higher education: a systematic scoping review. Education and Information Technologies, 29, 11281–11321. https://doi.org/10.1007/s10639-023-12223-4
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., Do, Q. V., Yan, X., & Fung, P. (2023). A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. arXiv. https://doi.org/10.48550/arXiv.2302.04023
Bearman, M., Tai, J., Dawson, P., Boud, D., & Ajjawi, R. (2024). Developing evaluative judgement for a time of generative artificial intelligence. Assessment & Evaluation in Higher Education, 49(6), 893-905. https://doi.org/ 10.1080/02602938.2024.2335321
Cardon, P., Fleischmann, C., Aritz, J., Logemann, M., & Heidewald, J. (2023). The challenges and opportunities of AI- assisted writing: Developing AI literacy for the AI age. Business and Professional Communication Quarterly, 86(3), 257- 295. https://doi.org/ 10.1177/23294906231176517
Cardon, P., Fleischmann, C., Logemann, M., Heidewald, J., Aritz, J., & Swartz, S. (2024). Competencies needed by business professionals in the AI age: Character and communication lead the way. Business and Professional Communication Quarterly, 87(2), 223-246. https://doi.org/10.1177/23294906231208166
Chappelle, C.A., & Enright, M. K., & Jamieson, J. M. (2008). Building a validity argument for the test of English as a foreign language. Routledge. https://doi.org/ 10.1080/15434303.2010.519418
Chapelle, C. A., & Lee, H. W. (2021). Conceptions of validity. In G. Fulcher & L. Harding (Eds.), The Routledge handbook of language testing (2nd ed., pp. 17–31). Routledge. https://doi.org/10.4324/9781003220756
Chui, M., Hazan, E., Roberts, R., Singla, A., Smaje, K., Sukharevsky, A., Yee, L., & Zemmel, R. (2023). The economic potential of generative AI: The next productivity frontier. McKinsey. https://www.mckinsey.com/capabilities/tech-and- ai/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier
Davenport, H., & Mittal, N. (2022). How generative AI is changing creative work. Harvard Business Review. https://hbr.org/2022/11/how-generative-ai-is-changingcreative-work
Deng, C., Lin, S., Tong, E. K., & Ma, J. (2026). Leveraging hybrid intelligence: How teacher agency influences behavioural engagement in human-AI collaborative writing feedback practices. International Journal of TESOL Studies, 8 (1), 82-107. https://doi.org/10.58304/ijts.251108
Deng, J., & Lin, Y. (2023). The benefits and challenges of ChatGPT: An overview. Frontiers in Computing and Intelligent Systems, 2(2), 81-83. https://doi.org/10.54097/fcis.v2i2.4465
Dursun, A., & Li, Z. (2021). A systematic review of argument-based validation studies in the field of language testing (2000–2018). In C. A. Chapelle & E. Voss (Eds.), Validity argument in language testing: Case studies of validation research (pp. 45–70). Cambridge University Press.
Estacio, R. D. (2025). AI integration in research writing: A quantitative study among students and teacher-researchers toward a human-AI model. Journal of Institutional Research South East Asia., 23(3), 373–392.
Getchell, K., Carradini, S., Cardon, P. W., Aritz, J., Fleischmann, C., Stapp, J., & Ma, H. (2022). Artificial intelligence in business communication: The changing landscape of research and teaching. Business and Professional Communication Quarterly, 85(1), 7-33. https://doi.org/10.1177/23294906221074311
Guo, B., Zhang, X., Wang, Z., Jiang, M., Nie, J., Ding, Y., Yue, J., & Wu, Y. (2023). How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection. arXiv. https://doi.org/10.48550/arXiv.2301.07597
Haan, K. (2023, April 4). How businesses are using AI in 2023. https://www.forbes.com/advisor/business/software/ai- in-business/
Han, J., Yang, Y., & Liu, G. (2025). Are teachers assessing work written by students or by AI? A rapid literature review of research on detecting content generated by Generative AI. European Journal of Education, 60(4), e70240. https://doi.org/ 10.1111/ejed.70240
Hwang, Y., Lee, J. H., & Shin, D. (2023). What is prompt literacy? An exploratory study of language learners’ development of new literacy skill using generative AI. arXiv. https://doi.org/10.48550/ arXiv.2311.05373
Ibrahim, H., Liu, F., Asim, R., Battu, B., Benabderrahmane, S., Alhafni, B., et al. (2023). Perception, performance, and detectability of conversational artificial intelligence across 32 university courses. Scientific Reports, 13, 12187. https://doi.org/10.1038/s41598-023-38964-3
Johnson, M. S. (2025). Responsible AI for measurement and learning: Principles and practices (Research Report No. RR- 25-03). ETS. https://www.ets.org/Media/Research/pdf/RR-25-03.pdf
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of educational measurement, 50(1), 1-73. https://doi.org/10.1111/jedm.12000
Ken, H. (2026). AI feedback to L2 writers. International Journal of TESOL Studies, 8 (1), 146-156. https://doi.org/10.58304/ijts.260420
Kim, J., Yu, S., Lee, S. S., & Detrick, R. (2025). Students’ prompt patterns and its effects in AI-assisted academic writing: Focusing on students’ level of AI literacy. Journal of Research on Technology in Education, 1-18. https://doi.org/10.1080/15391523.2025.2456043
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024). AI literacy and its implications for prompt engineering strategies. Computers and Education: Artificial Intelligence, 6, 100225. https://doi.org/10.1016/j.caeai.2024.100225
Kong, A. (2021). A dual case study of perceptions of strategy-based training and peer review stances among Hong Kong secondary students: An activity theory perspective. In B. L. Reynolds & M. F. Teng (Eds.), Innovative Approaches in Teaching English Writing to Chinese Speakers (pp. 43-67). De Gruyter Mouton. https://doi.org/10.1515/9781501512643
Kong. (2024). Examining the argument-based validity of the HKDSE speaking test by investigating the domain definition, explanation, and utilization inferences. Asian Journal of English Language Teaching, 33(1), 53-86. https://doi.org/10.65961/AJELT-2024-1-004
Kong, A., & Bui, G. (2019). Reader stances and writer responses in L2 peer review: A study of L2 writing literacy among Hong Kong secondary school students. Asian EFL Journal, 21(3), 140-186. https://www.asian-efl- journal.com/main-journals/2019-main-journal/volume-23-issue-5-2019/
Kong, A., & Teng, M. F. (2023). The operating mechanisms of self-efficacy and peer feedback: An exploration of L2 young writers. Applied Linguistics Review, 14(2), 297-328. https://doi.org/10.1515/applirev-2020-0019
Lee, S.S., Li, N., & Kim, J. (2024). Conceptual model for Mexican teachers’ adoption of learning analytics systems: The integration of the information system success model and the Technology Acceptance Model. Education and Information Technologies, 29(11), 13387–13412. https://doi.org/10.1007/s10639-023-12371-7
Legislative Council Secretariat. (2025, October). Measures to support development of AI education tools in the United Kingdom and Singapore (Essentials ISE11/2025). Research Office, Research and Information Division, Legislative Council Secretariat.
Li, H., & Pan L. (2024). Network of discourses: resistance and negotiation within Chinese students’ AI-assisted EFL writing. International Journal of TESOL Studies, 6 (3), 128-145. https://doi.org/10.58304/ijts.20240309
Lin, H. C. K. (2025). Writing is coding: When writing becomes the starting point for AI cross-media learning and narrative. International Journal of Online Pedagogy and Course Design (IJOPCD), 15(1), 1-23. https://doi.org/10.4018/IJOPCD.385016
Long, D., & Magerko, B. (2020, April). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1-16). https://dl.acm.org/doi/10.1145/3313831.3376727
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13–103). Macmillan Publishers.
Mollick, E. (2023, September 16). Centaurs and cyborgs on the jagged frontier. Retrieved from https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged
Nelson, D. B., Gackiere, A. E., LeGrand, S. E., & Guberman, D. A. (2025). Collaborative intelligence: Towards practical, critical and cooperative teaching & learning with AI. Thresholds in Education, 48(1), 57-80.
Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187-192. https://doi.org/ 10.1126/science.adh2586
Sawaki, Y., & Sinharay, S. (2013). Investigating the value of section scores for the TOEFL iBT® tetT. ETS Research Report Series, 2, i–113. https://doi.org/10.1002/j.2333-8504.2013.tb02342.x
Singla, A., Sukharevsky, A., Hall, B., Yee, L., & Chui, M. (2025). The state of AI in 2024-2025: Transitioning from experimentation to impact. McKinsey & Company. https://www.mckinsey.com/capabilities/quantumblack/our- insights/the-state-of-ai

Park, M. (2021). Domain definition inference for virtual interactive Aviation English Test (VIAET) for military air traffic controllers. In C. A., Chapelle & E. Voss (Eds.), Validity argument in language testing: Case studies of validation research (pp. 73-95). Cambridge University Press. https://doi.org/10.1017/9781108669849.007

Ren, J., & Wang, W. (2025). Assisting research proposal writing with large language models: Evaluation and refinement arXiv. https://doi.org/ 10.48550/arXiv.2509.09709
Rice, R. (2025). Prompting purposefully: Teaching AI rhetorical literacy in grant writing. Small Bytes Blog Post Submission | Fall 2025. Retrieved from https://www.depts.ttu.edu/tlpdc/ai-resources/small-bytes-Rich-Rice.pdf
Teng, M. F. (2024). “ChatGPT is the companion, not enemies”: EFL learners’ perceptions and experiences in using ChatGPT for feedback in writing. Computers and Education: Artificial Intelligence, 7, 100270. https://doi.org/10.1016/j.caeai.2024.100270
Teng, M. F. (2025). Understanding EFL student writers’ metacognitive awareness in utilizing ChatGPT. System, 135, 103848. https://doi.org/10.1016/j.system.2025.103848
Teng, M. F. (2026). Validation and profiling of GenAI-mediated vocabulary learning strategies. Asian Journal of English Language Teaching, 35(3), 15-39. https://doi.org/10.65961/AJELT-2026-3-002
von Garrel, J., and Mayer, J. (2023). Artificial intelligence in studies—use of ChatGPT and AI-based tools among students in Germany. Humanities and Social Sciences Communications, 10, 799. https://doi.org/10.1057/s41599-023- 02304-7
Wang, C. (2025). Exploring students’ generative AI-assisted writing processes: Perceptions and experiences from native and nonnative English speakers. Technology, Knowledge and Learning, 30(3), 1825-1846. https://doi: 10.1007/s10758- 024-09744-3
Wen, H., & Hu, P. (2025). Architectural gaps in generative AI: Quantifying cognitive risks for safety applications. AI, 6(7), 138. https://doi.org/10.3390/ai6070138
Williams, T. (2023, February 25). Some companies are already replacing workers with ChatGPT, despite warnings it shouldn’t be relied on for “anything important.” Fortune. https://fortune.com/2023/02/25/companies-replacing- workers-chatgpt-ai/
Zapata, G. C., Saini, A., Tzirides, A. O. O., Cope, W., & Kalantzis, M. (2024). The role of AI feedback in university students’ learning experiences: An exploration grounded in activity theory. Ubiquitous Learning, 18(2), 1. https://doi.org/ 10.18848/1835-9795/CGP/v18i02/1-30
Zhang, Y. (2025). GPT is all you need: AI, cognitive scaffolding, neuroplasticity and AI, AI-human interaction. Frontiers in Psychology, 16, 1549755. https://doi.org/10.3389/fpsyg.2025.1549755