The Epistemic Downside of Using LLM-Based Generative AI in Academic Writing

publications The Epistemic Downside of Using LLM-Based Generative AI in Academic Writing

There is now widespread use of large language model-based generative artificial intelligence tools in academic research and writing. While these are convenient, quick, and output enhancing, they also arguably incur ethical issues, such as questionable authenticity and plagiarism. Here, I explore epistemological aspects of AI use in academic writing and posit that there is evidence for three related pitfalls in AI use that should not be ignored. These include one, epistemic detriment or harm in terms of illusions of understanding, two, potential for cognitive dulling or impairment, and three, AI dependency both habitual and or emotional. Thus, any potential infringements of academic ethics aside, AI use in academic writing incurs intrinsic problems that are epistemic in nature. These epistemic downsides call for restraint and moderation beyond regulatory measures to address ethical issues in AI use.

One. Introduction

Large language model-based generative artificial intelligence agents have dramatically changed the way academic research is being pursued and reported. It is undeniable that AI agents have greatly facilitated research and development in many subfields of science, engineering, as well as the arts and humanities. For example, in accordance with the notion of having an "AI scientist" within the context of a laboratory or research group, large language models has been proposed to become de facto "Co-PIs" in research tasks ranging from literature triage to hypothesis generation. However, the use of such agents in academic writing has been controversial, largely because of perceptions of unoriginality and nontransparency, impediments such as AI-based plagiarism or Aigiarism, a portmanteau of 'AI' and 'plagiarism', as well as inaccuracies and fabrications resulting from AI hallucination. Earlier attempts at the inclusion of AI agents such as ChatGPT as coauthors in academic papers were quickly outlawed, and most journals and publishers have issued guidelines pertaining to the disclosure or declaration of AI use in academic writing.

Despite the prevailing guidelines, the uptake of full compliance is, perhaps not unexpectedly, somewhat untenable. Several studies have revealed signs of fairly widespread undeclared use of AI on published works and preprints. It is perhaps unsurprising that computer science papers should have AI-generated content, but the American Association for Cancer Research has also found a significant number of AI-generated manuscript abstracts and peer review reports, many of which are undisclosed. The percentages of cases highlighted in the reports above likely underestimated the prevalence of AI writing in citable published works and preprints. This is because AI crafted text is difficult to distinguish from those written by humans and AI detection software is not particularly reliable.

Should we even be against the extensive use of AI in academic writing? A recent survey by Nature showed that researchers are split in their views. Although the issue remains controversial, given the current trend and rapid development, it might be a common perception that "resistance is futile" and one should thus either embrace AI or risk falling behind. Issues of academic ethics associated with AI use have been extensively discussed, and large language model-based generative transformers are becoming less error prone. A particularly strong argument made by proponents of AI use in academic writing is that AI use will improve language dexterity that would facilitate academic communication for the less privileged, and the leveling of the competitive playing field for non-native users of English. Even prominent scholars in the field who were cautious in drafting earlier AI-use disclosure guidelines have reversed their stance to propose that such disclosures should be voluntary.

Research and academic ethics or integrity considerations notwithstanding, it is important to recognize generative AIs such as large language model-based generative pretrained transformers hereafter AI for short as what they are meant to be. Alvarado, for example, has articulated that AI is an epistemic technology which is "primarily designed, developed and deployed to be used in epistemic contexts". In essence, AI is made to be, and should be, facilitating the acquisition of knowledge. However, is this notion always true? I posit that while this would apply for many aspects of academic research in general, epistemic pitfalls lurk in the specific context of academic writing. I refer to academic writing in a broad sense, from students writing term papers and project reports to academic researchers at all levels writing their research papers. Below, I shall first review the findings and evidence for these epistemic downsides and then discuss why it is still best to do, or at least intellectually self-anchor, one's own writing in academic work.