Researchers find AI-written content in 13.5% of 2024 biomedical papers
A new study reveals large language models (LLMs) like ChatGPT are quietly shaping scientific writing. Researchers from the U.S. and Germany analyzed 15 million biomedical abstracts on PubMed and found signs of AI influence in over 13% of papers published in 2024.
The team tracked word usage shifts before and after ChatGPT’s release. They spotted a big jump in "stylistic and flowery" words like "showcasing," "pivotal," and "grappling." Typical content-heavy words declined. They also found verbs and adjectives replaced many nouns as dominant excess words this year. 
Past attempts to detect AI in academic texts stumbled on biases from assuming specific AI models or prompts. This study sidestepped that by focusing solely on excess word use patterns, borrowing a "before-and-after" method from COVID-19 mortality research analysis.
Differences in AI usage appeared across research fields, countries, and publication venues. Authors estimate at least 13.5% of 2024 biomedical papers had some LLM involvement.
The results landed in the open-access journal Science Advances.
Concerns about AI-generated content spreading unchecked through peer-reviewed science keep growing as LLMs like ChatGPT and Google Gemini soar in popularity.
Charles Blue, lead author of the Science X article, credits careful human monitoring to avoid the very AI influence they study.
90 are shown at r = 90. Excess words were manually annotated into content words (blue) and style words (orange). (B) The same but with frequency gap (δ) as the vertical axis. Words with δ > 0.05 are shown at δ = 0.05. Credit: *Science Advances* (2025). DOI: 10.1126/sciadv.adt3813″ />
More on the study:
Dmitry Kobak et al, Delving into LLM-assisted writing in biomedical publications through excess vocabulary, Science Advances (2025). DOI: 10.1126/sciadv.adt3813
