Skip to main content
Trying to Write a Paper with LLM Assistance

Trying to Write a Paper with LLM Assistance

March 2025

What's Happening Here?

Keith Webster tweeted out awareness of a blog post authored by Marjorie M.K. Hlava of Access Innovations. Posted to the Scholarly Kitchen, Hlava documents her experiences in asking three systems to rewrite a draft article and add metrics and citations, using Claude, ChatGPT, and Google Gemini. 

The results are WILDLY different. The amount of time needed for each attempt varied. During the interactions I further asked each for reference citations and metrics to add to the article.

As a sample of her experience:

Just for interest, I asked ChatGPT to give a reference to an article by Hlava (written by me). It came back with something I did not write in a journal that does exist, and of course the DOI does not check out.

  • Hlava, M. (2014). “The Importance of Metadata in Academic Publishing.” Journal of Scholarly Publishing, 45(4), 397-409. DOI: 10.3138/jsp.45.4.397

ChatGPT did the least reframing, reformatting, and adding of extraneous material, but the citations were a joke

Was It Worth The Effort?

Trying to Write a Paper with LLM Assistance - an experiment with Claude, Gemini and ChatGPT #AI via @scholarlykitchen.bsky.social scholarlykitchen.sspnet.org/2025/03/11/g...

[image or embed]

— Keith Webster (@cmkeithw.bsky.social) March 11, 2025 at 8:42 AM