Keith Webster tweeted out awareness of a blog post authored by Marjorie M.K. Hlava of Access Innovations. Posted to the Scholarly Kitchen, Hlava documents her experiences in asking three systems to rewrite a draft article and add metrics and citations, using Claude, ChatGPT, and Google Gemini.
The results are WILDLY different. The amount of time needed for each attempt varied. During the interactions I further asked each for reference citations and metrics to add to the article.
As a sample of her experience:
Just for interest, I asked ChatGPT to give a reference to an article by Hlava (written by me). It came back with something I did not write in a journal that does exist, and of course the DOI does not check out.
Hlava, M. (2014). “The Importance of Metadata in Academic Publishing.” Journal of Scholarly Publishing, 45(4), 397-409. DOI: 10.3138/jsp.45.4.397
ChatGPT did the least reframing, reformatting, and adding of extraneous material, but the citations were a joke
Was It Worth The Effort?
Trying to Write a Paper with LLM Assistance - an experiment with Claude, Gemini and ChatGPT #AI via @scholarlykitchen.bsky.social scholarlykitchen.sspnet.org/2025/03/11/g...