The Literary Quality of ChatGPT's Productions

Manuscript received November 16, 2024; accepted January 17, 2025; published April 16, 2025

Abstract—ChatGPT is capable of writing extended passages in natural language in response to prompts. This raises the important problem of trying to evaluate the literary quality of these productions. Can this quality be improved or are there inherent limitations? This paper attempts a preliminary investigation of this problem by focussing on just one kind of literary production, namely poetry. The question is related to Turing's classic 1950 paper in which he introduced the Artificial Intelligence (AI) project. Turing considers the example of whether computers can fall in love or enjoy strawberries and cream. Accordingly, the present paper gives a poem about love and strawberries and cream written by a human (an Elizabethan poet) and compares it to a poem on the same subject by ChatGPT. It is then argued that ChatGPT produces some poems which, because they are lacking in authenticity, are skilful imitations of good poems rather than good poems. This is contrasted with AI generated proofs in mathematics which establish the truth of theorems as well, if not better, than the proofs of human mathematicians.

keywords—Turing, human poems, computer poems, human mathematics, computer mathematics

Cite: Donald Gillies, "The Literary Quality of ChatGPT's Productions," Journal of Advances in Artificial Intelligence, vol. 3, no. 2, pp. 122-131, 2025. doi: 10.18178/JAAI.2025.3.2.122-131

Copyright © 2025 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).