The content production industry has undergone a revolutionary transformation in recent years, primarily due to significantly increased investments and speed of development in generative AI models. These models can now write code, create art, and write extensive and concise bodies of text. In this article, we will delve into the mystery behind why some of these models appear to be struggling when user text prompts request them to cite the sources used for their output.
Every student’s dream is an AI tool that can write an extensive and comprehensive essay and perfectly do in-text and bibliographical citations, without leaving detectable traces. In my experience, it would have been helpful if it could just recommend scientific papers based on a topic of your choice.
I have attempted to work with OpenAI’s ChatGPT to do this before and I thought it would be interesting to explore this topic so I have decided to try out 3 competing products of generative AI namely Bard, ChatGPT (3.5), and Bing AI.
First I asked Bing’s AI “Why can’t Bard AI list references”. To my surprise without asking for it, Bing AI included in-text citations referring to the websites it took its information from. It further provided other links to learn more about the topic of my question. The website it consistently referred to in its response was legitimate. Additionally, the other websites it mentioned in the “Learn more” box also exist and contain valuable content for further learning.
Then I moved on to Google’s chatbot Bard and asked it a question to see if its approach to answering questions is similar to Bing or if it will hide the sources it uses from users. Indeed, in line with Schwartz (2023), the answer Bing AI gave me, Bard did not seem to list its citations. So, I engaged in further interactions with it and after quite a while, when I asked, “How can generative AI be used for marketing,” it eventually listed two websites as its sources. These websites indeed exist, and once again, similar to Bing AI’s response, Bard only cited the specific website from which it directly quoted.
Lastly, I wanted to talk about my experience with ChatGPT and how OpenAI’s chatbot refers to articles it has used to generate content. Throughout the second half of last year, I have frequently engaged in interactions with ChatGPT to assist me with virtually anything. This included asking for information about the articles that were referenced in generating its responses, as well as requesting recommendations for additional research materials. And I had to quickly realize its capabilities are very limited in this specific domain. To showcase what I mean, I prompted the bot to write an essay and then I gave it instructions to list the sources it has used. The bot kept saying it was incapable of doing so, and that I should look for them myself, but I was not satisfied with this answer, and I kept asking in different ways but to no avail.
After tens of minutes of frustrating conversation, I have finally managed to persuade ChatGPT to reference the sources it has used. Nevertheless, as you can see in the picture below, after all that time invested, the reference list it provided was of zero value.
Based on my personal experience, the most powerful tool would be one that combines the capabilities of the competing chatbots, maintaining the creativity and criticality of ChatGPT and referencing abilities of Bing – I did not like Bard that much. While writing this article yesterday (27-09-2023), I stumbled across an article – written by Davis (2023) that piqued my interest. OpenAI just announced on Twitter that ChatGPT 4 will be able to access the internet through the new “Browse with Bing” feature available for premium subscribers, he reports. For those interested, the post can be found at the following link: https://twitter.com/OpenAI/status/1707077710047216095?s=20
References:
Schwartz, B. (2023, March 22). Google explains why Bard rarely lists citations and links to content creators. Search Engine Land. https://searchengineland.com/google-explains-why-bard-rarely-lists-citations-and-links-to-content-creators-394635
Davis, W. (2023, September 27). CHATGPT can now search the web in Real time. The Verge. https://www.theverge.com/2023/9/27/23892781/openai-chatgpt-live-web-results-browse-with-bing
Your article perfectly expresses the typical annoyance that many of us have with generative AI’s reference skills. In contrast to ChatGPT’s existing limitations, it is clear that tools like Bing AI and Bard provide more trustworthy citations. Your concept of a hybrid strategy that combines ChatGPT’s originality and Bing’s referencing skills looks viable and optimistic. A potential solution to the reference problem is hinted at by the fact that ChatGPT 4 will have an internet connection via Bing. The evolution of generative AI depends on this constant discussion of its shortcomings and developments. We can only hope that as these developments advance, AI-generated information will become more trustworthy and respectable for scholarly and research uses. Thank you for sharing your experiences and for these great insights!