Nowadays, many different GenAI applications can assist students in writing tasks. Assisting can be just brainstorming about ideas or helping structuring an essay, but it can also be writing the full length essay with minimum to no student intervention. The latter is probably not the smartest choice as probably every school institution has GenAI policies, usually prohibiting to present the AI’s work as your own product. That’s why I was surprised while reading the course manual of Digital Business, (part of the RSM’s BA program) last academic year. I needed to write two essays about digital topics, of which the first essay had to be co-written with a GenAI tool. I chose my familiar friend, ChatGPT, to assist me in taking on this challenge. Indeed, it became more of a challenge than I initially expected it to be.
A promising start
Every topic that the essay had to include was summarized in the assignment description. The mandatory first step was to copy/paste the description and use it as the first prompt for the GenAI application. At first glance, ChatGPT answered the prompt with already a quite impressive essay. But some things were not sufficient according to the essay description and my own preferences. The essay was way too long, did not include sources and it had more a kind of question-answer format. After this, I asked ChatGPT for more theory and explanations in order to develop my own desired theories that I wanted to include in the answer. This learning process was really nice as the interaction with the AI was on a deeper level: understanding core concepts.
Shift in focus
Eventually, I prompted my own desired answer with bullet points and let my companion write the full length essay. Here is where my focus shifted from understanding theory to more superficial tasks such as finding sources, citing, essay structure and much more. Finding sources and citations has caused me the highest levels of frustration, due to my wrong expectations of ChatGPT’s capabilities.
Problems, problems and more problems.
Initially, I thought that finding references with Chat was a piece of cake (so naïve, I know). During the article search process, I was already experiencing difficulties as many articles did not fully connect to the theories. After finally finding a suitable article, I came to the conclusion that ChatGPT has another very nice treat (not). When asking for citations from the article, ChatGPT liked to generate fictional citations that suited my text in the best way possible. Really sweet of my companion, but a bit illegal.
As I was dedicating so much time and prompts to article search and correcting the model if it was wrong again, the system started to slow down. With every input from my side, the lag increased. At the end, I had to wait minutes for the answer to finally show up.
Little did I know what kind of surprise I would receive at the end, just before handing in the essay. When reading the reference list one more time, I saw really weird looking website links. Immediately, I got a very bad feeling which was confirmed shortly after searching the internet: some articles did not even exist.
Lessons Learned
Luckily, I had some time (about two hours) left to erase the AI-generated articles and search for new existing articles. The latter turned out to be impossible given the time frame, so I had to rewrite or erase text in order to present it as my own work. Fortunately, I submitted the assignment in time and received a good grade for it
But the most important thing I received was a better understanding of ChatGPT’s capabilities and limitations. I actually overestimated its capabilities and faced the consequences of it. I spent way to much time on superficial interaction and the correction of the GenAI application, which was clearly visible in the interaction document I had to submit along with the final essay. The interaction document was basically a copy/paste of the entire interaction with the GenAI model. The length of my document was 140+ pages!
For the second essay, I tried a different approach because of this experience. Before prompting anything, I did my own research and searched for articles myself. I did use ChatGPT for understanding theory and later on for formatting, spelling and grammar correction. Remarkably, I received a lower grade for this essay but I did not really care as I saved much time. I had significantly reduced the number of interaction pages, thereby also reducing my frustrations and headaches.
Really interesting story about your experience with Generative AI, and in this case ChatGPT. Funny to hear that the experience also involved an RSM program, where you had to write an essay with the help of ChatGPT on purpose. Normally, you hear the opposite. I think this assignment was also a great way to help you learn which prompts work best, as you clearly saw that they could change the quality of the outcome. Your experience with bad or non-existing references sounds very familiar, as I’ve also experienced ChatGPT providing articles that supposedly supported my arguments, but in reality they didn’t. In the end, you realize that even though writing an entire essay together with ChatGPT might seem easier, the extra problems and quality checks might actually take more time than writing it without ChatGPT at all. This is really something to think about.
Such a nice and funny story. I had the same thing in some group projects when I checked the reference list and saw that some links still had the ChatGPT name in them. I also know the problem with fake sources very well. Now I only use ChatGPT to get ideas for search words when I look for real articles, and that works much better.
Hey Roald, I really liked reading your blogpost, it is very well written. I also had the course Digital Business last year and I ran into the same problems as you: ChatGPT made up fake sources, it used too many words, the text was superficial, etc.. But, during our second essay, I found a way to use AI that worked for me. I used it as a brainstorming tool. I asked it to give me arguments for and against a given question. This worked really well for me because I could put certain thoughts down on paper that I would have struggled with articulating without the use of AI. After reading this blog I do have a question for you: Do you feel like using AI in your way is ‘dumbing you down’? I personally feel like this does happen to me, when I use AI in the way I mentioned (a lot of other blogposts have written about this). So I’m curious if this happens to you too, with your limited use of ChatGPT during the writing of assignments, in particular that of Digital Business.
Hey Ravin, thank you for commenting! On one side, I think GenAI is dumbing me down. It is making me somewhat lazier as I do not search the entire internet very often: it is becoming my own kind of webbrowser. With one prompt everything is done for me. On the other hand, because of this fast information delivery it is possible to understand concepts quicker. Combined with the interactive trait of ChatGPT it is easy to ask for elaboration, extending my knowledge even faster.
But listening to my thoughts and feelings, I think my dominant view is that AI is dumbing me down. Recently, I caught myself in my thoughts: “I feel so dumb asking ChatGPT these things. Where did the time go when I did things completely on my own?”. I really felt I had become less creative and more lazy, making me thinking again about the use of GenAI. In which instances should I use GenAI in order to balance creativity and efficiency?