Picture: (Meier, 2023)
Context
Generative AI has gained enormous attraction recently, quickly intervening in a multitude of our life’s aspects. As its presence has expanded rapidly, a vast range of private as well as business use cases is being explored. Out of curiosity about its potential to provide value for businesses, I have attempted to assess its use in project management in an environment close to my heart; student consulting (CoinTelegraph, 2023).
As part of the board of a student consulting firm, our branch is part of a global organization aiming to help social impact organizations by providing affordable to free services while offering students the chance to develop themselves and gain practical experience. Thus, as part of this model, all organizational members worldwide are volunteers. Therefore, as our office has been in the process of vast growth, resource limitations and corresponding allocation are crucial for operations to run sustainably. A process that remains risky and could prove to be a recurring bottleneck each cycle is that of new client recruitment. Here, one of the most time-consuming tasks in that of developing project briefs.
In such briefs, a description of the client is outlined and, most importantly, the scope of the project is laid out and tied to a first conceptualization of tasks that will likely need to be undertaken. Not only does such a document aid in the following stages of the engagement but it also helps to assess feasibility and matching with internal capabilities. Moreover, it lays a foundation for the student consultants and project leader to start assigning tasks. Finally, those project briefs of engaged leads are sanitized and absorbed into a globally administered knowledge- and database. This is why there is a consistent template and structure to the briefs, which is reused in the small-scale simulation run for this blog post.
Experimentation
Hence, also minding the security of proprietary data, three highly sanitized versions of past project initiations were used in order to ask ChatGPT to develop a project scope for a hypothetical brief. Here, copied text from an initiation via each of three common channels was used, being an incoming e-mail, an interest registration form submission and explorative meeting notes. All of these contain information about the background, underlying problems and needs of the potential client organization.
After providing ChatGPT with the overarching structure of our project brief template and a description of the intention of developing the document, I asked it to concisely write out a project scope that focuses on topics on the one side while keeping in mind underlying client challenges on the other in order to test how close it comes to how humans may write out a brief.
Impressions
What stands out, despite the natural differences in project topics across the three examples, is the apparent correlation of the developed scopes with the input given to ChatGPT. While generally impressive in generated scopes even with limited input lengths, it seems to skew towards how much a certain subtopic is mentioned in the text. For example, if a client stated it is developing a new product and wants to develop a promotional strategy, the developed scope does not get into detail on that most likely the product attributes and consumer preferences need to be researched unless specifically told so in a follow-up prompt. This is where, acknowledging the unexplainability of the black box, there seems to be some difference with scoping based on inserted meeting notes. These longer texts already contain more practical and human thoughts on what implications the wants of a client have on the project tasks further down the line. Despite this higher level of comprehensiveness and accuracy, it does seem sensitive to how much text is placed under a section in the meeting notes, with more a lengthy notes section on seeking partnerships taking up considerably more of the developed scope rather than the asked-for industry trends analysis, even though later in the notes it is stated that only two out of six people on the team would likely work on this matter.
In conclusion, the potential time-savings are great and potential is high. Nevertheless, providing a structure to follow may not be not enough for cases where such a model is not internally governed. In comparison to self-written scopes for these past projects, it seems difficult for ChatGPT to accurately attribute proportional attention and consequential thinking to given inputs even if asked for. This underwrites the role of human intervention in value extraction from this tool. Beyond being sensitive to the textual inputs given which may have to be balanced out in the future, it remains very sensitive to prompt writing. Clear and repetitive specification of how and for what purpose ChatGPT should filter and use the information does guide it in the right direction, which may explain why it performed better in converting the contextualized meeting notes rather than a very brief section in a form. Moreover, while not being conclusive about the written scopes after using chained prompting, this deliberate use did enable later follow-up questions without having to repeat what was first indicated in a first contextual-task prompt (StakeholderMap, 2023).
Ultimately, even when contextualization and clearly bounded expectations are part of the prompt writing, first impressions are that it remains hard to control for its biases. Therefore, its practical use for project brief writing appears to still require human intervention, at which stage those involved may still likely want to retain control over its full development, even if time-consuming. This is reiterated elsewhere, concluding it has a support function in developing documentation for project management if adjusted and reviewed manually to match its original purpose (Parm, 2023). At least for now, it may remain more of an efficiency booster rather than automation, although it may be enhanced in combination with project software. Additionally, further experimentation may explore to what extent asking ChatGPT how it assesses the comprehensiveness of the information put in, also allowing it to ask ourselves questions about what it would like to know more of to perhaps output more contextualized and less biased scopes (Blore, 2023).
In case of any thoughts on experimenting further with this use case as part of managing projects, how to better control for its potential input sensitivity biases or when having better alternatives in mind, feel free to let me know in the comments!
References
Blore, Z. (2023, June 20). Unleash the Power of ChatGPT: The Ultimate Game-Changer for Project Management! Retrieved from SimplifyChange: https://www.simplifychange.co.uk/our-blog/unleash-the-power-of-chatgpt-the-ultimate-game-changer-for-project-management
CoinTelegraph. (2023, July 10). How to use ChatGPT for project management. Retrieved from CoinTelegraph: https://cointelegraph.com/news/chatgpt-for-project-management
Meier, K. (2023, May 30). How to write a project scope document from scratch. Retrieved from teamwork.com: https://www.teamwork.com/blog/project-scope-template/
Parm. (2023). How can ChatGPT help in Project Management? Retrieved from Parm: https://parm.com/en/chatgpt-in-project-management/
StakeholderMap. (2023). How to use ChatGPT to manage your projects! Retrieved from StakeholderMap: https://www.stakeholdermap.com/project-management/chatgpt-ai-for-project-management.html
