No ratings yet.

Picture: (Reisenwitz, 2022)

Context
In our last blog post, we explored the potential of ChatGPT in the use case of developing project scopes as part of project briefs in (student) consultant firms. There, its potential became clear while its risks in dealing with prompts and skewed information inputs also surfaced.

In this article, another potential application to the world of consulting project management is discovered. This is deliberately one that instead of having to generate a new piece of text takes already complete information and uses it to suggest a clear-cut decision.

Here, the discovery pertains to one of the decisions that plays a role in optimizing internal operations and is a key determinant of the future success of a project as well as consultant satisfaction; their allocation to projects and compositions in teams (Nexus Technology, 2023). As this often turns out to be such a complicated decision with a multitude of stakeholders and views involved, ChatGPT may help to comprehensively assess the information and give insight into new dimensions we may have to more actively consider.

Experimentation
Luckily, we possess sanitized data on projects and student consultants that do not have to be further cleaned for use. For this small experiment, three projects that ran in the same cycle were picked and their sanitized and shortened project briefs as sent to consultants were given as input in a consistent format to ChatGPT. This includes the background of the client, including size and industry as well as a description of the client’s problems and sometimes expectations in terms of tasks to be done; think of market research, a competitor analysis, developing an outreach strategy, etc. This is done with the aim of replicating the decision made in the past as closely as possible, without telling it the metrics used for the respective allocations.

In addition, the anonymous data about consultants is extracted from our online workspace and combined in simple textual, entered form. Here, to make the analysis more explainable amidst many potential variables, only their year of study, current study program and current professional (or volunteering) role title were used as indications of successful applicant’s interests and potential performance.
Now, a prompt asked ChatGPT to assign eighteen student consultant profiles over the three projects with different scopes and types of clients. In this case, the personal preferences of student consultants as submitted are left out of the equation to better explore how ChatGPT would analyze the problem.

Impressions
Interestingly, while not specified prior, ChatGPT assigned more people to the first project and less to the second, where usually a consistent maximum of six people per project is adhered to out of communication, team size and leadership complexities. By breaking down the project scope and seemingly matching that with keywords found in the study program or job title of the applicants, it did form three seemingly logical teams in which students with backgrounds in marketing, finance and legal are each grouped under a project. What it may not have automatically considered, however, is the fact that the desired project success mentioned as a goal does not just depend on assumed expertise. It also depends on collaboration, how well people work together and to what extent personal backgrounds match. With students ranging from different cultural backgrounds, from 1st year to last-year university bachelor or master students and from entirely different study backgrounds, this may give rise to complementarities in behavior.

Beyond the risks accompanying biases from training data and prompts, others have expressed similar comments regarding the difficulty of ChatGPT with coming close to humans’ cognitive abilities in solving problems and in exploring ambiguous decisions (Wizly, 2023). This may not only hold in analyzing the challenges of clients but also in enhancing internal decision-making. Under this notion and as potential learning from the experiment in the previous blog post, I asked ChatGPT what other factors it would ideally like to know and keep in mind if again being asked to make this decision. It now did highlight a set of interesting dimensions, from what I perceive as practical to interpersonal and internal to external. These include the availability of consultants, valid as no matter the match in knowledge, commitment and flexibility play a key role in individual and group performance. As part of here hard to assess interpersonal and leadership skills, an overlooked determinant for our own process is someone’s level of client relationship management skills. Despite being hard to fully assess, a proxy used internally is that of the number of projects done prior as it could give an indication of exposure to managing expectations and presenting. Externally, ChatGPT seems to have based its decision and explanation thereof more on the overlap between the most mentioned topics in project briefs and consultant profiles. In this case, this is the overlap between the project scope and the educational and professional title of a successful applicant. Nevertheless, the project brief also contains information on the client organization and even contact person, including their seniority, title, the type of firm and industry. In theory, these could give reason to predict what types of consultants, backgrounds and experience level they would prefer to have on their project.

Most importantly, ChatGPT may just not define project success as a goal in the same way we do right away. For us, this is a mix between clients being served with high-quality interactions and recommendations, while simultaneously offering applicants the chance to grow and learn. In that sense, the given information about consultants’ current roles and study backgrounds may prove an equally important indicator not of what they already know, but also in what areas they may still grow. Even with prompts stating this is part of the aim, it was difficult to find a balance in which it did not either primarily look at project success or rather than consultant learning. Finally, as it possibly could not properly consider that team allocations may change over time, the conclusion may not only be that the output depends on quality input and comprehensive prompt chains, but also on the role to be fulfilled by ChatGPT. During this experimentation process, it proved to be most insightful and efficient as an ideator for such a multifaceted decision. When asked to brainstorm about all kinds of factors that may influence a project allocation decision, it could help me consider new characteristics of projects, client contacts and student consultants if deliberately asked for. In fact, some of these may augment the way in which personal fit interviews and the information asked from applicants through forms are treated. When having explored these topics in an internal version of ChatGPT-like applications, prompt engineering may enable better extracting value prioritizing metrics such as project objectives, budgeting, resource availability and task importance (AI for Work, 2023). In combination with the possible need to redesign project brief templates for this purpose, it could enable the development of tailored project resource allocation plans.

Feel free to engage in the comments if you have explored similar applications for resource allocation, either in your job or even in planning out your private life. What does it take for ChatGPT to come close to developing explainable dynamic resource allocation reports at human cognitive standards?

References:

AI for Work. (2023). What Makes a Great Resource Allocation Plan. Retrieved from AI for Work: https://www.aiforwork.co/chatgpt-prompt-articles/create-a-resource-allocation-plan-with-this-chatgpt-prompt-for-chief-operating-officers-executive-management
Nexus Technology. (2023, August 28). Thinking Outside the Bot: Leveraging ChatGPT for Streamlining Internal Processes. Retrieved from NexusTech: https://www.nexustech.je/news/thinking-outside-the-bot-leveraging-chatgpt-for-streamlining-internal-processes/
Reisenwitz, C. (2022, November 26). How Team Analytics enables smarter resource allocation. Retrieved from clockwise: https://www.getclockwise.com/blog/team-analytics
Wizly. (2023, June). Consulting And ChatGPT: What Does The Future Hold. Retrieved from Wizly: https://www.wizly.app/post/consulting-and-chatgpt-future

No ratings yet.

Picture: (Meier, 2023)

Context
Generative AI has gained enormous attraction recently, quickly intervening in a multitude of our life’s aspects. As its presence has expanded rapidly, a vast range of private as well as business use cases is being explored. Out of curiosity about its potential to provide value for businesses, I have attempted to assess its use in project management in an environment close to my heart; student consulting (CoinTelegraph, 2023).

As part of the board of a student consulting firm, our branch is part of a global organization aiming to help social impact organizations by providing affordable to free services while offering students the chance to develop themselves and gain practical experience. Thus, as part of this model, all organizational members worldwide are volunteers. Therefore, as our office has been in the process of vast growth, resource limitations and corresponding allocation are crucial for operations to run sustainably. A process that remains risky and could prove to be a recurring bottleneck each cycle is that of new client recruitment. Here, one of the most time-consuming tasks in that of developing project briefs.

In such briefs, a description of the client is outlined and, most importantly, the scope of the project is laid out and tied to a first conceptualization of tasks that will likely need to be undertaken. Not only does such a document aid in the following stages of the engagement but it also helps to assess feasibility and matching with internal capabilities. Moreover, it lays a foundation for the student consultants and project leader to start assigning tasks. Finally, those project briefs of engaged leads are sanitized and absorbed into a globally administered knowledge- and database. This is why there is a consistent template and structure to the briefs, which is reused in the small-scale simulation run for this blog post.

Experimentation
Hence, also minding the security of proprietary data, three highly sanitized versions of past project initiations were used in order to ask ChatGPT to develop a project scope for a hypothetical brief. Here, copied text from an initiation via each of three common channels was used, being an incoming e-mail, an interest registration form submission and explorative meeting notes. All of these contain information about the background, underlying problems and needs of the potential client organization.

After providing ChatGPT with the overarching structure of our project brief template and a description of the intention of developing the document, I asked it to concisely write out a project scope that focuses on topics on the one side while keeping in mind underlying client challenges on the other in order to test how close it comes to how humans may write out a brief.

Impressions
What stands out, despite the natural differences in project topics across the three examples, is the apparent correlation of the developed scopes with the input given to ChatGPT. While generally impressive in generated scopes even with limited input lengths, it seems to skew towards how much a certain subtopic is mentioned in the text. For example, if a client stated it is developing a new product and wants to develop a promotional strategy, the developed scope does not get into detail on that most likely the product attributes and consumer preferences need to be researched unless specifically told so in a follow-up prompt. This is where, acknowledging the unexplainability of the black box, there seems to be some difference with scoping based on inserted meeting notes. These longer texts already contain more practical and human thoughts on what implications the wants of a client have on the project tasks further down the line. Despite this higher level of comprehensiveness and accuracy, it does seem sensitive to how much text is placed under a section in the meeting notes, with more a lengthy notes section on seeking partnerships taking up considerably more of the developed scope rather than the asked-for industry trends analysis, even though later in the notes it is stated that only two out of six people on the team would likely work on this matter.

In conclusion, the potential time-savings are great and potential is high. Nevertheless, providing a structure to follow may not be not enough for cases where such a model is not internally governed. In comparison to self-written scopes for these past projects, it seems difficult for ChatGPT to accurately attribute proportional attention and consequential thinking to given inputs even if asked for. This underwrites the role of human intervention in value extraction from this tool. Beyond being sensitive to the textual inputs given which may have to be balanced out in the future, it remains very sensitive to prompt writing. Clear and repetitive specification of how and for what purpose ChatGPT should filter and use the information does guide it in the right direction, which may explain why it performed better in converting the contextualized meeting notes rather than a very brief section in a form. Moreover, while not being conclusive about the written scopes after using chained prompting, this deliberate use did enable later follow-up questions without having to repeat what was first indicated in a first contextual-task prompt (StakeholderMap, 2023).

Ultimately, even when contextualization and clearly bounded expectations are part of the prompt writing, first impressions are that it remains hard to control for its biases. Therefore, its practical use for project brief writing appears to still require human intervention, at which stage those involved may still likely want to retain control over its full development, even if time-consuming. This is reiterated elsewhere, concluding it has a support function in developing documentation for project management if adjusted and reviewed manually to match its original purpose (Parm, 2023). At least for now, it may remain more of an efficiency booster rather than automation, although it may be enhanced in combination with project software. Additionally, further experimentation may explore to what extent asking ChatGPT how it assesses the comprehensiveness of the information put in, also allowing it to ask ourselves questions about what it would like to know more of to perhaps output more contextualized and less biased scopes (Blore, 2023).

In case of any thoughts on experimenting further with this use case as part of managing projects, how to better control for its potential input sensitivity biases or when having better alternatives in mind, feel free to let me know in the comments!

References

Blore, Z. (2023, June 20). Unleash the Power of ChatGPT: The Ultimate Game-Changer for Project Management! Retrieved from SimplifyChange: https://www.simplifychange.co.uk/our-blog/unleash-the-power-of-chatgpt-the-ultimate-game-changer-for-project-management
CoinTelegraph. (2023, July 10). How to use ChatGPT for project management. Retrieved from CoinTelegraph: https://cointelegraph.com/news/chatgpt-for-project-management
Meier, K. (2023, May 30). How to write a project scope document from scratch. Retrieved from teamwork.com: https://www.teamwork.com/blog/project-scope-template/
Parm. (2023). How can ChatGPT help in Project Management? Retrieved from Parm: https://parm.com/en/chatgpt-in-project-management/
StakeholderMap. (2023). How to use ChatGPT to manage your projects! Retrieved from StakeholderMap: https://www.stakeholdermap.com/project-management/chatgpt-ai-for-project-management.html