When we began the AI Strategy Lab assignment, I initially approached the assignment as a solution-first, problem-later mindset. Over time, I realised that we were trying to find a problem that matched our solution rather than starting with a problem itself. My team came up with the idea of implementing AI in Canvas to answer student questions while providing real-time class insights to tutors and professors. As the work progressed, I began to realise that the real problem was the information gap between students and instructors, rather than simply the lack of AI features in Canvas, because students generate fragmented signals about what they don’t understand or are struggling with while instructors have limited time to interpret those signals and implement changes in the course as it is ongoing. My main responsibility became the prototype, experimentation/evaluation and evidence, which made me change my thinking from whether we could build an AI system to asking whether AI actually improved the quality and efficiency of information available to instructors.
One of the most important changes in our project was narrowing the problem itself. At first, our idea could easily be described as two AI features, namely, a chatbot helping students and a dashboard for instructors. We realised that evaluating both features separately would be too broad, and with my team, we decided to focus on one underlying problem: the information gap between students and instructors. The student-facing chatbot wasn’t valuable merely because AI could answer questions but because it could use these questions to generate structured insights that could improve an instructor’s real-time insights into recurring confusion and where their students are struggling. This brought both parts of the system into one value logic rather than two separate features.
This was one of the first points where I understood more clearly what the course meant by treating technology as an input rather than the business model itself.
Initially, I spent considerable time and effort thinking about the full product: a course -grounded assistant with anonymous questions, clustering of recurring issues through an instructor dashboard that fully worked. As the deadline was closing in and the evaluation was still pending, I became increasingly concerned that a technically good prototype wasn’t allowing us to test our core assumption and whether our idea was strategically useful. 2 days before the deadline, I scrapped the overly technical prototype and focused on the evaluation around our core assumption: could GenAI transform fragmented student questions into accurate and useful class-level insights while reducing the amount of information an instructor had to process manually?
One of the most useful results from the prototype was not when it worked perfectly, but when it failed. In an early validation test, the system identified the intended patterns but also produced 2 additional clusters that were not supported by the underlying data. Initially, I thought this simply meant that our prototype wasn’t working properly and needed to be fixed. On reflection, however, it exposed a much more important governance issue. If an instructor acts on a false pattern generated by the AI, the system could actually worsen the information gap it was supposed to reduce. This made me change the design of the prototype by introducing traceability back to underlying questions and involving the possibility of human oversight wherever possible, distinguishing AI confidence from priority, and simplifying the evaluation prototype.
I also learned to be more disciplined about what our evidence actually supported. For example, while our prototype reduced the number of first review items from 30 questions to 13 review units, this did not necessarily prove that instructors experienced less cognitive effort. Earlier in the project, I might have presented such a number as proof of efficiency, but by the end I was more careful to separate what we actually measured from what we could only assume.
The team structure also taught me that individual workstreams in a strategy project cannot be improved independently. My team and I, especially in the last 2 days before the deadline, sat together and went through each other’s parts while putting the slide deck together in order to have everything on the same page. I could have designed the prototype that summarised information effectively, but that alone did not show that the product should exist. The evaluation had to connect to the value proposition, economics, governance risks and eventual recommendation. Working with my team forced me to understand and explain what my results actually meant for the wider strategy rather than treating prototype performance as an end in itself. Furthermore, discussions with my teammates in the last few days before the deadline helped me tweak and redesign my prototype to align with the overall vision; my teammates also helped me with the design choices.
Overall, this project changed how I think about AI in business strategy. At first, I focused mainly on what AI could do, but I gradually realized that just because something is technically possible does not mean it creates real value. I learned that simply adding AI to a platform is not enough to stand out, especially when competitors have access to similar technology. What matters more is how well AI solves a problem, fits into existing processes and produces reliable results. Looking back, my biggest takeaway was learning to question our ideas rather than just focusing on building them. Instead of asking “What can we do with AI?”, I now ask “What problem are we solving, why is AI the right solution, and how can we prove it works?”
