When we first formed our team, I volunteered to take on the prototype and evaluation parts of our AI Strategy Lab project. Later on I ended up working on governance and future market test, while my teammate took over developing the prototype. At that time, I thought this would be straightforward, because I already had an idea of how the prototype should look and I was eager to start. However, I was wrong, because the evaluation part was more complicated than I expected. What followed was quite a challenging and educational academic experience, forcing me to confront my own assumptions about AI, evidence, and what it means to design a strategy that actually holds up.
Our project addressed a real problem that many of us encountered: airline flight disruptions create sudden peaks in passenger recovery demand that human intensive processes cannot scale to handle efficiently. Our solution was an agentic AI system that prepares complete recovery packages (new flights, hotels, transfers, meal vouchers) for employee approval. My job was to prove this would work, identify what could go wrong, and stress-test the strategy against future scenarios where everyone has AI.
The first major challenge came when I tried to design the evaluation. The professor’s feedback on our proposal was clear: we needed to compare manual vs. AI-assisted processes on measurable metrics. But how do you evaluate something that doesn’t exist? We couldn’t build a real airline system, we didn’t have a database to work with, and even with that – time would be limited. The teaching assistant clarified that a simulation was acceptable, so we just needed to create realistic scenarios and compare outcomes. I designed three disruption scenarios and built a simulation comparing manual handling against AI-assisted handling on resolution time, labour cost, and error rate. But the assumptions behind the simulation became the hardest part. How long does it take for staff to re-book passengers manually? What is the error rate? I had to conduct an extensive literature search, and even after that some assumptions couldn’t be entirely justified. I had to be honest that some of my numbers were estimates, not facts.
What surprised me the most was how much the evaluation depended on assumptions I couldn’t fully verify. The cost per error was based on compensation ranges, but the error rate itself was an estimate. I learned that in AI strategy, you rarely have perfect data, you have to make reasonable assumptions, state them clearly, and be transparent about limitations. This was uncomfortable, but important.
Governance was another challenge. I initially thought of it as a list of risks, but the AI Canvas changed how I approached it. It forced me to separate: prediction (what will happen?) from judgment (which errors are worse?) and action (what do we do?). This wasn’t just an academic exercise, it fundamentally shaped how I thought about AI in decision-making. The AI predicts, human judges, the system acts, the outcome feeds back. The feedback loop was something I did not consider before, and it made me realize that AI isn’t a one-time solution, but an evolving system that improves over time.
The Three Tests for Responsible AI gave me a structured way to think about ethics. Before I had an assumption that ethics are vague and philosophical, but these tests made it concrete. Was data collected legitimately? Does the model infer sensitive characteristics? Can customers question the outcome? Who monitors the system? These questions are answerable, and our answers felt substantive.
The Future-Market Test was the moment where I had to push myself even further. I had to consider multiple stakeholders, not only competitors also having AI: What if platforms offered cross-airline rebooking during disruptions? What if partners used AI to dynamically price hotels during surges? These scenarios felt abstract at first, but the more I thought about them the more I realized they were plausible. It became clear that in an agentic future, invisible inequality could become the norm. Our defensibility had to come from data, relationships, and market design, not the AI itself.
Collaboration was harder than I expected. We were a team of four with different schedules and different levels of technical expertise. Since I underestimated the workload for evaluation, I had to ask my teammate to take most of the work with the prototype. Thankfully, she was ready to step in. There were moments of confusion, for instance I wasn’t sure how detailed the slides for the deck should be, and I wasn’t sure how much evidence was enough. As mentioned before, the feedback we received from the teaching assistant and the professor helped clarify expectations: a low-code prototype or a well-designed simulation was acceptable, as long as it demonstrated the workflow and tested strategic assumptions. This reassurance allowed me to proceed with confidence during moments of doubt.
What I learned from collaboration is that clarity matters more than perfection. My teammates probably didn’t need me to be the smartest person in the room. They needed me to be clear about what I was doing, what I needed from them, and where I was stuck. When I asked for help, my teammates responded. When I shared drafts, they gave feedback. The project worked out better because we worked together, not because one person had all the necessary answers.
Most fundamentally, this project changed how I think about AI and business strategy. I used to think AI was all about technology: the model, the algorithms, the tools. Now I see it as an amplifier. AI alone doesn’t create advantage, it amplifies what you already have. Our advantage came from proprietary disruption data and deep partner relationships, not the AI itself. I also learned that AI strategy is not just about building something that works, it’s about proving it works, stress-testing it against future scenarios, designing governance and all other crucial parts that make it trustworthy.
Through this project I learned to question my own assumptions, to be transparent about limitations, and to design for a future where AI is everywhere. And once again I learned that transparency and collaboration brings best results.
