In Team 20 (Anisha, Raoul, Tom and me) we designed an AI tool for Howden, a Dutch insurance broker. The tool reads a mid-sized client’s documents, then makes a risk profile for each insurance policy and advises which insurers should take which share. My teammate Raoul works at Howden, so we figured it’s best if he built the prototype. Therefore I decided I’d be happiest and most productive working on the first half of the argument: why the problem matters, how big the opportunity is and how the business model works. That was slides 2, 3 and 4. If those slides were wrong, the economics, the prototype and the recommendation that came after them would all be based on the wrong thing. My bachelor’s in Business Administration helped here, because it had already taught me to start from economics and the business model before the technology, which is how I approached the problem slide and the opportunity slide.
My first argument was reasonable, and I liked it. A broker spends so many hours placing one insurance line that a small policy costs more to place than it earns. I figured it would be a good opening because it starts with money instead of our technology. But the whole argument depended on our estimate of 6 to 10 broker hours per policy. I got that number from Raoul’s colleague at Howden. It was not in a company record and nobody had measured it. Our slide that showed the loss per policy looked precise even though it was based on a guess. We looked for a second source and double-checked the cost of a broker hour against salary data obtained by Raoul and CBS labour cost figures. That gave us a good indication of the cost per hour. It still did not verify our hours, so we decided to call them estimates, as that is what they are.
Those estimates are also what broke my first argument, which was disappointing at first. After we ran the numbers with our estimates, the AI saved about 32 euros per line before fixed costs. This meant that a 2,000 euro policy still lost 41 euros. My slide 2 said small policies lose money, and we expected the tool would fix that, but it didn’t. I was a bit frustrated but realised we had to adapt and change our goal with the tool. So we changed our recommendation. Instead of focussing on saving money, the goal was now to free up broker time, and testing that in a pilot before rolling out. I realised I was too focused on cost saving, and that was the wrong angle. What the tool gives Howden is more capacity, which is only worth something if the brokers actually end up using the extra time. We wrote that in the deck too: we assumed no reduction in payroll.
The first design we came up with also depended on AI too much. We had to draw a line at what actually reaches the clients or insurers. We drew the line at AI reading and drafting and the broker deciding and approving. This may have made our tool a little bit less impressive, but also made it more believable.
In general, a big problem in our proposal was the lack of evidence. We had limited data from Howden due to confidentiality and still no clean number for how long placing a policy takes. Our prototype also gives outputs which are fictional and we prepared ourselves. It is more focused on how the workflow would work rather than AI actually reading the documents. The 42 minutes saved per policy which drives our whole case is an assumption.
A decision I’m happy with is slide 4, where we scored our business model against the six features of transformative business models (Kavadias, Ladas and Loch, 2016), and it only ticked two. I wrote the part that explains what this means: this tool won’t transform an industry but could be an improvement for Howden. I made that decision myself and didn’t check it with the team before putting it in the slide, so I don’t know if someone in my team would’ve wanted to push for a bigger claim. I’m still happy with this decision, because I don’t think we would’ve been able to argue differently. However, next time I would discuss it with my teammates first before adding it to the slide.
We did not receive a lot of feedback during this project. We got none on our short project proposal and the feedback we got from our peers at Demo Day was also limited. This meant we couldn’t depend on someone telling us we’re on track, so we mostly had to test our own work a lot.
We started this project kind of slow. We had no clear roles and difficulty with coordinating our busy lives. This led to some discussions and few decisions being made in the beginning. If I started again, I would reach out to Raoul earlier about missing numbers from Howden. This late contact and design caused us to only know how weak our evidence was when there was little time to fix it.
Before this course, I thought we would mainly be focussing on picking the right tool to solve our problem. I now think that the tool matters less than the work around it. Reading documents with AI already exists and is easy to buy. What makes brokers like Howden unique is their human side. Their current selection of insurers, data and the fact that they’re responsible for their advice. Just because something can be built does not mean that it is wanted. Will insurers actually rely on a file that AI drafted? Our whole case depends on that. Therefore we recommend testing it first with a pilot where we just measure the minutes our tool saves compared to a human. If it saves less than 22 minutes per policy, we simply stop.
Kavadias, S., Ladas, K., & Loch, C. (2016). The transformative business model. Harvard Business Review, 94(10), 91–98. https://hbr.org/2016/10/the-transformative-business-model
