From idea to impact: lessons on how to implement AI in large organizations
The Erasmus MC case
By Sophia Zitman & Evertjan Peer
Many AI projects start with big ideas and impressive proofs-of-concept, but often fail before they create any real value inside a large organization. The barrier is rarely the technology. It’s making the leap from a pilot to a solution that fits the organization’s actual processes, strategy, and people.
At KickstartAI, our focus is on closing this implementation gap. A recent project with Erasmus MC — where we built a system to predict patient inflow — in their busy emergency department, offers clear lessons on what it really takes to move AI from idea to impact.
The initial spark vs the real need
The project began with a clear technical proposal: build a model that predicts patient number arrivals in the ER so staff could manage capacity better. On paper, it was a solid idea — technically sound and tied to an obvious need.
But experience has taught us that many AI projects stall when they start with technology first. A clever model alone rarely delivers lasting impact. That’s why, right from the beginning, we asked the kinds of questions that make or break large-scale AI initiatives. And these aren’t just “nice to have.” They’re essential to making sure a solution is sustainable, valuable, and actually used.
We wanted to know:
What is the real operational problem here, beyond the initial idea?
How does the current process work, and where are the true bottlenecks?
How does this challenge tie into the organization’s broader strategy and management priorities?
How might it change the day-to-day work of staff on the ground?
And who will ultimately own, use, and maintain the solution?
By asking these questions, we ensure that the solution:
Tackles actual bottlenecks, not assumptions.
Fits with the organization’s strategy and priorities.
Aligns with how people really work, so it’s more likely to be accepted.
Has a clear owner and a path for long-term maintenance.
💡TIP: Our approach goes beyond just asking questions in meetings. We shadow staff, observe their workflows, and sometimes even step into the work ourselves. This hands-on approach grounds our understanding in reality, validates assumptions, and surfaces the things that really matter.
Lesson 1: Invest time in defining the business problem
When Erasmus MC first came to us, the question on the table was: how can we reduce the moments the ER goes on “red” — reaching maximum capacity and unable to safely accept new patients?
It was a real challenge, but also extremely broad. “Going red” could be triggered by many different things, such as patient inflow, discharge bottlenecks, room availability, or staffing shortages.
That’s why we began with a dedicated 3-4 month “Use Case Identifier” phase before any intensive model development began. The goal wasn’t to write code, but to understand the problem deeply from the hospital’s perspective.
Stakeholder interviews
Talking to everyone involved — from frontline staff to management — surfaced one key insight: the central role of the managing nurse. This person is the operational hub of the ER, deciding which nurse takes which patient, when to redirect patients elsewhere in the hospital, and how to keep flow moving.
Walking the floor
We shadowed staff and asked them to explain their work “as if to a new colleague,” making sure nothing obvious was left unsaid. We saw firsthand how big of a task it was for managing nurses to make capacity decisions with incomplete information. They were the ones truly “feeling the pain.”
Mapping the process
With the workflows laid out, we could identify the key decision points and constraints. It became clear where there was room to maneuver — such as staffing patterns and patient routing — and where there were hard limits. Mapping also revealed exactly where timely information could make a difference, and who needed it most.
This deep dive refined the objective from a vague ambition (“avoid going red”) to a concrete solution: building a model that predicts patient inflow to help managing nurses make proactive decisions.
Together with the nurses, we discovered that short-term predictions (half an hour ahead) were too late to act on, while long-range ones (six hours out) were too uncertain. The sweet spot was a 1-4 hour window: short enough to be reliable, long enough to take action.
💡TIP: That upfront time investment turned a broad challenge into a specific, actionable tool designed to support managing nurses directly, while tying into the hospital’s strategic goal of reducing ER overcrowding.
Lesson 2: Assess feasibility beyond the algorithm
Feasibility isn’t just about whether you can build an algorithm. It’s about whether the solution works in the real world.
Data availability & quality
Do we have the necessary data, and is it reliable and fit for our purpose? It’s key to testing assumptions about what kinds of data are actually useful.
In the Erasmus MC case, weather data seemed like an obvious predictor — nurses even suggested it: when it freezes, cyclists fall and ER visits rise. But when we linked historical weather patterns to patient inflow, the data told a different story. Erasmus MC focuses on severe, complex trauma, not everyday accidents like a broken wrist from a bike fall. Such cases are directed to other hospitals.
Predictive power vs. uncertainty
How accurate can the prediction realistically be, and how much inherent uncertainty exists in the process? ER patient flow has significant inherent variability, for example. Acknowledging these limitations early and designing ways to manage uncertainty (like providing prediction ranges or scenarios) was key to keeping the tool practical and trustworthy.
Technical integration
Can the solution be embedded into the existing IT infrastructure? And who owns these systems?
Erasmus MC had experience with AI, but mostly in isolated environments for very specific medical use cases. Our project needed to run on a hospital-wide system and support daily operations.
That meant pulling together expertise from multiple departments. Different teams knew parts of the puzzle — machine learning lifecycles, data pipelines, system administration — but rarely had they been connected in a single project. The challenge was to design something that worked immediately without being blocked by unfinished projects, while still aligning with where the hospital’s infrastructure was heading. That called for balancing speed with future-proofing, and thinking carefully about long-term ownership and maintenance.
💡TIP: Ignoring any of these factors leads to solutions that work in a lab but never make it into practice.
Lesson 3: Plan for adoption from day one
An AI solution only creates value when people actually use it — consistently, and within their daily workflows. Adoption isn’t about dropping a new tool on someone’s desk; it has to be built into the process from the very beginning. This involves four steps:
User interface & experience
What’s the best way to present AI’s output so it feels intuitive and actionable for end users?
We involved nurses in designing the interface from day one. In early sessions, we asked them to explain how they would present the information themselves when onboarding a new colleague, for example. Some leaned toward color systems, others preferred graphs, and some chose simple tables. These differences gave us valuable clues about what felt natural. We then created mock-ups in several formats and tested them again. The focus wasn’t on aesthetics but on actionability and readability: could nurses quickly digest the information, and did it help them take the right actions?
Interpretation & transparency
How should staff understand the AI’s output, including its uncertainties and limitations? Transparency is key, especially around prediction ranges and confidence levels. Taking the tim to explain to staff how to read and use these signals is also necessary to ensure that the system isn’t just functional but also trusted.
Process change
What existing processes need to adapt, and what new actions should be triggered by the AI’s insights? This meant looking closely at how predictions would slot into nurses’ existing routines.
Building enthusiasm & sustaining adoption
A major benefit of involving end-users early is the feeling of inclusion.
Delivering a solution “out of the blue” would have felt like a surprise. Instead, involving nurses from the start helped create enthusiasm, foster early adopters, and build ambassadors who could spread the word among colleagues.
As we later prepared for rollout, we also thought about how to sustain adoption. One idea was peer-to-peer sharing: for instance, giving nurses a stage in weekly meetings to talk about how they use the tool.
💡TIP: Thinking about adoption from day one — and making it visible along the way — is what turns a technical project into something that sticks. Start small, involve users closely, and you create the conditions for trust, enthusiasm, and long-term use.
Conclusion: Weaving AI into the organizational fabric
Successfully implementing AI in large organizations takes a holistic approach. It starts with understanding the problem and operational context, checking what’s actually feasible with the data and systems in place, and planning early for adoption.
The Erasmus MC project showed that success when AI isn’t treated as a standalone tool, but as part of the hospital’s fabric — tied to its strategy, processes, and people. That’s what turns a promising idea into something that provides tangible impact day to day.
💡 Want to learn more about how we accelerate AI adoption in the Netherlands? Check out our projects page, take a look at our website, or follow us on LinkedIn!
