Why managing GenAI projects requires a playbook tweak
By Nic van Vliet
KickstartAI recently partnered with ING in a controlled pilot setting to explore how AI can support financial health. Our prototype solution was a GenAI-powered financial health assistant designed to provide instant, conversational answers to complex financial questions within clearly defined scope and guardrails.
Building with Large Language Models (LLMs) taught us a vital lesson: success often depends less on the code itself and more on how we manage probabilistic systems. While developing a prototype for a potential customer-facing GenAI chatbot, we realized that traditional tools like rigid PRDs, linear roadmaps, and deterministic “if-then” logic aren't the right fit for products that learn and respond with variability. What we needed, instead, was to rethink our management approach a bit.
Here are the three core lessons we learned about managing a GenAI project compared to the development of a traditional tech product.
1. Aligning expectations on the “response freedom” spectrum
In a conventional software product , the UI is a closed loop: a user clicks a button, and a hard-coded result occurs. With GenAI, the interface is a blinking cursor and a natural language prompt, introducing infinite possible responses.
The Play: Calibrating the level of "freedom" for a risk-averse tool like a financial assistant is more art than science. If you turn freedom down to zero (minimizing creativity), the chatbot becomes deterministic but may struggle to synthesize information from the RAG (Retrieval-Augmented Generation) system. Dial it up too far, and the chatbot risks producing out-of-scope or unsupported responses.
The Mindset Shift: Managing GenAI means moving from "defining features" to "establishing behavioral parameters." In regulated environments, this requires an early and explicit alignment on what the model can and cannot do.
Reality Check: For high-stakes flows, "human‑in‑the‑loop" confirmations are essential. This ensures that for the most important actions, what you want to happen actually happens. While response freedom is an asset for conversation, it is a liability for execution; the more an agent acts, the more deterministic its guardrails must become.
2. High-level scoping and iterative collaboration
While traditional tech projects often rely on a rigid Product Requirement Document (PRD) finalized upfront, GenAI development requires quicker and constant iteration. Success depends on shifting focus from the initial prototype to a continuous cycle of evaluation and refinement. Because LLMs are inherently conversational and unpredictable, a static scope is rarely sufficient.
The Play: Instead of a 20-page PRD, start with a high-level scope and iterate continuously within clearly bounded guardrails. This often involves hours onsite testing various scenarios with the partner to align on the "feel" of the chatbot's responses and ensure the output meets expectations in practice.
The Mindset Shift: Prioritize high-level scoping and rapid prototyping. Define the intent and guardrails first, then engage in intensive feedback loops with domain experts to evaluate the model early and often. We find that this "trial-and-refine" process is key in the development cycle, allowing you to build an Evaluation Framework that scales with the product (see point 3).
The Reality Check: PMs must guard against infinite iteration. Constant back-and-forth can lead to scope creep and unpredictable budgets. To reduce this risk, use time-boxed sprints and still define clear Minimal Viable Requirements (MVR) to ensure the project has set go / no-go dates.
3. Closing the loop: scalable evaluation & maintenance*
Early GenAI development often relies on how the responses feel, especially when it comes to customer-facing AI. While this works for a prototype, a customer-facing product requires a more structured approach to quality control.
The Play: Deploy hybrid evaluation frameworks. Use domain experts to grade nuance and complex logic, while implementing Automated Evaluation (LLM-as-a-judge) to catch edge cases across the "long tail" of potential user queries. This is crucial in ensuring consistent quality when dealing with millions of possible inputs.
The Mindset Shift: Move from "one-and-done" testing to continuous monitoring. Unlike static code, models suffer from Prompt Drift when providers update underlying weights. A sustainable playbook must include a live maintenance strategy to regularly re-calibrate prompts and monitor model performance over time.
The Reality Check: In our collaboration with ING, KickstartAI focused on exploring how telemetry and monitoring could be integrated in line with existing banking infrastructure and governance standards. Since production-grade chatbots are a relatively new frontier, we are essentially building the map as we go.
*We are still in the early stages of establishing this monitoring and observability, and we look forward to sharing our findings as we discover how to best ensure long-term reliability in such a nascent and complex environment.
The GenAI PM mindset
In this new era, project management evolves from an “if-then” to a “trial and refine" mindset. By bridging the gap between probabilistic potential and deterministic reliability, we can build GenAI products that harness the full potential value of AI moving forward.
For a deeper technical look at our collaboration with ING, read our full technical blog here.
