From Insight to Implementation: Why Agile Breaks Down in Data Science
And How to Fix It
Published: July 2025
By Amy Humke, Ph.D.
Founder, Critical Influence
“We tried Agile. Two-week sprints. Story points. Daily standups.
We got missed estimates, unfinished models, and lots of apologies.”
Agile promises speed, adaptability, and collaboration. That sounds great until you try it in data science.
While the core philosophy of Agile aligns with how data teams actually work—iterative, collaborative, responsive to change—the usual frameworks don’t. Traditional Agile assumes predictability. Data science work is anything but.
If you've ever missed a sprint commitment because the model still didn't perform, or watched your project backlog collapse under constantly changing requirements, this article is for you.
Whether you lead a data team or do the hands-on work, these lessons matter. For doers, it’s about reclaiming space to explore without burning out. For leaders, it’s about letting go of the illusion of control and investing in structures that actually support insight-to-impact work.
Let’s walk through where Agile breaks down in data science, and what a better approach might look like.
1. Why Agile Fails Data Science Teams
(It's Not the Values, It's the Templates)
Agile is a mindset. But most teams adopt it as a method—usually Scrum.
That's where things go wrong.
Scrum assumes that work is:
- Well understood and easy to estimate
- Deliverable in 2-week chunks
- Traceable through user stories with clear acceptance criteria
But data science work starts in ambiguity. Even if the end goal is clear (build a churn model, forecast sales), the path to get there is filled with decisions, unknowns, and pivots.
You don't know if the data has the right signal. You don't know which features will matter. You may not even know if the problem you're solving is the right one until you get into the weeds.
That makes Agile's usual tools—sprints, story points, velocity—feel disconnected from how data science works.
Doers need flexibility to ask the right questions. Leaders need to stop forcing clear plans before the real discovery even starts.
2. The Estimation Problem
Why Data Science Doesn't Fit Neatly into Sprints
Scrum thrives on predictability. But data science tasks are exploratory and uncertain. Estimating them precisely is nearly impossible.
You can't reliably forecast:
- How long it will take to clean a messy dataset you haven't seen (and may not even have access to)
- Whether a new model will outperform the baseline
- How many iterations it will take to calibrate thresholds that work
And when you try to timebox these activities into two-week sprints, you have a choice: ship something incomplete, or miss the sprint. Neither feels good.
What works better? A structured but flexible framework grounded in milestones.
Here's how I break it down:
| Phase | Goal |
|---|---|
| 1. Problem & Success Alignment | Define the business problem, metric targets, and working hypothesis |
| 2. Data Feasibility & Exploration | Validate data quality and surface early signals |
| 3. Baseline Modeling | Run quick models to test feasibility |
| 4. Iterative Feature Engineering | Build and refine features to improve performance |
| 5. Model Optimization & Calibration | Tune, calibrate, and prep for deployment |
| 6. Stakeholder Review | Package and present decision-ready outputs |
| 7. Production & Drift Monitoring | Implement deployment and post-launch checks |
| 8. Post-Launch Evaluation | Evaluate real-world model performance and ROI |
Each of these chunks represents a meaningful unit of work—a milestone. They don't always fit into a neat two-week box. That's the point.
Doers need milestones they can actually meet. Leaders need progress markers that reflect real progress—not just ticket closure.
3. Story Decay
Why Static Requirements Fall Apart Midway
User stories are a staple of Agile. But in data science, they fall apart fast.
Let’s say your story is:
"As a marketing director, I want a churn model to target at-risk customers."
By the end of your initial modeling, you discover churn isn't driven by customer behavior but by poor onboarding experience. Suddenly, the original story is no longer aligned with reality.
That's story decay. And it’s everywhere in data science.
The problem isn't that you're doing it wrong. It's that user stories assume the goal won't change. However, data exploration constantly reveals new insights that reshape what success looks like.
What works better? Hypothesis stories.
"We hypothesize that onboarding experience predicts churn. We'll build a model to test whether onboarding variables explain at least 30% of churn variation."
Hypothesis stories are explicitly designed to be tested, refined, and invalidated. They treat learning as the output, not failure. They preserve agility without making teams feel like they're constantly rewriting history.
Doers need permission to change course. Leaders need to treat pivots as progress, not backtracking.
4. What Is Data-Driven Scrum
And How Do You Use It in a Fixed Sprint Team?
Once you realize traditional Agile doesn't fit data science well, you might start looking for better options. One that comes up frequently is Data-Driven Scrum (DDS).
DDS is a lightweight Agile framework built specifically for data and ML teams. It keeps the spirit of Agile—iteration, collaboration, continuous improvement—but swaps fixed sprint timelines for capability-based milestones.
Instead of asking, "What can we finish in two weeks?" DDS asks, "What capability can we complete next?"
- Are the assumptions (hypothesis) tested?
- Is the data validated?
- Have we tested the baseline model?
- Are the features finalized?
- Does the model hit performance thresholds?
Each phase finishes when its goal is met—not when the sprint timer runs out. This removes the artificial pressure to ship something half-baked just to close a sprint.
And the structure is already familiar. The eight milestone chunks outlined earlier (from problem alignment through post-launch evaluation) fit naturally as DDS iterations. You’re not inventing a new workflow—you’re aligning Agile with how data work actually happens.
Doers benefit when milestones are respected over minutes. Leaders benefit when capability, not clock time, defines iteration success.
5. Making DDS Work When Your Whole Team Shares the Same Sprint Cycle
My team doesn't run custom cadences for each model build. We operate in fixed two-week sprints across the analyst, engineering, and data science teams. It's clean for coordination—but tricky when you're in the messy middle of a modeling project.
So how do you keep a fixed sprint rhythm while respecting the reality of exploratory data work?
1. Distinguish the Type of Work
Not all tickets behave the same. So don’t treat them like they do.
- Discovery tickets are exploratory. You're testing a hypothesis or validating feasibility. Timebox them (e.g., 2–3 days), and define "done" as "learning captured," not "model delivered."
- Implementation tickets are build-and-ship. These are pipeline builds, dashboards, model deployments—tasks with clear inputs and outputs.
- Maintenance/support tickets are quick-turn items: bug fixes, data checks, monitoring alerts.
Just naming these differences helps the team plan more realistically and reduces the pressure to deliver something deployable from every task.
2. Use DDS Milestones as the Bigger Picture
While sprints move on a two-week cadence, project work can span multiple sprints. That's normal. A DDS-style milestone, like finalizing a feature set, might take one sprint or three. That’s fine—as long as:
- You're clear about which part fits in the current sprint
- You track momentum across sprints
- You treat discovery work as valuable, even if it doesn’t produce a deployable asset yet
Think of DDS as your initiative roadmap, and sprints as the containers you use to get there.
3. Redefine "Done" for Exploratory Work
A discovery task is done when you've tested the hypothesis, documented what you learned, and decided what comes next.
That might look like:
- "We found that support tickets strongly correlate with churn."
- "Data quality issues block us from testing X; recommend fixing upstream first."
- "Model A outperformed baseline, but precision is too low to move forward."
This keeps sprints focused and valuable—even when the outcome is “not yet.”
4. Keep Stakeholders in the Loop About Learning
Agile reviews tend to spotlight what was built. But in data science, some sprints yield models—and some yield insight. Make both visible.
Stakeholders who understand that learning reduces risk will support your process. Stakeholders who only want deliverables probably wouldn’t be happy anyway.
Doers need sprint structures that reward real progress, not just finished code. Leaders need visibility into the path, not just the outcome.
6. Lessons for Leaders and Doers
What Agile Gets Right—and What You Have to Change
Agile gets a lot right. We want short feedback loops. We want close stakeholder alignment. We want the ability to pivot.
But the rituals of Agile—story points, velocity tracking, burndown charts—only work when the work itself is predictable. And in data science, it’s not.
The goal isn’t to throw out Agile. The goal is to make it work for the kind of work we do. That means:
- Tracking progress by milestones, not just tickets
- Writing stories that can evolve as we learn
- Defining "done" in terms of clarity, not just code
- Giving teams room to explore, without guilt for missing a sprint
The best Agile for data science is flexible, outcome-focused, and honest about what exploration takes.
Doers need space to solve the right problem well. Leaders need frameworks that can handle the truth of discovery without panicking.