DoorDash Deploys Multi-Agent LLM System to Automate Massive Technical Debt Cleanup of Stale Feature Flags

Engineering organizations at hyper-growth technology companies face an invisible, compounding tax on productivity known as technical debt. Among the most persistent and resource-draining forms of this debt is the accumulation of stale feature flags—code conditional wrappers used to test new features or manage progressive rollouts that remain embedded long after experiments conclude. DoorDash, operating a massive microservices architecture managing tens of thousands of active configurations, has confronted this challenge head-on. The company has developed and successfully evaluated a sophisticated multi-agent Large Language Model (LLM) system designed to fully automate the tedious, error-prone process of identifying, analyzing, and removing dead feature flags at scale.

The innovation, recently accepted for presentation at the prestigious IEEE International Conference on Software Maintenance and Evolution (ICSME 2026) industry track, represents a watershed moment in how modern engineering teams leverage generative artificial intelligence. By combining live experimentation telemetry, human-in-the-loop validation, isolated Git worktrees, and rigorous automated testing pipelines, DoorDash has managed to turn a multi-hour manual chore into an automated operation executed in minutes at a fraction of the cost.

The Scale of the Problem: Over 60,000 Flags Across Hundreds of Repositories

To understand the magnitude of DoorDash’s achievement, one must examine the sheer scale of its experimentation infrastructure. The food delivery and local commerce giant maintains a platform that manages more than 60,000 concurrent feature flags spanning approximately 623 distinct code repositories. With product and engineering teams continuously deploying updates, the organization mints an estimated 2,300 new feature flags every single month.

While feature flags are indispensable for modern continuous delivery—allowing teams to decouple deployment from release, perform canary testing, and execute A/B tests—they have a notoriously short functional shelf-life. Once an experiment concludes or a feature is fully rolled out to 100 percent of users, the conditional blocks governing that feature should theoretically be stripped from the codebase. In practice, however, engineers frequently move on to new projects, leaving dead flags behind. Over time, these forgotten wrappers accumulate, bloating the codebase, obscuring business logic, increasing cognitive load for developers, and occasionally introducing security or performance vulnerabilities.

DoorDash established a strict internal definition to identify this technical debt: a feature flag is officially classified as "stale" when it has remained entirely unmodified for a minimum of 90 days, continues to have active references scattered throughout the code, has not been explicitly archived or retired by its creators, and is not covered by any manual exclusion lists.

To surface these relics, the company instituted a daily automated pipeline that scans the experimentation ecosystem and automatically generates Jira tracking tickets for every identified stale flag. However, generating a ticket is the easy part. The real bottleneck lies in the manual remediation required to safely extract the flag without breaking production services.

Anatomy of Complexity: Why Traditional Tools Fall Short

Removing a feature flag sounds deceptively simple: find the conditional statement, delete it, and clean up the associated variable. In enterprise-grade codebases, particularly those utilizing complex architectural patterns like dependency injection, the reality is vastly more intricate.

At DoorDash, application architectures often distribute a single flag’s definition, its client-side retrieval call, and its underlying business logic across multiple disparate files. Consequently, what appears to be a trivial Boolean flag can easily require synchronized modifications across anywhere from five to twenty distinct files, including unit tests, integration tests, and configuration schemas.

Historically, the software industry has attempted to solve this through rule-based static analysis. The most notable open-source pioneer in this space is Uber’s Piranha, a tool that utilizes Abstract Syntax Tree (AST) transformations to automatically identify and strip out stale feature flag code based on predefined syntactic patterns. While Piranha proved effective at Uber and other companies, DoorDash discovered during its architectural evaluations that the tool was fundamentally unsuited for its own codebase.

DoorDash’s heavy reliance on dependency-injected wrappers meant that the relationships connecting a feature flag to application logic were semantic rather than strictly structural. Because matching syntax alone could not reliably map these distributed dependencies, traditional AST-based scrapers frequently missed vital references or risked breaking core application logic. This limitation highlighted the urgent need for a more intelligent, context-aware system capable of reasoning about code semantics—a gap that led DoorDash to conceptualize its multi-agent LLM framework.

The Two-Phase Architecture: Combining Claude Models with Model Context Protocol

Built utilizing Google’s Agent Development Kit (ADK), DoorDash’s automated remediation workflow is divided into two distinct, highly coordinated phases: orchestration and execution. This division ensures that human governance and machine intelligence operate in harmonious synergy.

In the first phase, an orchestrator agent powered by Anthropic’s Claude Sonnet model polls Jira to retrieve newly created stale flag tickets. Once a ticket is ingested, the orchestrator initiates a multi-repository search to locate all relevant code references. Simultaneously, it queries DoorDash’s centralized experimentation platform via the Model Context Protocol (MCP).

Model Context Protocol serves as a standardized communication standard enabling AI models to securely and seamlessly connect with external tools, APIs, and data repositories. Through MCP, Claude Sonnet retrieves critical metadata regarding the target flag, such as its historical rollout percentage, final target value, and initial creation context. Once this telemetry is gathered, the system compiles a comprehensive diagnostic report. Crucially, before any code manipulation begins, a human engineer reviews this report and explicitly confirms the target value, serving as an essential safety gate.

Once human approval is secured, the workflow transitions into its second, highly autonomous phase. Up to four cleanup agents powered by Claude Opus are deployed concurrently in isolated Git worktrees—up to four agents operating simultaneously per repository to maximize throughput without risking branch collisions.

Rigorous Validation: Ensuring Zero Regressions at Scale

Deploying generative AI to write or modify production code always introduces valid concerns regarding code safety, hallucinations, and silent regressions. To mitigate these risks, DoorDash engineered a rigorous, multi-layered validation pipeline that every AI-generated pull request must successfully clear before human review.

Operating inside ephemeral Git worktrees, each Claude Opus agent is tasked with locating all lingering flag references, determining the optimal refactoring strategy, and rewriting both the source code and its corresponding test suites. Once the modifications are written, the agent triggers an automated validation suite.

This verification process includes running application builds, executing full test suites, measuring patch coverage via JaCoCo (a popular Java code coverage library), and performing static code analysis through Detekt to catch style violations or structural anti-patterns. To prevent state pollution or caching conflicts between concurrent tasks, Gradle is executed without its background daemon, ensuring complete isolation across worktrees. Furthermore, each agent operates under a strict one-hour operational timeout to prevent infinite loops or runaway resource consumption.

A pull request is submitted for final human engineering review only if all automated compilation, testing, and coverage checks pass cleanly.

Empirical Results and Performance Evaluation

To validate the efficacy and economic viability of the system, DoorDash conducted a rigorous empirical evaluation across 50 distinct stale feature flags spanning varying degrees of architectural complexity. The results, which formed the basis of their ICSME 2026 research paper, exceeded initial expectations.

Out of the 50 evaluated stale flags, the multi-agent system successfully produced fully usable, review-ready pull requests for 45 of them. On average, the automated workflow completed an entire cleanup cycle in just 13.8 minutes. Financially, the compute and API token costs averaged a remarkably modest $4.79 per cleanup. When measured against DoorDash’s internal productivity benchmarks—which estimated that a manual cleanup by a human software engineer typically consumed between one to two hours of focused work—the AI system delivered staggering efficiency gains.

A granular breakdown of the outcomes across the 50 test cases revealed:

• 31 first-pass merges (pull requests accepted and merged without requiring any manual developer modifications).
• 14 revisions (pull requests requiring minor adjustments by human engineers).
• 5 engineer interventions (instances where the complexity necessitated manual intervention).

When categorized by code complexity, simple flags achieved a flawless 100 percent single-pass cleanup rate. Medium-complexity flags achieved a 94 percent success rate, while highly complex flags achieved an 85 percent success rate. The five instances requiring direct human intervention were almost exclusively cases involving deep, convoluted call chains and cross-interface parameter threading that challenged even advanced language models. Most importantly, DoorDash reported zero bugs or production regressions stemming from any of the 50 evaluated changes following deployment.

Implications and Future Roadmap

The successful deployment of DoorDash’s multi-agent LLM cleanup system signals a fundamental shift in how engineering organizations manage the lifecycle of technical debt. By moving away from brittle, rule-based scrapers toward semantic, tool-augmented AI agents, companies can reclaim thousands of developer hours previously lost to mundane administrative maintenance.

Building upon this success, DoorDash has outlined an ambitious roadmap for the future iteration of the platform. Upcoming enhancements include the integration of advanced confidence scoring mechanisms to automatically fast-track lower-risk cleanups without requiring manual gatekeeping for every single ticket. Additionally, the team is developing a post-cleanup code quality analysis pass designed to identify semantic side effects—such as misleading variable names or orphaned constants left behind after a conditional block is removed.

As artificial intelligence continues to transition from a conversational novelty to an embedded infrastructure partner, DoorDash’s pioneering work demonstrates that autonomous agents, when paired with robust guardrails and human oversight, can successfully shoulder the heavy lifting of enterprise software maintenance.