• Recent MIT Study Findings: Only about 5% of GenAI applications create real business value. Most teams are experimenting but the underlying systems for delivering software haven't adapted.
• Challenges in Prototyping vs. Production: Building prototypes is easy with prompt tweaks and API calls, but turning them into trusted production systems requires real engineering like reliability, consistency, etc. Traditional tools and workflows weren't designed for systems that change behavior over time.
• CI/CD Pipelines and Agentic Systems: CI/CD pipelines were built to test code but agentic systems don't behave like static code. The real question is whether the system can make trusted decisions in changing conditions.
• Rethinking Software Supply Chain: We need to accept that the old way of building and delivering software doesn't fit agentic use-cases. The software supply chain should move towards delivering continuous intelligence about the software.
• Evolving from CI to Continuous Intelligence: In the agentic era, the goal of CI changes from checking code to trusting intelligence. We need to validate how agentic systems behave in real-world conditions.
• EVALS as New Unit Tests: In agentic systems, EVALS are like unit tests for non-agentic software, measuring performance, reliability, and safety. They can run offline, online, or continuously.
• Integrating EVALS into Delivery Chain: Integrating EVALS into the delivery chain ensures continuous validation throughout the software lifecycle, from CI to CD to production.
• Treating Supply Chain as a Living System: The software delivery process should become a continuous feedback loop like a living system, adapting and improving with user signals and data.
• Conclusion: The future of software delivery depends on shifting focus from code correctness to behavioral trust. Evolving the supply chain into a feedback system creates more intelligent and trustworthy pipelines.