
What a difference just a few months can make! Since just last July when I attended Tech Field Days AI Field Day #8, the standard DevOps cycle and patterns we’ve used in the last decade appear to be transitioning significantly.
The introduction of several families of new AI models has accelerated dramatically, and that’s spurred IT organizations to consider shifting to either implementing or at least considering the deployment of open weight models within neo-clouds or even their own on-premises / private cloud. As a result, agentic AI application development and deployment strategies have also been forced to evolve dramatically.
Lossy DevOps Strategies Are Demanding Attention
CoreWeave provided a broad yet enlightening journey into the real world of agentic AI development efforts when they presented at AI Field Day #9 in early October 2026.
Based on their clients’ experiences, it’s clear that significant enterprise IT organizations – not just VC-supported startups working with theoretical agentic AI workloads – are now constructing and deploying agentic AI applications that must be reliable and provide consistent value within their production environments.
Another reality of Agentic AI development today: IT shops are often developing within and deploying across a plethora of different clouds because the data they’re running against is widely dispersed throughout the enterprise.
What CoreWeave noticed is that deployments are still relatively lossy propositions.

Agentic AI application deployment procedures are nascent when compared to traditional DevOps deployment methodologies. The lessons each team learns as they progress through deployment phases are being lost because the new DevOps patterns resemble a figure-eight loop rather than a straight line from concept thru coding and testing and ultimately successful deployment.
The new model CoreWeave proposes is more like a Mobius loop that starts with observation of an agentic system for issues, detection of true issues, careful diagnosis and curation of those true issues, finding a solution to improve its performance, and then evaluating whether the solution was actually effective in terms of improved performance.
(Interestingly, a recent McKinsey article similarly recognized the change in DevOps for Agentic AI: it also featured a similar figure-eight loop graphic to represent that shift in strategy.)
CoreWeave Forge: Casting New Agentic AI Development Methodologies
In response to these issues, CoreWeave recently launched Forge into the agentic AI arena. As they demonstrated, Forge has been specifically designed for the agentic AI development cycle to overcome the loss of crucial lessons learned within each stage of development.
CoreWeave’s key viewpoint is that continuous 24-hour delivery is achievable – but only with architectural discipline and standardized workflows so agents can operate reliably at scale.
Forge leverages the concept of post-training to implement a better path to continuous improvement. The tricky part is to select which post-training methods will best compare the expected results from AI model execution against what was actually returned – usually defined as drift – to isolate what particular model weights need to be tweaked so they perform better during the next round of production iterations.
Forge uses either supervised fine tuning (SFT), reinforcement learning (RL), or distillation – a combination of SFT and RL – within in the AI development loop to obtain specific improvements. SFT works best for improving a model’s responses based on specific, desired outcomes while RL is aimed at insuring a model to follow instructions more precisely.

I see distillation as perhaps the most intriguing option, as it uses production data from prior model runs, relabels that data for use with a stronger but smaller model, and then conducts several training passes to improve performance, thus using a sufficiently-robust model with lower token costs to speed development cycles. (A recent article about how Bank of America and Capital One are following that path lends credence to that strategy.)
The CoreWeave team discussed how Forge can dissect inference workloads to isolate how effective prefill operations (when the key value cache is initially populated in anticipation of LLM requests) versus decode operations (when an LLM returns hopefully valuable results).
Stochastic, Unpredictable Workloads Make Inference Performance Tweaking Difficult
Forge offers three specific inference analytics options, each with increasing customer control over inference monitoring operations. First, serverless inference via a simple API endpoint where a customer’s workloads generate intense and unpredictable bursts of activity.
Dedicated inference lets a customer provide specs to provide a predictable performance level while CoreWeave handles the monitoring cluster. Finally, for customers whose models have been developed independently of standard LLMs and demand tight control over network traffic and exchanged data, CoreWeave offers their solution completely within the client’s own managed Kubernetes cluster computing environment.

CoreWeave provided an extensive example of how their inference monitoring toolsets work via a live demo of HuggingFace’s Qwen 2.5 0.5 billion parameter LLM running against a sample inference workload deployment.
Agent Lens: Dissect Agent (Mis)behavior Now, Regression Test It Later
As compared to the programs and code I’ve been building, testing, and monitoring before agents became the current rage, the stochastic behavior of LLMs makes them maddeningly difficult to discover issues until they actually occur in production. The sheer volume of activity a typical AI agent is expected to handle at scale makes it particularly susceptible to debug.
CoreWeave claims that Agent Lens not only makes short work of seeing agentic performance issues during real-world performance, but it also makes it easier for non-coders who are experts in their particular domains – for example, a machinist on the factory floor who knows an agentic AI application is telling her to do something foolish or destructive based on years of hands-on experience – to actually leverage qualified end-user insights to create valuable testing scenarios based on observed failures.
From my perspective as an experienced integration and system tester, thr capability to essentially deploy a qualified, AI-enabled judge of agentic AI performance within CoreWeave Forge infrastructure offers me the opportunity to deploy the knowledge base and expertise of that machinist who first observed aberrant behavior during a peak workload two weeks ago without having to flag her down while she’s on vacation.
Real-Time Agentic AI Diagnosis
The CoreWeave team wrapped up their deep dive into Forge with a thorough demo of how it would be used during an inference workload to dissect a sample help desk agentic AI application. That involved asking Forge for what code changes would be needed to deploy appropriate instrumentation for agent performance observations.
I found it particularly intriguing to see the Agent Lens toolset isolate specific malperformance in real time, identify concentrations of particular poor responses, tag those responses, and then eventually apply either human curated or automated judge curated suggestions for performance improvement.
And once Agent Lens has finished its evaluations, CoreWeave showed how Forge can then take advantage of them to build out alternatives for more efficient inference using either SFT, RL, or distillation to determine which model deployment(s) would produce better performance as compared to prior deployments with reliable comparative statistics.
A New DevOps Loop: Infinitely Better Agentic AI, Or Just a Demolition Derby?
Finally, as a 45-year veteran of writing applications and managing databases – from IBM mainframe-only, through client server, into Web 2.0, and now this Agentic AI revolution – it’s hard to ignore the new SDLC models favor a figure-eight loop.
While that might represent hopes for an infinitely more efficient DevOps model, the figure eight is also a great source for exciting near-collisions at a county fairground raceway demolition derby, depending on the vehicles in use. I’m hoping that our future Agentic DevOps efforts actually fare better than some of the more spectacular ones (I am not making this up!) involving school buses.
One thought on “CoreWeave Forge: A Crucible For Improving Agentic AI DevOps Reliability”
Comments are closed.