As part of my first serious Oracle DBA assignment in 2001, the first production database I’d ever built was hosted on an HP8500 ProLiant server – at the time, a serious computing beast with 8 removable CPUs and an enormous 8GB of physical memory.
I spent plenty of frigid days in our server room building storage arrays, attaching servers to storage and networking, and keeping a close eye on the hardware to insure everything remained fully available and performant. And over the next five years, that ProLiant simply never quit, even when under stress of demanding OLTP and reporting workloads.
Agentic AI Is Inference-Intensive. Robust Compute Is Still Crucial
Of course, that HP8500 is now ancient technology – it would likely melt down if asked to handle 1% of today’s generative AI, RAG, and agentic AI workloads that have dramatically expanded the breadth of requirements for extremely reliable and performant compute resources.
The latest ProLiant series we saw as HPE presented at AI Field Day #8 recognize these new computing demands in several dimensions.
HPE’s Senior Principal Product Manager Bharath Venkant talked at length with us on the changing dimensions of AI compute, from on-premises thru public and hybrid cloud, plus the growing need for AI inference at the edge.
I found HPE’s orientation on AI refreshing: They realize there’s an incredible divergent breadth of compute requirements for customers as journey towards whatever AI destination they’re headed, and they’ve (wisely!) focused on meeting customers wherever they might be right now on that roadmap.
The HPE product continuum I reviewed at AIFD8 offers everything from a DevOps AI “starter pack” sporting two NVIDIA RTX Pro 6000 GPUs all the way up through a continuum of small, medium, and large computing setups.
And for the most extreme AI compute requirements, HPE offers an expansion rack option housing as many as 64 NVIDIA H200 (“Hopper”) GPUs.
HPE’s AI Engine strategy thus promises to simplify an IT shop’s their AI-specific compute needs as they traverse shifting training and inference requirements because ever-increasing demands can be met within an identical, reliable hardware framework.
Inference At Razor’s Edge Demands Reliable Compute
I’ve always thought of HPE focused on building servers for pristine data center racks, so I was pleasantly surprised to see their offerings for AI at “the edge.” Factory automation, warehousing, logistics, and biomedical production immediately popped into mind … but certainly not battlefield situations.
HPE built their ProLiant EL2000 2U server chassis specifically for extreme edge conditions; it’s rated for military-grade ruggedness and resilience and plugs into their EL220 and EL240 Gen12 servers.
Extreme conditions – vibration, temperature, dust, intermittent connectivity, variable access to power and cooling – meant HPE had to build compute components to accommodate some potentially challenging edge constraints.
And while AI workloads at the edge are more likely focused primarily on inference to interpret critical situations and leverage agentic AI to recommend immediate action with human oversight, I’d foresee that data collection for retransmission back to a “home base” for later model fine-tuning could also be a requirement.
Private Cloud AI: Blueprints Sure Help
While I’ve spent the majority of the past decade either presenting at conferences or helping clients build complex database solutions, I’ve found myself immersed in DIY home remodeling. Fortunately, my successes outnumber my failures, but I’ve noticed my most successful projects start with intense and detailed planning, especially a drawing of what I intend to create.
Unfortunately, it’s an open secret that many IT shops struggle with laying out plans – much less building – a comprehensive AI solution that fits their organization’s needs. Mark Seither, PCAI Senior Principal Solutions Architect, explained in depth HPE’s alternative to DIY AI: their Private Cloud AI solutions.
The smallest system available – aka their developer system – uses a DL-325 server as a control plane plus a DL-380A that hosts two 2 NVIDIA RTX Pro 6000 GPUs.
This gives system admins and their DevOps teams a chance to get familiar with HPE AI capabilities – what I like to think of as the targets the “poke it with a stick” orientation – before deciding which configuration is likely needed for current production demands.
Moving up the sizing scale, the medium and large configurations shift from RTX Pro 6000s to NVIDIA NVIDIA H200 (“Hopper”) GPUs, with the largest system accommodating 10 H200s. And as mentioned previously, expansion racks can be added that handle up to 64 H200s. (HPE also mentioned they’re planning a 128 H200 expansion rack in the near future.)
If I were tasked with building out a new AI platform strategy, I see several advantages of this approach:
- Since HPE deploys identical software across the entire platform family, it’s relatively simple to upgrade from, say, the developer system to a larger Private Cloud AI systems.
- Speaking of the value of planning ahead: HPE leveraged NVIDIA’s published blueprints to create the most efficient and lowest-cost options for immediate deployments.
- Finally, the partnership with NVIDIA offers advantages for AI implementations. NVIDIA’s open-source Nemotron 3 family of models, datasets, developer tools, and inference capabilities look pretty intriguing!
A Real-Life Example: Why Is My Discount Denied?
As an experienced developer, DBA, and presenter about technology, I’ve seen the value of the Don’t just tell me – show me maxim, especially when it comes to all things AI. I wasn’t disappointed with an all-too-brief demonstration of HPE’s quick-to-market AI capabilities.
Michael ran a live demo of their AI Essentials toolset to show how simple it would be to construct a chatbot that leveraged Langflow Agentic AI Workshop.
In under five minutes, he had deployed a reasonably sophisticated chatbot using the Nemotron 3 Nano LLMs from NVIDIA that accessed travel reimbursement policies retained in an internal document store. He then simulated requests from customers trying to understand why a particular travel refund request wasn’t handled to satisfaction .
How Do You Know Where You’re Going If You Don’t Know Where You Are?
So many IT shops these days are hearing the same drumbeat of their CIOs:
Look, We know {generative AI | RAG | agentic AI | predictive AI} is the solution – now go find problems to solve!
I found HPE’s orientation and roadmap for building a practical compute strategy refreshing because it was grounded in real-world use cases and offered a consistent vision for meeting a potential or current customer wherever they are in their AI journey, and their close partnership with NVIDIA adds stability and flexibility to whatever that journey may potentially lead.
