About

Recognized as a global visionary and thought leader in automated machine learning (autoML), time series forecasting, and responsible AI, H2O.ai is the trusted AI partner to more than 20,000 organizations around the world. Its platform, the H2O AI Cloud, enables businesses, government entities, nonprofits and academic institutions to accelerate responsible innovation and push the boundaries of what’s possible with artificial intelligence.

Challenge: Improve insights into resource utilization and attribution

As a company that thrives on democratizing AI and leveraging open source solutions, H2O.ai equally values its ability to build its cloud generative AI solutions in its customers' cloud environments – and to open up H2O.ai’s own cloud environments for testing. That’s a great service for customers, says Michal Malohlava, H2O.ai VP of Engineering. But it also meant that H2O.ai needed to gain greater visibility into cloud costs, and track usage and budgets by business unit. 

“We’re focused on delivery and customer satisfaction – and we’re growing rapidly,” Malohlava says. “But we’re also looking at what we can optimize in the business, like cloud costs.”

Finding answers to questions about cloud costs wasn’t as simple and efficient as H2O.ai wanted it to be. The large and complex cloud environments, spread across AWS, Azure, and Google Cloud Platform (GCP), complicated the task of attributing costs to the correct business unit. In addition, generative AI adoption was exploding across all sectors of technology, increasing the cloud budget for H2O.ai.

To serve customers, H2O.ai was also using costly resources such as GPUs to run AI and machine models, and wanted to view costs and resource utilization in real time. But AWS Cost Explorer and the native tools in Azure and GCP couldn’t break down those costs accurately, leaving most of the bill attributed to engineering. 

“Something as simple as people forgetting to shut down cluster machines over the weekend could cause spikes in budget,” Malohlava says. “That made it hard to go to the finance department to track costs or come up with ways to improve efficiency.”

Solution: Cloud Asset Governance to identity and eliminate waste

With a goal to ensure that every cloud resource was in active use, H2O.ai adopted Harness Cloud Cost Management (CCM). Malohlava has deployed CCM in stages, beginning with simple cost tracking and automated notifications about cost anomalies, and later addressing governance and idle resources with CCM Cloud Asset Governance. 

With CCM Cloud Asset Governance, Malohlava can dig deeper into inefficient use of cloud resources, proactively manage these resources, and ensure that all cloud resources match corporate standards and policies. Harness CCM enables H2O.ai to label machines and assign owners to them, which helps stakeholders better attribute costs to the proper owners.

Harness CCM also looks for trends in overspending or overestimating the necessary cloud resources. For example, costs for Amazon EBS volumes made up about $2.5 million–half of the company’s annual cloud operating costs– because teams were overestimating the size of the volumes needed. Harness CCM and its Cloud Asset Governance feature helped Malohlava detect monthly spend on EBS volumes of $32,000 per month, of which 60% was unused – and then delete the unused volumes. 

Result: $35,000 cost savings in first 30 days

To date, H2O.ai has saved $87,000 in the first 4 months of using Harness CCM by leveraging the automated governance policies in Cloud Asset Governance to automatically find and eliminate cloud waste. H2O.ai is also using Cloud AutoStopping™ to dynamically shut down idle VMs and containers on all root cloud accounts that are connected to Harness, and then run them on fully orchestrated Amazon EC2 Spot instances. 

Malohlava saves time as well as money. Before H2O.ai began using Harness, he spent about 15 hours per week creating and managing cloud cost reports. With Harness, Malohlava can automatically generate the reports and set alerts for budget overruns.

“Harness has very much simplified my work life,” Malohlava  says. “We've uncovered over $35,000 per month in cost savings during our first 30 days of using the feature, and expect to continue to see more over time. We’re excited about Harness and its capacity to refine our cost strategies as our business continues to grow.”