Attributing Costs Using Harness Cloud Cost Management

Fundamentally, managing cloud costs is a simple, three step process: Attributing Costs, Optimizing, and Governing. The cloud has enabled teams to move quickly and leverage the latest technology and services to innovate faster than ever. Developers love to optimize, and inculcating the cost-centric mindset and optimizing for cost as just another boundary condition will help the finance and engineering teams operate efficiently. Cloud finops principles suggest that to optimize, developers, finance teams, cloud budget owners, and stakeholders need visibility into the costs coming from various public clouds.

Developers want to see the costs of the precise set of microservices that they are responsible for. Moreover, they’d like to see the exact areas of potential wastage, along with reliable recommendations. Then, the cloud budget owners want to see the costs of resources that they are responsible for, as well as governance guardrails. On the other end of the spectrum, the finance teams want to slice-and-dice the holistic data to make informed decisions about reservations, as well as accurately plan and model the future costs. But how? 

CCM Perspectives 

Harness CCM Perspectives let you group your resources in ways that are more meaningful to your business needs. It is a crucial cloud cost management tool. It provides a unified view of your multi-cloud cost data with the flexibility to manage governance and provide contextual optimization opportunities. 

Harness Custom Cloud Cost Perspectives empowers users to view the data that they own or manage by simply adding Rules. In the example above, we have a multi-cloud scenario where a particular user owns resources across clouds. By adding three simple rules to the perspective builder, we can see a quick preview of the costs broken down by products across clouds. Users can also change the default date range, group by, and visualization options. 

Perspectives help optimize, govern, and visualize data contextually. Once we define the perspective’s scope using the rules, reporting can be set up to receive emails delivered as per schedules. Budgets can also be set up within a perspective’s scope. This enables users to set granular alerting to targeted user groups to be notified when hitting certain cost thresholds. The perspectives even go beyond this by providing actionable recommendations to reduce costs and meet the governance guardrails that are in place.

Perspectives also offer deep insight into Kubernetes cost spend with metrics, such as Idle cost – which is a metric of spend wasted based on the request and actual utilization, and Unallocated cost – which is a measure of the costs being paid for that are spinning up the nodes but not being requested. To find the root cause of the cost spikes, deep Kubernetes visibility with the flexibility to break down the costs by namespaces, workloads, and nodes with the drilldowns to a specific workload with the utilization trends is extremely important. Perspectives make it straightforward to get to the root cause of the costs by pairing it with recommendations to save costs. CCM offers four perspectives out of the box with the flexibility to create custom perspectives as per user need. 

Beyond Perspectives – Introducing Custom Dashboards 

Custom Dashboards powered by business intelligence (BI) lets users bring in data across Harness modules, as well as cater to advanced visualization, reporting, and alerting use cases. Users can see costs in the context of the deployment timelines and root cause of the exact build that caused spikes, as well as predict cost increases when enabling a certain feature flag. Therefore, customers using multiple Harness modules enjoy the flexibility of seeing correlated data quickly within Harness. 

One of the key features of the Custom Dashboards is the flexibility to create custom fields. Custom fields are essentially a computed field governed by a definition, which is a combination of supported functions and first class fields. Let’s look at some use cases and custom field examples with the corresponding visualizations. But before diving into the examples, here’s a quick overview of the fields and functions used in the examples:

${unified_table.aws_gcp_azure_account_project_subscription} -> is a first class field exposed for consumption in the dashboards. CCM exposes 50+ first class fields to construct visualizations.

case, when, matches_filter -> are a small subset of functions supported by Harness BI Dashboards to create custom fields. There are close to 20 functions currently supported to create custom fields.

Example 1: Filtering Fields With a Certain Naming Convention 

case(
when(matches_filter(${unified_table.aws_gcp_azure_account_project_subscription}, `%ce%`),
 ${unified_table.aws_gcp_azure_account_project_subscription}),
 "Others"
)

This custom field is essentially a regex search for all of the accounts with the substring ce. Any Account, Project, or Subscription without this naming convention will be bucketed into Others. Any first class field with a certain naming convention paired with these custom fields can simplify dashboarding and enhance visualizations.

Example 2: Bucketing Costs 

case(
 when(
${unified_table.aws_gcp_azure_account_project_subscription} = "ccm-aws" OR
${unified_table.aws_gcp_azure_account_project_subscription} = "ccm-gcp" "CCM"),
 when(
 ${unified_table.aws_gcp_azure_account_project_subscription} = "ff-aws" OR
${unified_table.aws_gcp_azure_account_project_subscription} = "ff-azure" OR
${unified_table.aws_gcp_azure_account_project_subscription} = "ff-gcp", "FF"),
"CI/CD"
)

The custom field attributing costs from various accounts into cost buckets. The fields used for defining the cost buckets could range across clouds – GCP Products, AWS Usage Type, Azure Subscriptions, Labels, Tags, Kubernetes constructs such as namespaces, workload names, cluster names, etc. Therefore, the custom fields are extremely flexible when defining the cost buckets. Bucketing is incredibly useful for creating visualizations with business context, as well as cost by products, customers, cost centers, engineering teams, etc. The following is an example of how using the bucketing custom field will show up for end users to consume in scheduled reports and dashboards:

Custom Dashboards also provide deep visibility into cloud inventory, such as the underutilized AWS Elastic Computing (EC2) instances, Orphaned EBS Volumes and Snapshots, and many more. Custom Dashboards is a single pane of glass for all reporting and custom visualization needs, along with the added flexibility of features such as custom fields. 

What’s Next? 

Apart from perspectives and custom dashboards, CCM will soon release a Business Mapping feature that will let you bucket costs and enable the sharing of common infrastructure costs across cost buckets. Stay tuned for more updates. 

In the meantime, why don’t you continue your CCM reading journey? Here are a couple suggestions: BigQuery Costs Shocking $15k In Wasted Cloud Spend and Cloud Cost Workload Recommendations.