AutoStopping Rules for Kubernetes Clusters

Businesses typically overspend on the cloud by roughly 35%, reflecting a significant gap in effective cloud cost management. And one of the biggest contributors to cloud cost bill shock is idle resources, along with unallocated resources. Wouldn't it be wonderful if we could switch off the idle cloud resources when they're not in use, and then turn them back on when needed? What if this could be automated? What if this automation, which resonates with the principles of FinOps, could be seamlessly integrated?

Here, we are introducing Harness Intelligent Cloud AutoStopping Rules to manage your idle resources both dynamically and automatically. AutoStopping Rules make sure that your non-production resources run only when used, and never when idle. Furthermore, they let you run your workloads on fully orchestrated spot instances without having to worry about any spot interruptions. With AutoStopping Rules configured, you will:

• Stop paying for cloud resources that you forgot to turn off.
• Reduce costs associated with non-production cloud resources by up to 70%.
• Stop paying for cloud waste, a primary objective in cloud finops practices.

How Are AutoStopping Rules Unique?

AutoStopping Rules are a dynamic and powerful resource orchestrator for non-production workloads. The following are some of the most significant advantages of incorporating AutoStopping Rules into your cloud resources:

• Automatically detect idle times and shut down (on-demand) or terminate (spot) resources.
• Enable the running of the workloads on fully orchestrated spot instances without worrying about spot interruptions.
• Predict idle times statically, especially during work hours.
• Allow the accessing of stopped or terminated machines, which is impossible with forceful shutdowns.
• Stop cloud resources without computing optimization, only start/stop actions.

You can also use the AutoStopping dashboard to view a summary of all of the AutoStopping Rules that you have created in a simple and intuitive interface.

Before AutoStopping Rules, there were only resource schedulers that dealt with comparable issues. However, they came with various imitations:

• Impossible to statically predict idle times.
• Teams can’t access stopped machines.
• Minimum optimization.

AutoStopping Rules Benefits

The following are some of the most significant advantages of adopting AutoStopping Rules:

• A measurable cloud bill reduction, with actual savings of 70% or more.
• Automated checks to make sure that no idle resources are operating and no cloud waste occurs.
• No opportunity for expense overruns due to human error.
• No need to remember to shut down or terminate resources once they've been used.
• After the initial setup, no further manual involvement is required.
• Simplified integration with current infrastructure provisioning techniques.

Using AutoStopping Rules for K8s Clusters

You now have a better understanding of the power and uniqueness of AutoStopping Rules. Let's explore how AutoStopping Rules can help you with your K8s clusters.

AutoStopping Rules support:

• AWS EKS
• GCP GKE
• Azure AKS

The steps for creating AutoStopping Rules are simple:

First, select the cloud account in which workloads are running that you want to manage using AutoStopping Rules (AWS, GCP, or Azure).

Second, define the AutoStopping Rule. Here, you also specify the idle time for your resources. This is the time that the AutoStopping Rule will wait before stopping the idle resources.

Third, select the resources that you want to manage using this rule. You can also specify advanced configurations, for example, adding dependencies between two or more AutoStopping Rules. This is useful for when you want one rule to make one or more rules be active based on its/their traffic.

Fourth, update the resource definition YAML of the Kubernetes AutoStopping Rule that will be applied to the cluster.

The specification in the YAML file is the same as a Kubernetes Ingress/Non-Ingress with additional metadata. These are the configurations for the Ingress/Non-Ingress that the AutoStopping Rule will create for your Service.

Ingress Example With AutoStopping Enabled:

apiVersion: ccm.harness.io/v1
kind: AutoStoppingRule
metadata:
   name: testK8s
   namespace: default
   annotations:
       harness.io/cloud-connector-id: awstest3
       nginx.ingress.kubernetes.io/configuration-snippet: 'more_set_input_headers "AutoStoppingRule: default-testK8s";'
spec:
   idleTimeMins: 15
   hideProgressPage: false
   ingress:
       rules:
           - host: <replace with your domain name, for example, qa.harness.com>
             http:
                 paths:
                     - path: /
                       pathType: Prefix
                       backend:
                           service:
                               name: <replace your service name, for example, test>
                               port:
                                   number: <replace with your service port, for example, 80>

Non-Ingress Example With AutoStopping Enabled:

apiVersion: ccm.harness.io/v1
kind: AutoStoppingRule
metadata:
   name: <postgres>
   namespace: <dev-poc>
   annotations:
       harness.io/cloud-connector-id: <connector_id>
spec:
   idleTimeMins: <40>
   workloadName: <postgres>
   workloadType: <Deployment>

Autoscaling vs. AutoStopping Rules

Autoscaling is a function that automatically scales your resources up or down to meet changing demands. Tools such as cluster autoscaler and Karpenter are used for autoscaling.

Cluster autoscaler automatically adjusts the Kubernetes cluster size when one of the following conditions is true:

• There are pods that failed to run in the cluster due to insufficient resources.
• There are nodes in the cluster that have been underutilized for an extended period of time, and their pods can be placed on other existing nodes.

As per Google Cloud, cluster autoscaler works on a per-node pool basis. When you configure a node pool with cluster autoscaler, you specify a minimum and maximum size for the node pool.

Cluster autoscaler increases or decreases the node pool size automatically by adding or removing virtual machine (VM) instances in the underlying Compute Engine Managed Instance Group (MIG) for the node pool. Cluster autoscaler makes these scaling decisions based on the resource requests (rather than the actual resource utilization) of Pods running on that node pool's nodes. It periodically checks the statuses of Pods and nodes, and takes action:

• If Pods are unschedulable because there are not enough nodes in the node pool, then cluster autoscaler adds nodes up to the maximum size of the node pool.
• If nodes are underutilized, and all of the Pods could be scheduled even with fewer nodes in the node pool, then cluster autoscaler removes nodes down to the minimum size of the node pool. If the node can’t be drained gracefully after a timeout period (currently 10 minutes), then the node is forcibly terminated.

You've probably figured out that the cluster autoscaler has its own set of complexities and limitations.

However, AutoStopping Rules provide cloud cost savings to the most granular extent. It is a dynamic and powerful resource orchestrator for non-production workloads. We have already discussed its uniqueness and capabilities.

The following example shows how idle resources in the K8s clusters are handled and autostopped according to the defined rule. View the cumulative savings for your resources by autostopping the idle resources.

Harness' AutoStopping solution also realizes that, under certain conditions, you may require your resources to be up-and-running regardless of the defined rules. You can configure an ECG heartbeat agent and/or fixed schedules with the rules to provide a solution for this.

ECG

By using AutoStopping Rules, you can automatically turn off cloud resources when idle, and then turn them back on when required. Furthermore, you define the number of minutes that the AutoStopping Rule should wait before stopping the idle resources.

By default, AutoStopping Rules listens to HTTP/HTTPS traffic. On the other hand, a resource could be working on long-running background jobs, such as machine learning (ML). In this case, using network traffic alone to detect resource idleness may not be the best strategy. Therefore, you can configure a heartbeat agent called ECG with AutoStopping Rules.

Moreover, you can assume ECG as an event emitter or a listener for your rule. It will send the usage record for the configured rules.

ECG comes with the following pre-installed watchers:

• Metrics
• Process

Configure either metrics or process watchers for your rules. The metrics watcher lets you send heartbeat signals when the specified metric threshold is reached. The process watcher watches for the existence of processes that match the given condition.

Metrics example:

# A heartbeat will be sent when the metrics are greater than or equal to the configured threshold

[metrics]
cpu = "40"
memory = "5Gb"

Process example:

# A heartbeat will be sent when a process with the matching condition is found

[process]
condition = "python*"

Fixed Schedules

Fixed schedules are also supported by AutoStopping Rules. In certain scenarios, you wouldn’t want your resources to go down or up. For example, every Friday at 5 p.m., you want your ABC resource to go down. AutoStopping Rules makes this very simple. You can schedule downtime for your ABC resource. During this window, the resource will be forced to go down regardless of the defined rule. Furthermore, you can specify uptime for your resources in the same way.

Get Started With Managing Idle Cloud Resources

Embarking on your journey towards cloud cost savings and efficient cloud finops has never been easier. The next logical question would be how to start using this intelligent and automatic cloud cost management solution:

• Simply sign up for a demo, and a Harness specialist will get you started with Harness Cloud Cost Management. Our dedicated specialists are ready to guide you through the process, ensuring you unlock the full potential of your cloud infrastructure while optimizing costs.
• Check out these docs if you want to learn more about how to create AutoStopping Rules for your K8s resources.

Start maximizing your cloud investments today with Harness, your trusted partner in cloud financial operations.