The reality of migrating to the public cloud is that it doesn’t meet expectations around lowered operational costs. To add to this despair organizations struggle to find the main contributors to cloud costs. Therefore the concern for cloud costs is not surprising, as confirmed by ZDNet’s article Cloud Cost Control Becoming a Leading Issue for Businesses. The article shares global public cloud services and infrastructure spending will reach a market valued at $370 billion by 2022. Unlocking the benefits of the cloud allows cloud customers to become more agile, secure, and collaborative, yet services, tools, and other add-ons quickly contribute to the rise of costs.
A major contributor to cloud costs is cloud storage. In a study conducted by Fortune Business Insights, the cloud storage market estimated at $49.13 billion in 2019 will reach $297.54 billion by 2027. As cloud customers journey into artificial intelligence, machine learning, and the internet of things, storage costs will only continue to grow. Based on the 2019 State of the Cloud Report from Flexera, the top cloud initiatives in 2019 was to optimize the existing use of cloud resources in order to produce cost savings. In order to save on cloud storage costs, this blog post will explain the basics of cloud storage services and costs.
Public cloud solutions offer a range of services to store, access, govern, and analyze data. In AWS this comes in the form of object storage, file storage, block storage, and solutions for migrating or backing up data in the cloud.
What are the types of storage in AWS?
Amazon EFS, Amazon EBS, and Amazon S3 are AWS’s three different storage solutions. Let’s discuss each of these in more detail.
|Storage Service||AMAZON S3||AMAZON EBS||AMAZON EFS|
|Features||Can be publicly accessibleObject StorageWeb interfaceDurableScalableSlower than EBS and EFS||Accessible only via the given EC2 MachineFile System interfaceBlock StorageHardly scalableFaster than S3 and EFS||Accessible via several EC2 machinesObject storageScalableFaster than S3, slower than EBS|
|Use Case||Good for storing backups||Good as an EC2 storage volume||Good for shareable applications and workloadsServerless file storage|
Amazon Simple Storage Service (Amazon S3) is the solution for storing objects in a flat, non-hierarchical environment. Each object file contains a unique identifier key so that the object can be accessed from any service via a web request. Files can only be up to 5TB of data.
Amazon’s Elastic Block Store(EBS) is block-level storage for cloud compute instances like Amazon EC2 instances. They act as storage volumes similar to local disk drives used for machines. There are different types of volumes including Solid State Drives (SSD) and Hard Disk Drives (HDD).
- There are two kinds of SSD EBS:
- IOPS SSD (io1) – highest performance SSD volume designed for latency-sensitive workloads. Use case: NoSQL and relational databases
- EBS General Purpose SSD (gp2)* – a volume that balances price and performance. Use case: low latency interactive apps, dev & test.
- There are also two kinds of HDD EBS
- Throughput Optimized HDD (st1) – Low-cost HDD volume for frequently accessed, throughput intensive workloads. Use case: Big data and log processing.
- Cold HDD (sc1) – Lowest cost HDD volume for less frequently accessed workloads. Use case: data requiring few scans per day.
Amazon Elastic File System (EFS) is a file system storage service used for scaling and managing latency-sensitive storage needs. Amazon EFS volumes are mounted to various AWS services and can be accessed by multiple virtual machines. This is great for running servers, shared volumes, big data analysis, and other scalable workloads.
Understanding these cloud storage options enables teams to make conscious trade-off decisions between cost, quality, and speed for their cloud storage use cases. So let’s discuss cloud storage as it relates to cost.
Cloud Storage Pricing
A core principle in managing and optimizing costs is around considering costs an efficiency metric. There are four components that comprise cloud storage pricing, the cost of storage, the cost of requests and data retrievals, the cost of data transfers, and the cost of feature management and replication. Some of these costs are nonexistent for certain types of storage services.
|Storage Service||AMAZON S3||AMAZON EBS||AMAZON EFS|
|Requests and Data Retrievals||✔ |
Per GB orPer 1,000 requests
|Data Transfer||✔ |
Per GB out from Amazon s3
|Management and Replication||✔|
Per object, GB or batch job
Pricing based on snapshots
Amazon EBS pricing is simpler than Amazon S3 pricing, you simply pay for the cost of per GB-month or storage and managing snapshots. Amazon EFS has the simplest pricing structure as you only have to select an EFS class (Stand or EFS IA, Infrequently Accessed) and pay for what you use.
What about Database Cloud Services?
I recommend applying the principles shared in this blog post to database cloud services. Amazon’s Relational Database Service (RDS) cost is also based on different instances, tiers, and usage. Pricing for these instances varies, often the cost of storage rate is per GB-Month, but in other cases, this can be charged at an hourly rate. The four components of cloud storage costs also apply to database cloud services cost. Bringing awareness of the differences in cost of these services is a great place to start for optimizing cloud spend.
FinOps teams can remove the need for engineers and operations teams to focus on rate negotiations within the cloud, instead, they should focus on usage optimization and variable cost model of the cloud. This blog post shares the key concepts behind cloud storage services. Optimization is about making continuous adjustments in cloud usage and optimization through visibility into cloud spend, utilization, and resource consumption. Harness’s Cloud Cost Management product can help you do this within the public cloud. Learn more about Cloud Cost Management and request a demo today.