FinOps presents more of “a big data challenge”

There are many Cloud usage/Finance management tools that help cloud engineering team in spend analysis, visualizing amortized investments, time usage-based cloud services and volume usage-based cloud services. These tools have address basic cloud usage and spend across multi cloud.

Even with multi cloud FinOps tools, there is always human intervention required for decision making at scale. Considerable amount of human effort is spent on analyzing the cloud usage data sets across multi cloud. The challenge doubles when FinOps team is trying to integrate organization’s finance processes with invoices from multiple cloud vendors.

FinOps presents more of “a big data challenge” as there is so much Cloud usage data on daily/weekly or monthly basis to reconcile across multiple public cloud providers and Business units. (Application, Databases, services). For every usage of cloud service runs per second, there is a unique SKU getting charged. In certain cases, a invoice might be thousands of rows. On top of that there are thousands of SKU that appear on invoices.

For large enterprise, when cloud usage is at peak, Engineers might not have time to analyze huge amount of cloud usage data and hence, they rely on central FinOps/Data Science team to analyze cloud usage datasets.

FinOps team seek help from Data Analytics and Data Science team to work on cloud usage dataset.

Some of the use cases (using R, Python and PowerBI), where Data Science team help Engineers and FinOps team for cloud optimization

• Real time Analyze various Business units' cloud usage vs invoices from all the cloud providers plus on prem vendors. Then provide direction which business unit should invest on cloud heavily. Identify the cloud usage that is enabling business revenues hence supporting the FinOps team/policy decision to continue scaling. Identify the cloud services that is not enabling any business revenues hence supporting the FinOps team/policy decision to decommission/scale those services down.
• Engineering team need more detail analysis of recommendations from each FinOps tool. For example, a Cloud Engineer would like to identify which services AWS or Azure, or Google are good for different kind of workloads (CPU intensive or memory intensive)…