Moving Beyond Reactive Dashboards

Managing cloud costs at the scale of 14,000 accounts presents a significant operational challenge. For BMW Group, the journey began with the Cloud Efficiency Analytics (CLEA) platform, an in-house FinOps initiative developed in collaboration with Reply. Initially, CLEA relied on visualization tools like Amazon Quick Sight to provide employees with visibility into their cloud usage. However, the team realized that dashboards are inherently reactive; they only show what has already transpired, and only when a user actively chooses to examine the data.

To address this visibility gap, BMW and its partners sought to transition from static reporting to a proactive alerting model. According to the AWS Machine Learning Blog, the updated architecture now runs automated anomaly detection on a daily basis. By comparing actual spending against predicted trends, the system can automatically notify account owners when costs depart from expected patterns, allowing for faster intervention.

Data Pipeline and Forecasting Architecture

The scale of the operation is immense, processing approximately 3 billion rows of data across 500 columns each month. CLEA ingests billing information from AWS Cost and Usage Reports, ensuring data consistency by aggregating it into a daily cost per account per service. Because the pipeline is scheduled only after the delivery of billing reports is confirmed, the system maintains high data integrity with a one-day analysis lag.

With thousands of accounts utilizing services such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3), the volume of potential time series data is vast. To manage this, BMW employs Prophet, an open-source forecasting library developed by Meta, to establish baselines for each account-service combination based on 365 days of historical data. This approach allows the system to generate expected spend metrics that adapt to historical seasonal trends.

Orchestrating Scalable Analysis

Efficiency was a primary design requirement, resulting in a serverless architecture that completes the entire analysis for 14,000 accounts in roughly 20 minutes for approximately $50 per month. The process is orchestrated by AWS Step Functions. The workflow begins with a Lambda function that identifies active accounts and saves a list to storage. A Distributed Map then triggers concurrent Lambda functions to forecast costs for each account, ensuring the system remains performant as usage grows.

Refining Alerts to Minimize Noise

A primary challenge in implementing automated alerts is avoiding 'alert fatigue' caused by false positives. BMW discovered that fixed-dollar thresholds were insufficient because smaller accounts would be ignored while larger, evolving accounts would trigger constant, unnecessary notifications. Instead, CLEA uses a multi-layered filtering approach that accounts for the specific trajectory of each service.

The detection logic calculates the 'impact'—the difference between actual and expected spend—and flags anomalies that fall outside the confidence intervals produced by the Prophet model. From there, additional filters are applied. These include account-cluster filtering, which requires a minimum dollar impact based on the account's historical scale, and service-specific thresholds for volatile workloads. For instance, services like AWS Glue and Amazon Athena often exhibit higher variance by design, and thus they are subject to a 60% deviation threshold rather than the standard 40%.

Sources

• AWS Machine Learning BlogHow BMW Group detects cost anomalies across 14,000 cloud accounts