Circle K Cuts DevOps Incident Triage Times With Amazon Bedrock
By deploying an agentic triage system on Amazon Bedrock, Circle K successfully reduced its mean DevOps incident triage time by up to 75 percent.

The Scaling Challenge of Cloud-Native Infrastructure
For many DevOps teams, handling alerts involves a familiar and time-consuming manual investigation pattern. Before any remediation can begin, engineers often spend 45 to 60 minutes pulling logs, checking service health, and correlating symptoms across disparate tools. At Circle K, this operational challenge scaled directly alongside its expanding business footprint. The enterprise operates more than 17,200 convenience and fuel retail stores across 27 countries, backed by a cloud-native infrastructure that powers point-of-sale systems, loyalty applications, and electric vehicle charging applications.
As digital operations expanded, the overhead required for incident triage grew proportionally. More services naturally generated a higher volume of alerts, and each alert historically demanded manual investigation before troubleshooting could even commence. To resolve this bottleneck, Circle K turned to foundation models made accessible through [Amazon Bedrock](https://aws.amazon.com/bedrock/), a fully managed service that allows organizations to leverage advanced AI models via a single API.

Automating Context Gathering and Investigation
Before building the automated solution, Circle K's DevOps team monitored infrastructure across multiple AWS accounts using Grafana. A typical operating day brought alerts concerning container restarts, elevated error rates, latency spikes, and failed deployments. Investigating these signals manually required a repetitive sequence of steps, including reviewing recent operational logs in [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/), examining application-level traces, checking task status, and cross-referencing internal runbooks.
Recognizing that these manual queries would only multiply as more services came online, the team launched a research and development initiative. Their primary goal was to automate repetitive context gathering while preserving the operational judgment of human engineers. They evaluated multiple approaches, including self-managed open source frameworks, but ultimately selected AWS managed services to avoid the substantial overhead of model hosting, security controls, and prompt versioning.
Designing the Agentic Architecture
The initial iteration of Circle K's solution was built using [Amazon Bedrock Flows](https://aws.amazon.com/bedrock/flows/) to coordinate a specialized group of cooperating agents. The automated pipeline triggers whenever a Grafana alert publishes a notification to an Amazon SNS topic, which subsequently forwards the payload to an Amazon SQS queue. A dedicated AWS Lambda function polls the queue, extracts the necessary payload data, and invokes the workflow.
Inside the orchestration layer, multiple agents execute tasks in parallel to investigate the root cause of an incident. The AWSResourceExplorerAgent checks infrastructure details across workload accounts, including container health and metric data. Concurrently, another specialized component queries application-level logs using [Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/) to keep individual context windows manageable given the high volume of log data.

Accelerating Response Times and Maintaining Security
Following the parallel exploration phase, a final analysis node synthesizes the gathered data into a structured summary delivered directly to Slack. On-call engineers now receive immediate, actionable context rather than raw alerts, allowing them to begin troubleshooting immediately. According to internal incident response metrics, this agentic approach has successfully reduced mean triage times to under 15 minutes, translating to a 67 to 75 percent reduction in initial investigation overhead.
Data privacy and security remained a critical requirement throughout the deployment. Because production logs and internal infrastructure details can contain sensitive information, Circle K benefited from the native data handling assurances provided by the platform, ensuring customer operational data is not utilized to train underlying foundation models.
Sources
- AWS Machine Learning BlogHow Circle K accelerated DevOps incident triage to under 15 minutes with Amazon Bedrock
Continue chronologically





