Cloudflare Cuts DNS Build Delay and Memory Usage
A new targeted build planner and scheduler have optimized Cloudflare's DNS propagation pipeline, drastically reducing memory consumption and latency for massive zones.
Radwan AbdulhadiEditor and technology writer
Scaling Pressures on the DNS Pipeline
Cloudflare has significantly updated its DNS propagation pipeline to handle massive growth in record updates and zone setting modifications. When the company last wrote about our DNS propagation pipeline in 2022, the Zone Builder processed approximately 250 DNS record changes every second. Today, that figure has nearly tripled to about 700 changes per second, alongside roughly 100 additional events such as zone setting changes.
While the sheer volume of events is managed by the global network, the primary challenge stems from how some of those tasks scale with zone size. Certain domains now contain more than 20 million records, with expectations that these numbers will eventually reach the hundreds of millions.
The Evolution from Full Builds to Targeted Planning
Each event triggers a build process that reads the current database state, converts it into the edge format, and writes modifications to Quicksilver. Previously, zone setting changes would routinely trigger full rebuilds of entire zones, even those containing tens of millions of records, consuming immense computational capacity and memory.
Although per-record builds were already significantly faster than full builds, targeting zone settings proved more complex. Record events clearly identify individual changes, whereas zone events can originate from multiple Cloudflare products, lack explicit context regarding why a zone shifted, and can alter shared data across multiple zones with identical names.
How the New Targeted Planner Operates
To solve these challenges, engineers developed a targeted build planner alongside a new scheduler designed to combine work for identical zones. Rather than performing a full rebuild for routine updates, the system selects the smallest build it can mathematically prove is safe.
Customers typically modify DNS records via the DNS Records API or dashboard, while other services modify zone settings. These modifications generate Kafka events that are aggregated and passed to the Zone Builder for processing and distribution across the edge infrastructure.
Categorizing Build Types and Ensuring Safety
The updated architecture distinguishes between record keys and shared metadata keys. This distinction allows the planner to categorize work into five distinct build paths, ranging from isolated resource updates to full zone rebuilds when safety cannot be formally proven.
For instance, settings like Universal SSL may require updates to Certification Authority Authorization records at the zone apex. CAA records ensure certificate authorities issue certificates strictly in accordance with domain policies.
Measurable Performance Gains
The introduction of the targeted build planner and scheduler has delivered substantial operational improvements. Full builds have been cut by 40%, and peak memory usage across the pipeline has dropped by an impressive 85%.
Furthermore, the resulting memory headroom allowed Cloudflare to safely shorten necessary resilience waits in the pipeline. This reduction has effectively halved the total delay from the moment a DNS change is initiated until its data is successfully written to Quicksilver.
Sources
- Cloudflare BlogHow we halved DNS build delay and cut memory use by 85%
Continue chronologically
Related entity coverage
Related stories
Cloudflare Introduces Workers KV Instant Powered by Quicksilver
Cloudflare announces Workers KV Instant, a new mode offering sub-2ms p99 read latencies and global replication across its edge network.
Cloudflare Launches The Cold Start Startup Competition
Cloudflare introduces The Cold Start, offering five early-stage startups five minutes on stage at Connect for a $500,000 prize.
Internet Root Keys Set to Change on October 11
The DNS root is scheduled to change its key-signing key in a rare rollover event. Learn what this means for DNSSEC validating resolvers.
Cloudflare Q3 2026 Report Highlights Short DDoS Swarms
Cloudflare's Q3 2026 DDoS report reveals a shift toward short, high-frequency attacks alongside the collapse of CLDAP vectors and IoT botnet changes.
