InfoQ HomepageNewsCloudflare Details Unified Data Platform Where Billing Workloads Account for 53% of Queries
Architecture & Design
From Log Noise to Incident Intelligence: A Maturity Model for AI-Assisted Observability (Webinar Jul 9th)
Cloudflare has detailed Town Lake, its internal unified data platform, reporting that billing workloads account for 53% of all platform queries. The system, built alongside an AI-powered analytics agent called Skipper, is designed to unify access to operational, billing, security, and business data previously distributed across fragmented systems.
According to Cloudflare, its global network processes more than one billion events per second across over 330 cities in 120 countries. Over time, data accumulated across Postgres databases, ClickHouse clusters, Kafka streams, BigQuery datasets, and object storage systems, making discovery and analysis increasingly complex. Town Lake was introduced as a unified SQL interface to query across these systems while maintaining governance and access control.
The platform is built on a lakehouse architecture using Apache Trino, Apache Iceberg, Cloudflare R2 object storage, and DataHub for metadata management. Cloudflare stated that a single query can join data across Postgres, ClickHouse, and Iceberg tables without moving data between systems. Supporting services handle ingestion, transformation, access control, and detection of personally identifiable information (PII).
Town Lake platform architecture (Source: Cloudflare Blog Post)
A key design element is its default closed governance model, where newly onboarded datasets remain inaccessible until automated scanning and human review are completed. Cloudflare uses an internal service called Skimmer, which combines automated classification with AI-based analysis to detect sensitive data. Human reviewers then validate or adjust classifications before granting access.
On top of Town Lake, Cloudflare built Skipper to provide natural language access to enterprise data. The agent translates user requests into validated queries using metadata, schema definitions, transformation lineage, documentation, and runtime inspection to improve accuracy. It is used for billing analysis, customer support investigations, business intelligence, and security workflows. Cloudflare described the system as a unified access layer where Town Lake provides a single SQL interface to company-wide data, and Skipper enables plain-language queries with auditable results.
Dmitry Alexeenko, Head of Enterprise Engineering at Cloudflare, said
Behind every Cloudflare request is data. Lots of data. Our team built Town Lake, a unified data platform, and Skipper, an AI data agent that turns plain-English questions into insights in seconds.
Cloudflare said tasks that once required complex SQL or manual investigation can now be completed in seconds using its unified data platform and AI agent. Billing workloads represent a majority of usage on Town Lake, which processed 91,760 billing-related queries from 324 employees in a measured period, spanning billing analysis, support investigations, and operational reporting. The company also reported that simplifying AI agent prompts improved accuracy, while consolidating overlapping tools reduced incorrect selections. Incorporating SQL transformation logic and data lineage into the agent context further improved understanding of business semantics beyond schema metadata.
Patrick Joubert, CEO of Rippletide, commented on the architectural implications
Putting an internal AI agent on top of a unified analytics platform is exactly where infrastructure teams start feeling the control problem. If the agent can reason across operational data, enforcement has to sit close to execution. Distributed deterministic checks are what let the agent act without turning the data platform into an uncontrolled action surface.
Looking ahead, Cloudflare plans deeper integration of Skipper with internal chat, ticketing, and development workflows. The company is also expanding its Transformer pipeline so teams can define curated datasets using SQL and metadata files that are automatically deployed, monitored, cataloged, and surfaced through DataHub and Skipper. Cloudflare also expects to migrate additional Town Lake workloads onto R2 SQL as the system matures.
