Calif., Aug. 6, 2026 — Zilliz, a leading AI data infrastructure company and the creator of Milvus, recently announced a major update to VectorDBBench (VDBBench), an open- as a first-class dimension alongside performance under production conditions — not just how fast a system performs in an idealized test
Most benchmarks optimize for a single headline number: peak queries per second (QPS) on static, fully indexed data. But teams rarely choose a vector database on speed alone. They need to know what it costs to hit a target QPS, when newly written data actually becomes searchable, how filters and payload size change the query surface, how a system holds up across thousands of tenants, and how it responds on the first query after sitting idle. VDBBench now measures those dimensions directly.
“Most benchmarks answer one question — how fast can it go on ideal data,” said James Luan, VP of Engineering at Zilliz. “But teams buy on total cost and real production behavior. We built VDBBench as an open, reproducible benchmark so any team can see what a vector database will actually cost and how it will behave on their own workload — and decide for themselves.”
Building on VDBBench 1.0, which moved benchmarking closer to production by testing under streaming ingestion, filtering, recall, and concurrency, the latest release adds cost as a first-class dimension through four new cloud-oriented test cases:
- Insert readiness and write cost — separating when a write is accepted, when it becomes searchable, and when it is fully indexed, alongside the cost of loading data.
- Payload-aware search — how response shape (IDs, metadata, or full vectors) and filters change QPS, latency, and recall.
- Multitenant search — sustained throughput across many tenants and namespaces, as in SaaS workloads.
- Cold-start latency — the first query against a collection that has gone idle, versus the warmed path.
The results feed a new Cost Leaderboard with a cost-performance (Pareto) view that models operating cost at target QPS levels, showing where usage-metered serverless pricing is most efficient at low, spiky volume and where provisioned, flat-rate capacity becomes more cost-effective for the sustained traffic typical of production workloads.
Built to Benchmark Any Vector Database
VDBBench is open-source and supports more than 30 vector databases and search systems. To show the new cases in action, the Cost Leaderboard launches with a sample evaluation of several widely used managed vector databases — Pinecone, Turbopuffer, and Zilliz Cloud — chosen to illustrate how differently products can behave on freshness, filtering, multitenancy, and cold starts. The comparison is a demonstration, not a ranking: because the benchmark is open, teams can reproduce every case, swap in any other candidate, and run it against their own production-like data.
In the sample run, for instance, Zilliz Cloud returned newly inserted data as searchable immediately at a single-digit-dollar bulk-load cost, held a near-flat cold-start latency profile, and grew more cost-effective as sustained query volume rose — an illustration of the performance-and-cost balance the new cases are built to reveal. A product tuned for a different workload could show different strengths, which is exactly why every case is open to reproduce.
“This isn’t about naming a winner,” Luan added. “Different databases are built for different workloads, and the right choice depends on freshness, filtering, tenancy, budget, and many other factors. The results aren’t ours to declare — they’re yours to reproduce.”
The VDBBench Cost Leaderboard is available now at zilliz.com/vdbbench-leaderboard-v2. VectorDBBench is opentheir own candidates, and share results
For more details, read the VDBBench blog.
Zilliz is a leading AI data infrastructure company and the creator of Milvus, the world’s most widely adopted open-ion Docker pulls. Zilliz helps enterprises and AI startups make their unstructured data searchable, analyzable, and governable — turning text, images, audio, video, and more into a strategic asset for production AI
Zilliz’s technology centers on Milvus and Zilliz Cloud. Milvus is an open-source, lake-native vector database purpose-built for 100-billion-scale vector search. Zilliz Cloud extends that foundation into a fully managed Vector Lakebase platform, combining the high-throughput, low-latency serving capabilities of vector databases with the openness, scalability, and economics of multimodal data lakes. Zilliz powers more than 10,000 enterprises and AI-native startups worldwide, including MiniMax, OpenEvidence, Filevine, Exa, Salesforce, and Read AI.
Headquartered in Redwood Shores, California, Zilliz is backed by leading investors, including Aramco’s Prosperity 7 Ventures, Temasek’s Pavilion Capital, Hillhouse Capital, 5Y Capital, Yunqi Partners, and Trustbridge Partners. Learn more at Zilliz.com.
The Big Data Inside Amazon’s New Fire Phone
The new Fire phone that Amazon launched this week looks like your ordinary black smartphone,…
Three Reasons to be Scared of the Internet of Things
We know the Internet of Things forecasts: 50 billion connected devices by 2020. Apparently, there’s…
Wanted: Intelligent Middleware That Simplifies Big Data Analytics
We’ve seen tremendous technological innovation in the data analytics space over the past 10 years….
GPUs Tackle Massive Data of the Hive Mind
LIVE from GTC12 — The flock of birds that weaves seamlessly through the sky, propelled…
Why Hadoop on IBM Power
In the quest to achieve data-driven insight, Hadoop running on Intel X86-based processors has emerged…
Datanami’s Leverage Big Data Summit Wraps Up
Dialog and networking were on the Datanami agenda this week as we kicked off our…
What Google DeepMind’s Departures Say About the AI Talent War
Google DeepMind has spent the past few years at the center of the AI boom….
Why AI Still Hasn’t Had Its Einstein Moment
AI is having a remarkable run in science. It’s helping researchers predict protein structures, improve…
AI Infrastructure Spending Has Surpassed $1 Trillion. Here’s Where the Money Is Going and What’s Next
AI used to be a race to build the best model. Now it’s a race…
Token Optimization in Enterprise AI: How Context Architecture Determines Whether Your AI Program Scales Economically
Enterprise AI budget forecasting is mostly guesswork right now. Most organizations set their initial allocations…
AI Is Creating a New Market for Infrastructure Data. S&P Global Wants In
The AI boom has kicked off a massive wave of data center construction around the…
Why the AI Semantic Layer Is Becoming the Foundation of Enterprise AI
BigDATAwire recently spoke with Cindi Howson, Chief Data and AI Strategy Officer (CAIO) at ThoughtSpot,…
