By Neil Kumar, Principal Product Manager
Every data platform looks fine at small scale. The architectural questions vendors have argued about for a decade, like separated versus attached storage, federation versus unification, and slot pricing versus per-core, are mostly invisible while caches absorb your workload. Two platforms with radically different designs return the same dashboard in roughly the same time, and architecture feels like a preference for engineers to debate at conferences.
It stops feeling like a preference the moment your queries push past what cache can absorb. Past that point, architecture becomes the dominant variable in how your workload behaves: in latency, in cost, and in whether the system holds up at all.
Agentic analytics is pushing every team toward that threshold faster than they expect, and you may already be closer to it than you think. This post covers the threshold itself: what it is, how to know where you stand, and what it means for the architecture choices you’ve already made.
The threshold question
Start with a single diagnostic question: are your queries, or your agents’ queries, routinely touching enough of your dataset that cache stops absorbing the work?
There is sixty years of systems research behind that question. Peter Denning formalized working-set theory in Communications of the ACM in 1968: a system performs well when its working set fits in fast memory and degrades sharply when it doesn’t. The same principle governs every data platform on the market today. Cache hits paper over architectural differences. Cache misses surface them.
The threshold is not a data-size bright line. It depends on working-set size, query selectivity, concurrency, and access pattern. A 50TB workload of broad agentic scans can be past the threshold while a 200TB workload of narrow point lookups never gets close.
So instead of a size rule, use a check sheet. If several of these are true, you are likely approaching the threshold or already past it:
-
Broad scans. Most of your queries touch a non-trivial fraction of the dataset rather than a handful of rows.
-
Exploratory patterns. Query shapes rarely repeat, so yesterday’s cache doesn’t help today.
-
Climbing concurrency. Multiple agents, users, and use cases are hitting the same data at once.
-
Continuous load. The system never really goes idle.
-
Cross-modal queries. A single workload mixes relational, geospatial, ML, graph, vector, and real-time work rather than one shape of analytics.
-
Scale. Your dataset is at 100TB and growing. Treat this one as an anchor rather than a verdict. Most teams that haven’t tested past it haven’t seen the threshold yet, but it can show up earlier.
Why agents cross it faster
Existing data platforms were built around an implicit business intelligence assumption: high cache hit rates, repeating queries, and predictable patterns like daily refreshes and hourly dashboards. The cache absorbs the work, and the architecture underneath rarely gets stressed.
Agents violate every one of those assumptions. They query exploratively, and most of their queries are wrong turns the agent will discard. Each wrong turn is still a real scan, and since the next query is never the same as the last, caches don’t help. They branch, turning a single question into 10 to 30 sub-queries that touch different parts of the dataset with low locality. They never clock out, which quietly erases the auto-suspend savings that make consumption-priced platforms tolerable. And they cross modalities, so the cache that works for OLAP does nothing when the next call is an ML inference or a spatial filter.
Benchmarks of agent behavior, including AgentBench (Liu et al., 2023) and ToolBench (Qin et al., 2023), show the same patterns empirically: high branching, low repetition, and continuous tool calls. These are measured behaviors, not a hypothetical workload profile.
The hidden write workload
Most discussions of agentic data workloads focus on reads. What usually gets missed is that agents also write, heavily.
The data point comes from Databricks itself. Their 2026 State of AI Agents report, drawing on data from more than 20,000 organizations including over 60% of the Fortune 500, found that on Neon (the Postgres engine behind Databricks Lakebase) 80% of databases are now created by AI agents, up from 0.1% in October 2023. Database branches, the isolated test and dev environments agents use to try things, went from 0.1% to 97% over the same period.
That is what agents do with write access. They materialize scratch tables for multi-step reasoning, persist intermediate results across loop iterations, and branch environments to test alternative paths, experiment cheaply, rewind, and clean up. Databricks’s framing is right: agents interact with databases very differently than humans do.
The architectural consequence is that your working set is no longer just your data. It now includes the agent’s working state, and that state is growing at machine speed. This compounds the cache problem, because the agent’s scratch tables compete with your source data for the same cache and the same I/O path. It also breaks an economic assumption baked into most cloud data warehouse pricing, which treats writes as an occasional ELT cost rather than a continuous agent-generated one. Storage bills, slot consumption for writes, and compaction overhead all scale with how chatty the agent is.
This is the workload the OcientAIQ™ Unified Data Platform was engineered for on the write side: in-cluster loading that delivers seconds to time-to-query, reads and writes on the same local NVMe path, and per-core pricing that never bills per materialization. An agent can scratch, branch, and persist as much as its reasoning requires without degrading the read path or running up the bill.
The I/O question everyone avoids
With a cache-miss-dominated read workload and a growing materialization write workload contending for the same path, the binding constraint on latency becomes the least glamorous part of the system: how bytes get to and from the CPU.
The dominant cloud architecture (Snowflake on S3, BigQuery on Colossus, Databricks on cloud object storage) separates compute from storage and connects them over the network. That design is brilliant for elastic, intermittent workloads. For always-on agentic workloads it becomes a structural ceiling, because every cache miss is a network I/O and every materialized write takes the same round trip.
Ocient took the opposite approach with Compute Adjacent Storage Architecture (CASA), which puts data on local NVMe attached to compute via PCIe rather than the network. As our documentation describes it, the design “keeps records near and accessible for computation without a separate storage layer,” avoiding common database bottlenecks like network capacity limits.
The tradeoff is real. Separated storage is more elastic, and attached storage is faster per byte. Neither is universally right. The question is which tradeoff a workload at the threshold actually wants, and for workloads that are continuous, exploratory, and write-heavy, elasticity you never use is worth less than an I/O path you hit on every query. That is the bet OcientAIQ made deliberately, and it is why CASA has been the foundation of the platform from day one.
Six modalities, one copy
Agent reasoning loops rarely stay in one modality. A single business question can call for a rollup over relational data, a geospatial filter, an ML score, a graph traversal across a relationship network, a vector similarity search, and operational data that landed seconds ago. Six different shapes of analytics, and one agent question routinely touches four or five of them before it converges.
That cross-modal shape is also what carries a workload past the cache threshold first. Each modality has its own access pattern and its own derived structures, so a loop spanning all six is six working sets competing for cache at once rather than one. In a best-of-breed stack, where each modality lives in a separate engine reading from its own decoupled storage, every cache miss becomes a separate network round trip, and on dependent reasoning steps they compound.
OcientAIQ runs all six as first-class composable citizens in one engine over one owned, co-located copy: relational and OLAP, geospatial, machine learning through OcientML™, graph analytics, vector search, and real-time rollups. They compose inside a single native query plan, joinable like any other subquery, reachable from one mostly unified SQL surface, with GQL carrying the graph syntax. Graph and vector are the newest of the six and are rolling out to customers now. The payoff shows up exactly at the threshold: a cross-modal query stays one working set on one local NVMe path instead of fanning out to six remote stores, so the workload that pushes every other architecture over the edge fastest is the one Ocient absorbs without ever leaving the engine.
A brief word on economics
Cloud data warehouse pricing was designed for BI: pay when you query, suspend when you don’t. That is rational when usage is intermittent. For continuous agentic workloads, the warehouse never suspends and the materialization meter never stops. Worse, per-query and per-byte pricing penalize the exploratory behavior agents need, so teams either throttle their agents (which makes them worse) or accept unpredictable bills (which makes finance unhappy). This is why OcientAIQ prices per core rather than per query: agents can explore as widely as their reasoning requires while the cost stays fixed and predictable. The full pricing argument deserves its own post, so we’ll leave it there.
What this means for buyers
If your workload is read-heavy, low-concurrency, and intermittent, and your queries touch small subsets of small datasets, the conventional cloud data warehouse playbook is fine. Most teams are not past the threshold today.
But if you recognize several signals from the check sheet, including broad scans, exploratory queries, rising concurrency, always-on load, cross-modal work, and a growing dataset, you should be paying attention now. That doesn’t mean you need to switch architectures this quarter. It means the cost of getting it wrong grows quickly once you cross.
The teams already past the threshold are the customers Ocient serves today: CSPs running flow and CDR analytics, AdTech platforms processing billions of interactions, government and national security teams fusing multi-source data, and financial fraud teams scoring in real time. They hit the architectural ceiling years before the rest of the market, and they are why the platform exists. Much of the industry will join them in the next two to three years.
The right time to think about this is before
Every platform looks fine while cache absorbs the work. The right time to think about architecture is before you cross the threshold, not after, because by then the choice has already been made for you by workloads you can’t throttle and bills you can’t predict.
This is the problem Ocient was built to solve. The OcientAIQ Unified Data Platform brings AI directly to petabyte-scale enterprise data so agents, analysts, and applications get trusted answers without moving data across fragmented systems: compute adjacent storage for the I/O path, all six analytical modalities composable over one owned copy, and fixed per-core pricing for always-on workloads.
If you recognized your workload in the check sheet, the fastest way to find out where you stand is to test it against your own data. Book a demo and our team will walk through your workload profile with you. For the stack-level version of this argument, see our companion post on the modal tax and why agentic analytics breaks the best-of-breed stack.