By Matthew Orwin, Solution Architect, Ocient National Security Solutions
There’s a diagram we’ve used internally for a couple of years now, since early in Ocient’s work with data-intensive customers, and we still find ourselves pulling it up in almost every technical conversation. On one side, a long list of the data types an organization actually generates: network telemetry, identity and access logs, endpoint data, financial transactions, cloud and SaaS activity, badge and location data, DNS and hosting records, call and connection records, threat and intelligence reporting, vulnerability scan and asset inventory data. On the other side, a long list of the questions those data types can answer when they’re joined together rather than kept apart. In the middle, one engine.
We like this diagram because the two lists on either side of it are not really the point. What matters is the engine between them.
The pattern behind every use case we write about
Look back at our other writing on this platform and a pattern shows up, even though each piece treats a single, narrow problem. Our post on retrospective IOC sweeps joins flow and DNS history against a threat intelligence table. Our post on why prevention is a data problem joins vulnerability scan results against asset inventory, network topology, and patch history. Our post on what petabyte scale unlocks describes the same join pattern showing up in domains that have nothing to do with cybersecurity. In every case, the hard part was never the SQL. A join across two tables sharing an entity key (an IP address, a device ID, an account number, a badge ID) is not a novel idea. It is one of the oldest ideas in data analysis.
Historically, what made the join impractical, in most environments, was that the two tables never lived anywhere they could actually be joined. Security telemetry sits in a security tool built to ingest logs and alerts, not a fraud ledger or an HR system. Business data sits in a warehouse built for structured, comparatively low-volume records, not carrier-scale network flow. Getting both into the same queryable place, at the volume and retention depth either one alone would need, has always been a project, and by the time the project finishes, the question that motivated it has usually changed.
The diagram is our answer to that problem stated as plainly as we can state it: bring the data sources in on one side, put them all in one engine built to hold and query all of them at once, and the list of questions on the other side stops being a wish list and starts being a query. We will cover exactly what that unification takes in an upcoming post. The short version: one engine that ingests both without dropping anything, retains both at full history, and lets you query across both together, not three separate systems stitched together after the fact.
Why the list of outcomes keeps growing
When we first built this diagram, the list of things an organization could ask once its data was unified was shorter than it is now. It has grown every year since, not because we added features, but because customers kept finding joins we hadn’t anticipated.
- A fraud team realized their transaction data and the SOC’s authentication logs shared an account key. Joined, the two answer something neither system can answer alone: whether a login anomaly and a suspicious transaction are the same event, seen through two different tools. That’s the join banks are increasingly running to investigate coordinated fraud — correlating authentication logs against transaction data instead of working each in isolation.
- An insider-risk program realized their access-review data and the network team’s egress logs shared a device key. Later, the same program pulled in DLP alerts and HR signals — a resignation notice, a performance flag — because the technical alert rarely tells the whole story on its own. An unusual data transfer means something different from an employee who gave notice than from one who didn’t.
- A vendor-risk team realized two years of flow data and their procurement system shared a hostname. Joining outbound network traffic against the vendor list turned third-party risk monitoring from an annual questionnaire into a standing query: which vendors the organization is actually talking to, and whether that matches what was contracted for.
- A vulnerability management team realized their scanner output, asset inventory, and network topology data could finally be joined into one prioritized, always-current picture. Instead of a quarterly spreadsheet exercise — and the days it usually takes someone to work out by hand which of thousands of findings actually matter — the join does the prioritization: which vulnerabilities sit on which assets, reachable from where.
None of these were built as products. They were queries, written by someone who finally had both tables in the same place and asked, “wait, could I just join these?”
That is essentially what the outcomes side of the diagram represents. It is not a finished list. It is a running tally of what customers have asked so far, and the newest entries are usually the ones nobody on our side thought to put there first.
What has to be true for the middle of the diagram to work
None of this works if the engine in the middle can’t actually do three things at once, and we’ve made the case for each of them individually elsewhere on this website: ingest every one of those data sources at the rate it’s actually produced, without falling behind and forcing a choice about what to drop; retain all of it, at full resolution, for as long as the question might need to look back; and query across all of it, joined arbitrarily, fast enough that someone can ask a follow-up question in the same sitting instead of filing a ticket and waiting.
Most platforms are built to do one or two of those well. A tool built to ingest and alert on security telemetry usually isn’t built to also hold a transaction ledger. A data warehouse built to hold a transaction ledger usually isn’t built to ingest network flow at the rate a large enterprise produces it. Ocient’s position, the reason this diagram has held up as a way to explain what we do, is that we built specifically for the case where an organization needs all three at once, across data that doesn’t share an owner, a format, or a historical home, at a scale most platforms treat as an edge case rather than the starting assumption.
The engine in the middle is not, to be clear, a replacement for the tools on either side of it. The SIEM the SOC works in, the fraud system the fraud team uses, and the case management and BI tools each group already runs all stay in place. This layer sits underneath them as the shared, full-history data foundation they can all draw on and feed results back into. The diagram is about giving those existing tools a common substrate deep enough and complete enough to answer questions none of them can reach on their own, not about consolidating them into one screen.
There’s a fourth thing that has to be true, and it usually becomes the most important one the moment an organization looks seriously at the left side of the diagram. Once you’re bringing financial transactions, HR records, identity and badge data, and location history into the same engine as network telemetry, you have concentrated a large share of the organization’s most sensitive data in one place. That is exactly the point (it’s what makes the joins on the right side possible), but it elevates the importance of the question “where does this platform actually run, and who can see what’s on it?”
The exact shape of that question depends on who’s asking. For an institution operating across borders, it’s a data sovereignty question in the strict sense: whose legal jurisdiction governs this fused dataset, given where it sits and who operates the infrastructure. That question gets sharper the moment a US-headquartered vendor’s cloud is involved, since data held in a foreign data center can still be reachable by US authorities under the CLOUD Act if the operating company is American. For a US-based institution working with a US cloud provider, that particular question is largely already settled, and the concern shows up instead as third-party risk. The interagency guidance the OCC, Federal Reserve, and FDIC finalized in 2023 (SR 23-4, OCC Bulletin 2023-17, FDIC FIL-29-2023) requires due diligence and contractual protections scaled to exactly this kind of concentrated, sensitive dataset. It also shows up as cloud concentration risk, the Fed’s own supervisory concern about too many institutions depending on too few providers. Increasingly, it also shows up as an AI-governance question: whether any of this fused, sensitive dataset ends up training a vendor’s models along the way.
For institutions operating in or with the EU, the same question increasingly arrives as DORA, the EU’s Digital Operational Resilience Act, which imposes its own due-diligence and concentration-risk requirements on financial institutions’ cloud and ICT vendors. And wherever GDPR applies, there’s a real tension between data minimization — keep only what you need, only as long as you need it — and the multi-year retention that makes retrospective investigation possible in the first place. That tension doesn’t resolve itself, but it does argue for a platform that can enforce retention and access policy precisely enough to satisfy both.
This is a large part of why organizations come to us in the first place, whichever version of the question they’re asking: Ocient offers an open, hardware-agnostic, model-agnostic architecture that deploys inside the data centers you already control (on-premises, private cloud, or fully air-gapped) rather than requiring the data to move to us. You get full visibility and traceability into everything running against that fused dataset, because it is all running on infrastructure you own. We are not asking anyone to bring us their data to make our product better. We are building the engine that makes the data you already have to keep — and secure — useful.
The question that actually matters
Every organization we’ve done this work with eventually stops asking us what the platform can do and starts asking a different question of themselves: Given that they can now join anything against anything, at full history and interactively, what do they actually want to ask?
That is, genuinely, the fun part, and it’s also the part we can’t do for you. The use cases we’ve written about elsewhere (beaconing detection, retrospective sweeps, the petabyte-scale examples we’ve covered separately) are not an exhaustive list. They are the questions we’ve already watched customers ask. The diagram’s outcome list was short when we first drew it and it has never stopped growing since. Every organization we bring onto this kind of platform adds to it in a way we didn’t predict going in.
So we’ll end this post the way we end most of these conversations in person: the data sources on the left are probably close to what you already have. The engine in the middle is what we build. The list on the right is now yours to write.
Ready to find out what Ocient can do for you? Contact us.