By Jenna Boller, VP of Marketing at Ocient
Over the past year, Ocient has continued to enhance its real-time analytics (RTA) capabilities, leveraging our Compute Adjacent Storage Architecture (CASA), comprehensive indexing, continuous loading, and high concurrency features and capabilities to improve response times on real-time analytics workloads for our customers.
From adtech to telecommunications, our customers require fast response times for compute-intensive workloads that require large amounts of data to be always available and always fresh. To put our RTA capabilities to the test, our research team recently ran tests on the same hardware using the flattened star schema benchmark with a scale factor of 90000 (536 billion rows). The team had intended to run a 1 trillion row benchmark, which was loaded into Ocient, but that dataset proved to be too large to load into Druid on the same hardware.
Ocient vs. Druid Benchmark Results
The results of the benchmark demonstrated:
- Ocient’s superior query performance across 13 queries from the flattened star schema tests
- Ocient’s superior query performance while concurrently loading 1.3 million rows per second
- Ocient reaching maximum performance over Druid while loading 1.3 million rows per second + running 10 concurrent queries – one of the staples of an RTA workload
We’re excited to share the results of this RTA performance comparison alongside the significant cost savings Ocient offers in the full benchmark report, available here or check out the summary video below.
While Ocient supports several customers in the real-time analytics space, many of them are running mixed workloads with requirements that push far beyond the capabilities of an RTA database. Ocient customers have successfully consolidated historical reporting and data exploration – via machine learning – into a single Ocient deployment, retiring multiple point solutions in the process and realizing significant savings on cost, resourcing, and system management along the way.
In a recent solution we deployed for one of our adtech customers, we encountered multiple solutions deployed – Spark, Druid, MongoDB, and a homegrown Java ETL process – to manage the customer’s scheduled and ad hoc reporting for log-level aggregation analytics. The challenges they faced with this data ecosystem included:
- Massive data volumes – the customer needed to process billions of ad requests daily, with each request containing over 80 dimensions and 100 metrics for analysis
- Disparate data systems – moving data in between so many systems led to inefficiencies, data silos, and inconsistent data analysis
- Limited real-time insights – scaling this solution introduced additional challenges, limiting critical business functions like ad pricing optimization, audience targeting, and content strategy development
Ocient vs. Druid Architecture Examples
The solution architecture spanned RTA, data warehousing, and ETL solutions with data moving in between systems contributing to latency, complexity, and cost:
Ocient offered a robust solution designed to serve the various capabilities required, thus enabling our customer to consolidate their entire workload into Ocient. The workload migration included:
- Streamlining data operations – consolidating database systems and their entire data pipeline management into Ocient
- Shortened time to query – a roughly 2x improvement in data freshness from ingest to query availability
- Enhanced data capabilities and operations – enabling real-time optimization for ad pricing and content performance, advanced audience targeting, and predictive ad capabilities leveraging OcientML™
- Higher data cardinality – providing higher granularity of data and query execution across more dimensions of information
- Improved scalability and performance – leveraging automation for data ingestion and reducing the overall system footprint to right-size for future growth
The resulting solution architecture is effectively much simpler and easier to operate:
The solution migration, which was complex and multi-faceted in nature, was executed by our Customer Solutions and Workload Services team, saving the customer time and resourcing that could be invested in other areas of their business. In addition, the customer experienced zero downtime throughout the solution migration and re-platforming phase, and into production.
While industry standard benchmarks take a look at product performance on a specific set of database or data warehouse operations, there can be several hidden costs baked into the total cost of a solution development and deployment. At Ocient, we pride ourselves in developing end-to-end solutions tailored to our customers’ business requirements and use case.
In the case of the solution architecture above, the team at Ocient worked hand in hand with our customer to ensure the re-platforming of three disparate solutions into Ocient was smooth, opening the door to new capabilities and future growth without taxing the customer’s limited staffing and resources.
Let’s Talk Today
Whether you’re focused on real-time analytics or a variety of data analytics and data science capabilities, we’d love to explore how Ocient can address your data challenges while reducing your system cost, size, and energy consumption. Contact us to learn more about how Ocient can support your data engineering needs and enhance your real-time analytics capabilities.