By Shantan Kethireddy, VP of Customer Solutions at Ocient
In case you haven’t heard, Big Data is back from the dead in AdTech. The truth is it was never really gone, but the full promise and potential was frustratingly out of reach. Two things are changing that reality: a new breed of hyperscale data processing solutions are now capable of bringing in and cleaning up the petabytes of data coming from the AdTech ecosystem, in real time. And the rapid advance of AI and machine learning is finally giving AdTech firms practical and cost-effective access to the tools to make sense of that hyperscale, hyper-dimensional data — delivering prescriptive insights that connect “here’s what happened” to “here’s what you should do about it.”
We’re jumping into the hyperscale analytics future, and the current’s moving quickly.
Bringing hyperscale into the here and now
Let’s be real, though: Everything you just read is true. And it sounds great. But to be meaningful, we need to bring this lofty language to life in the here and now. AdTech companies are rightfully wary of big, bold claims about how technology can solve all problems.
But there’s a big difference between the many past promises of Big Data and where we are today: We’re already seeing real-world use cases and applications of these hyperscale tools. AdTech firms are using hyperscale data processing and analytics to mine the full scale and richness of their data, and they’re finding diamonds.
Four ways AdTech firms are applying hyperscale processing & analytics
1. Bid Opportunity Analysis
Real-time bidding (RTB) platforms produce petabytes of log and impression data every day, and bidders can process millions of bid opportunity requests per second. AdTech companies can capture myriad benefits if they can analyze this information. But that analysis must be both fast and precise — and it needs to be done continuously with fresh data. Conventional technologies can’t keep pace on their own, requiring a patchwork of point solutions (ETL, RTA, OLAP, data science) that forces AdTech companies to allocate important technical resources to managing this layered infrastructure, rather than focusing on revenue-generating activities.
Ocient’s AdTech platform is resolving these challenges, using hyperscale data processing and analytics solutions to both enhance and simplify their RTB campaign analytics ecosystems. AdTech companies are already using Ocient to combine transformation and loading, OLAP, ML, and RTA into the same platform — taking advantage of capabilities like I/O pipeline processing to query multi-dimensional structures instead of exploding tables into flattened structures for things such as segment analysis. This new generation of platform acts as both an SQL OLAP database and a transformation platform, as well as providing data exploration for machine learning (ML) and ML modeling/scoring. With an analytics engine that can keep up with RTB speed, these AdTech organizations are fully harnessing RTB data to improve return on ad spend, maximize yield on inventory, and pursue more sophisticated analytics objectives — all while recouping the price of replatforming in less than 18 months.
2. Attribution Analysis & Campaign Reporting
Attribution analysis is another area of massive unrealized potential. AdTech companies want to be able to analyze this complex, log-level data in real time, so they can understand which impressions are making an impact and which are falling short. Again, the keys here are fast streaming and transformation, fast aggregations, and precision on things like distinct counts/hyperloglogs (HLLs)—and that’s where conventional technologies get overwhelmed. As many as 20 billion records are coming into the system every day, but the bigger burden is preparing and transforming this raw, unaggregated data for effective analysis. As with all analytics, more data means more accurate insights, but AdTech companies struggle to balance the value of historical analysis and benchmarking against rising costs of storing all that historical data.
Done right, modern hyperscale data platforms are enabling AdTech leaders to take on the full firehose of attribution data and rapidly get it into usable format so they can scale up real-time analytics. They’re leveraging built-in hyperscale analytics capabilities to speed reporting and do ad hoc, time-sensitive queries without crashing the entire ecosystem. And because these platforms are built for a forward-thinking reality of Big Data, they’re able to cut down on the data storage footprint without losing the full richness and dimensionality of historical data.
3. Data Transformation & Activation
The third use case alludes to one of the core challenges common to nearly all AdTech data analytics workflows: converting an ever-more-heterogeneous stream of incoming data into a consistent, usable format. The conventional multi-step, multi-tool process is so common that everyone only uses the letters “ETL”. But each step in the extract, transform, and load process presents a risk to data fidelity. And each tool adds cost and administrative hassle.
The new generation of hyperscale data solutions brings the full ETL workload under one platform. State-of-the-art storage and processing can handle intensive operations without overwhelming resources, speeding time to analysis and insight. AdTech companies can activate more datasets and analyze data at full resolution, rather than being forced to downsample. This improves the accuracy and reliability of the business insights they extract. And on the back end, the consolidated ETL solution eliminates redundancies to reduce tech and admin costs.
4. Powering Machine Learning, AI & Data Science
AdTech platforms have been at the leading edge of AI and machine learning for the last two decades, eagerly using state-of-the-art tech to try to crack the code on Big Data. Those intelligent technologies have surged in sophistication in the past year, bringing the promise of Big Data back to life. But to take advantage, AdTech companies need to give AI and ML a steady source of rich and real-time fuel, feeding them massive amounts of multi-dimensional data.
With hyperscale data platforms like Ocient, AdTech companies are finally gaining the data processing speed and reliability they need to bring machine learning directly into their workflows. They’re using AI to work directly on raw source data, and embedding ML functionality within SQL statements to create a fully-fledged scientific computing platform. Moreover, by safely and securely unleashing ML and AI within their ecosystems, they’re able to answer previously-out-of-reach questions and uncover insights they’d never considered. This helps them maximize conversions, optimize CPM, and enhance targeting; get granular about how factors like device type and ad form influence outcomes; and do broader analysis on everything from fraud risks to emerging market.
Moreover, none of these uses cases come to life properly without the right workload management (WLM) behind them. Ocient is helping AdTech leaders put this critical piece of the puzzle in place, delivering a best-in-class WLM solution built to handle the realities of jobs often running concurrently with AdTech data. Whether it’s consolidated streamlining, file transformation, OLAP, real-time analytics, or data exploration with ML, Ocient’s WLM solution ensures it’s all harmonized using service classes that find the right resources at exactly the right time to optimize performance and minimize costs incurred across hybrid cloud environments.
See how Ocient is enabling the hyperscale future in AdTech
These four examples are just a sampling of the ways that Ocient AdTech customers are already using our end-to-end hyperscale data platform to harness their hyper-dimensional data and accelerate the path to new and better insights that drive performance. Download the full eBook to get deeper answers to the questions, “Why hyperscale?” and “Why Ocient?”
DOWNLOAD THE EBOOK: OCIENT FOR ADTECH