By Jenna Boller, VP of Marketing
Three years ago, Ocient began surveying 500 IT and data leaders to get a pulse on the latest trends, concerns, and opportunities within the high-intensity data and analytics industry. Each year, our survey and report has uncovered top challenges and priorities for the industry spanning data, AI adoption, and cloud computing.
Check out the latest trends, insights, and challenges among IT and data professionals in Ocient’s latest installment of “Beyond Big Data: Reaching New Altitudes.”
Year-on-year: data volume presents an opportunity and a challenge
In our latest report, respondents continue to validate that organizations (100% this year!) see value in scaling the amount of data analyzed for a business outcome or benefit – but over a third (35.9%) still can’t harness the full potential of their data. In other words, data collection is not an issue, but observing source data, cleaning it, preparing it, getting it into the right format and/or table, and then running machine learning, analytics, and more, remains a challenge.
While we always see a decent percentage of respondents facing challenges, more have now gotten started with replacing their existing data and analytics solutions. This number jumped from 35% last year to 41% in 2024. Fueled by the need to replace legacy systems and upgrade to modern platforms to support GenAI, IT and data leaders are making moves at a steadily increasing rate.
Ocient’s key findings for IT and data professionals in 2024
Since launching the Ocient Hyperscale Data Warehouse™ – our flagship product – in May 2022, we’ve tracked industry trends through conversations with customers, partners, and industry analysts, as well as through mainstream news and headlines.
This year, three key trends rose to the top, specifically:
- Unpredictable costs are limiting innovation – more than two thirds (68.3%) of respondents are often surprised by their data analytics spend with 40% of respondents feeling limited by cost constraints
- Cloud-only is on the decline – 88% of survey respondents are re-thinking their data analytics infrastructures, with only 25% (down from 35% last year) investing in cloud-only architectures
- Energy consumption for data and AI is now a top concern – growing concerns around energy consumption popped up for the first time in this year’s survey with more than HALF (53%) of our respondents citing concerns around energy consumption – and availability – to fuel their data and AI workloads
Here’s a quick preview of the key findings from this year’s Beyond Big Data report:
Five top trends from this year’s “Beyond Big Data” report – to unpack these, download the full report here
Now, to dive deeper into these findings…
The cost of managing data is stifling innovation
Cost unpredictability has long been a pain point for our Beyond Big Data survey respondents, and this year was no different. One data point that stood out:
40% of IT and data leaders report that their ability to use their data to the fullest is limited by cost.
Among the top drivers for variability within data and AI workloads:
- 64% said cloud costs are driving unpredictability
- 57% face challenges predicting costs associated with systems integrations
- 54% lack transparency around data movement costs
To read the complete list of cost culprits, download the full report
Data leaders are re-thinking their cloud strategies due to concerns about cost and control
When we first started briefing industry analysts about Ocient’s flexible deployment options years ago, they often skipped right over our on-premises options because “everyone is going to the cloud.” We heard this repeatedly – and sometimes vehemently – from the analyst community despite requests from our customers to continue developing on-prem and private cloud deployments for workloads where they needed more control over their data, or a CapEx pricing model to meet the requirements of their business.
Over the past six months, we’ve seen these conversations shift quickly as the rapid drive to invest in GenAI has quickly driven up the cost of cloud computing and encouraged organizations to bring more data and workloads back on prem. In addition to cost consciousness, IT and data leaders gain more control over their data and line of sight into future operating costs when their data resides on premises.
Energy efficiency for data and AI is quickly becoming a top priority
As you can see above, many of our key findings this year are either directly or loosely related to a shift in GenAI research and development. In fact, 93% of survey respondents plan to make investments in AI within the next 12-18 months. Our findings around energy efficiency are no different.
Overall, we see organizations prioritizing net-zero emissions goals and reporting on their greenhouse gas emissions. 75% of respondents said their organizations have formal carbon-neutral commitments.
As sustainable IT gains traction in the industry, we were pleased to see 31% of this year’s respondents say they would be motivated to switch or upgrade their current data warehouse or database solution to reduce the energy consumption of their workloads. Reducing energy consumption decreases both carbon emissions and the cost of running compute-intensive data and AI workloads – both great reasons to consider more energy-efficient data and analytics solutions.
Reaching new altitudes with a comprehensive approach to Big Data analytics
As we see the pendulum swing back from “all-in-on-cloud” to a more diversified approach, as priorities converge around lowering the energy consumption of compute-intensive data and AI workloads, and as we feel more pressure to control costs, our latest Beyond Big Data report outlines how IT and data leaders are tackling a fast-changing landscape and moving their organizations forward.
It will be interesting to see what the next 12 months bring in terms of new challenges and opportunities for IT and data leaders. Based on this year’s insights, I expect to see continued innovation around sustainable IT solutions that enable responsible business growth, a growth in hybrid and private cloud data analytics solutions, and cost pressures that may drive the consolidation of complex solution stacks.
At Ocient, we’re excited to be on the cutting edge of delivering energy efficient software and solutions – enabling customers with outstanding price-performance across their data and analytics ecosystems as well as custom solutions support to optimize their enterprise data architectures and accelerate time-to-market for new data products and services. We’d love to tackle your toughest data challenges and reach new altitudes together. To get in touch, you can contact us here.