Ocient Favicon
The Ocient Hyperscale Data Warehouse

To deliver next-generation data analytics, Ocient completely reimagined data warehouse design to deliver real-time analysis of complex, hyperscale datasets.

Learn More
Pricing Icon

Ocient is uniquely designed for maximum performance and flexibility with always-on analytics, maximizing your hardware, cloud, or data warehouse as a service spend. You get predictable, lower costs (and absolutely zero headaches).

See How
Customer Solutions and Workload Services Icon
Customer Solutions and Workload Services

Ocient offers the only solutions development approach that enables customers to try a production-ready solution tailored to their business requirements before investing capital and resources.

Management Services Icon
Management Services

Tap into the deep experience of the Ocient Management Services team to set up, manage, and monitor your Ocient solution.

Learn More
Ocient Favicon
About Ocient

In 2016 our team of industry veterans began building a hyperscale data warehouse to tackle large, complex workloads.

Learn More
Published February 15, 2023

At the Whiteboard with Ocient: Loading and Transformation

George Kondiles, Chief Architect and Co-Founder at Ocient explains how Ocient loads and transforms hyperscale data.

Kondiles discusses the loading and transformation infrastructure for the Ocient Hyperscale Data Warhouse. This infrastructure is crucial for analyzing and querying extremely large datasets.

Ocient’s loading and transformation technology and infrastructure were designed with three key principles in mind: linear scalability, low time to queryability, and exactly once semantics.

The “firehose” of information, arriving from sources such as S3 (where there may be large numbers of CSV files) or a queuing system like Kafka (which may contain JSON documents), presents a significant challenge. But Ocient’s technology is capable of receiving and handling the high volume and large amount of information and storing it in our foundation layer.