Opening Presentation: Harnessing Big Data: Constructing Data Pipelines
Data pipelines are intrinsic to most data operations, and are involved in many of the toughest challenges that so far have prevented enterprises from getting value out of their big data (and now AI) investments. It’s a rich field with many participants, including old standbys using tried-and-proven tech as well as emerging competitors using newer technology and techniques.
The requirements for building and maintaining data pipelines (and the rewards from doing it well) are well-defined enough that it warrants receiving more attention than it has. This session will explore the challenges surrounding the construction and operation of data pipelines, a method that many enterprises are relying on to move data from its source to where it can be analyzed and used for ML modeling.