Reliable Pipelines and High Quality Data without the Toil! (ODSC Data Engineering 2023)

How can data engineers prevent issues with pipelines from reaching data consumers like data science teams, and end-users like executives or customers—without adding a ton of extra work?

In this talk, Kyle Kirwan, CEO and cofounder of Bigeye, and former lead for Uber's data operations tools team, explains the core concepts behind data observability, and how they help identify pipeline and quality problems without adding a ton of overhead for data engineers.