Reliable data pipelines 101
Whether it’s for an executive dashboard or an ML model, reliable data is critical for the applications that make data-driven companies tick. But when it comes to creating reliable data pipelines, where do you start and what tools and processes do you need in place?
Whether it’s for an executive dashboard or an ML model, reliable data is critical for the applications that make data-driven companies tick. But when it comes to creating reliable data pipelines, where do you start and what tools and processes do you need in place?
Egor Gryaznov is the co-founder and CTO of Bigeye and was one of the first data engineers at Uber. Egor will draw from his experience supporting thousands of internal users and mission-critical workloads at Uber to provide an actionable guide to data pipeline reliability.
In this presentation, you will learn:
- How to approach building data pipelines for a data application
- What tools you will encounter in development and what you need to know about each
- How to create SLAs to better align with stakeholders
- How to manage the data that your application creates
