Thought leadership

September 26, 2024

From Data Analyst To Founder | Bigeye CEO Kyle Kirwan on Leadership in Tech

Bigeye CEO Kyle Kirwan speaks on his journey from Uber's data team to build a leading data observability platform

Kyle Kirwan

In this podcast episode, Kyle Kirwan, co-founder of Bigeye, walks Ben Book of GigaOm through his journey from a data analyst at Uber to launching a cutting-edge data observability company. With a background in industrial engineering, Kyle’s unconventional path gave him a fresh perspective on data science and product management, which has shaped how he navigates leadership in the tech space.

Kyle shares stories from his time at Uber, where he worked on scaling a global data platform. The lessons learned—managing data quality, lineage, and observability in a fast-growing enterprise—formed the foundation of Bigeye’s mission today.

Key takeaways:

The power of data in driving strategic business decisions and innovation at scale.
How to build trust in your data and make it central to high-impact areas of the business.
Tackling scaling challenges by focusing on operational efficiency and data reliability.
Why enterprises need data observability, particularly when balancing legacy systems with cloud technologies.

By the end, you’ll walk away with actionable insights into how data-driven leadership and smart decision-making can help enterprises scale their data systems and improve business outcomes.

Watch the full interview here:

share this episode

Resource

Monthly cost ($)

Number of resources

Time (months)

Total cost ($)

Software/Data engineer

$15,000

$540,000

Data analyst

$12,000

$144,000

Business analyst

$10,000

$30,000

Data/product manager

$20,000

$240,000

Total cost

$954,000

Role

Goals

Common needs

Data engineers

Overall data flow. Data is fresh and operating at full volume. Jobs are always running, so data outages don't impact downstream systems.

Freshness + volume
Monitoring
Schema change detection
Lineage monitoring

Data scientists

Specific datasets in great detail. Looking for outliers, duplication, and other—sometimes subtle—issues that could affect their analysis or machine learning models.

Freshness monitoringCompleteness monitoringDuplicate detectionOutlier detectionDistribution shift detectionDimensional slicing and dicing

Analytics engineers

Rapidly testing the changes they’re making within the data model. Move fast and not break things—without spending hours writing tons of pipeline tests.

Lineage monitoringETL blue/green testing

Business intelligence analysts

The business impact of data. Understand where they should spend their time digging in, and when they have a red herring caused by a data pipeline problem.

Integration with analytics toolsAnomaly detectionCustom business metricsDimensional slicing and dicing

Other stakeholders

Data reliability. Customers and stakeholders don’t want data issues to bog them down, delay deadlines, or provide inaccurate information.

Integration with analytics toolsReporting and insights

From Data Analyst To Founder | Bigeye CEO Kyle Kirwan on Leadership in Tech

The Four Pillars of Data Observability