Justin Berka, Nearside

Read on for a lightly edited version of the transcript.

Kyle: What's up, everyone? Welcome back to The Observatory. I'm Kyle. Today, I'm chatting with Justin Berka about building and managing analytics teams. Justin moved into data engineering from a background in econometrics.

He went on to lead the analytics teams at companies you've heard of like Uber and Intercom, as well as doing some angel investing in a number of data tooling companies. He now leads data at Nearside which is a banking platform designed for small businesses. Justin, welcome to The Observatory.

Justin: Thank you very much. Thanks for having me.

Kyle: Let's just dive right into some fun questions here. We're getting advice today on building and managing analytics teams. Maybe you could start with some advice. What have you learned over the last decade of building the data teams that a newer data team leader might like to know?

Justin: When you say “a decade” like that I feel really old.  But hopefully, I've picked up some useful things along the way.

For me, I think there are a few categories of things. One of the big ones for me is aligning the data team with the business and the business impact, specifically. I joke with people on my team that if you had a Justin Bingo card, all of the squares are basically “business impact.”

A lot of times you see data teams that are siloed in different parts of the organization, or structured differently. I think it's really important to align the work that the data team is doing with that business impact.

And that's not to say necessarily that you should put an exact revenue number to everything, because data teams work in such a matrixed fashion.  But it's important to be aligned with the top level goals of the company. And how can the data team help with that, rather than just falling into the pattern where data teams end up doing a lot of very rote work that has very different levels of ROI?  So business impact, certainly one of them that I think is important.

I think the other one that I didn’t used to think about but find much more important now: picking a few things and doing them well. Particularly when you're a data team at even a 100, 200, or 300+ person and up company, especially nowadays. There's a ton of interest in data, which I love. I love all of the interest in data, right? That's what keeps me employed. That's what keeps you indirectly employed. It's great. But I think it's easy to get overwhelmed.  

And again, all of the data people that I've worked with are genuinely nice people that want to make their coworkers happy, right? And I think we all feel that way. But I think it's too easy to get into that mode of doing work of different values and different ROI because you want to give everyone a little data time, and you want to be nice and help everyone out.

Unfortunately data teams, like many other teams and organizations, have to really prioritize ruthlessly and focus on making sure they’re aligned with the company goals, and driving impact in the ways that they can. Let's make sure we're focusing on a concise set of things, so that we're not spread too thin.

Kyle: Is there a way that you would approach that? When you start in a new role, do you have a playbook for how to figure out what the priorities are? You mentioned wanting to deliver data to five or six different teams. Do you have a go-to method for figuring out who those people might be and coming up with a framework for how to divide your time amongst those stakeholders?

Justin: It's a great question, and a great interview question I get sometimes. I think prioritization is one of those questions like, “how do you make a sandwich?” Which is to say, there are a lot of answers to it. My best answer at this point in my career is spend a lot of time talking to people, especially all of the different leaders and functional heads when you're joining a role.

Understand that there are usually two lists. There's a list of what that particular leader or team wants to get from the data team. And then the other list: what are your team's top priorities right now or for this quarter or for this month?

Often, what I find is, those two lists aren't fully aligned. So one of the really important roles of a data team is to be translators, and take that list of what’s important for your team and actually translate that into what are the actions that the data team is going to take?

I really try to get that first list of what's important to you this quarter, what's important to you this year, rather than what data do you want to look at. And I often say to people, “I will do that translation, I'm happy to do it, it's part of my job, or it's part of my team's job.” And so I think that helps to do that business alignment, because you know these are the things this team is going for, rather than just the data they're looking for.

Kyle: You mentioned, what's important this quarter, what's important this year. That reminded me of a framework that I was given by a product management mentor of mine a few years ago, which was to think about things in terms of hills and mountains. The idea was, what are the hills that you have ahead of you? So that would be things that you need to accomplish on a weekly or monthly basis. As a data team, that might be: hey, we need to do a check in on our data model, is the data model okay right now? That needs to be a recurring conversation versus the mountain-type goals. That would be a much longer term thing that you have to push towards. So that’s a really interesting framework that you propose there. A new leader could use that framework and communicate back to leadership and say, “hey, these are the things that sounds like they matter on a quarterly basis. These are things that sound like they matter on an annual basis or like even longer-term than that. Is that right?”

Justin: Absolutely. And not only when you join a role, right? Like a lot of other teams, data teams should be doing their own quarterly roadmapping and their own quarterly plan. Okay, we're going to take all of this inbound, we're going to take the top level business initiatives, and then we're going to take the things that we need to be working on that are coming from the team, whether that's data model data, quality data, pipelines, observability, you name it.

And we're going to synthesize all of that into something that we can socialize and communicate with other teams to make sure everybody understands what the data team is working on this quarter. That really helps with that second piece of focusing on a few things.The end result of that is telling people no. One of my discoveries is on the data side, it's much easier to tell people no if they know what the organization is getting instead, right? If you just tell them no, those are hard conversations. They're much easier if you say, well, we're not going to work on this for your particular team. But we're doing this shiny, cool thing for another team. Always a much easier conversation.

Kyle: Got it. That's super interesting. Maybe we can change gears a little bit. We've talked a little bit about building the team or what it's like coming into a new team and having those conversations with them. So specifically, when you're going out, you're hiring data engineers, or you're hiring data scientists, or any other role that you would staff on a data team - what are you looking for?

Justin: I think a lot of people talk about teams as individuals and data professionals on their teams as being T-shaped. And I certainly subscribe to that. But I also value having people who have a broad range of experience across the data space, anything from data engineering, analytics, data science, visualization, machine learning, AI, that can be the other end of the spectrum from data pipelines. I think it's really important to have people who can work across some chunk of that spectrum. There area very few people who can work across all of that from neural nets to deep data pipeline work. But having some cross-section of that, and then ideally people who have some depth in at least one of those areas. Maybe it's business, data science, maybe it's data engineering or ML modeling. And looking for the right combination of those different Ts, you end up getting good depth across different parts of the spectrum.

Kyle: So does that mean that as you hire the team, you're keeping track in your mind of where you’re filling in the parts of the T around your team? Are you planning your next hire based on the composition of those current specializations that you've already put together?

Justin: It depends. I would say sometimes, yes. I think it also depends. Sometimes you're coming into a team already. And there's a small number of incremental hires that you're making. Sometimes you need generalists, or sometimes you have a specific need in mind for those people. So it really honestly depends on the situation and where the existing team falls along that T.

Kyle: And maybe it's sort of related to that, when you are thinking about that T or when you're when you're in the earlier stages of looking for someone to join the team… how early do you feel you're able to identify that? Do you have any specific techniques for looking for the deep part of this T-shaped person that you put into your hiring process?

Justin: I would love to be able to say that I can identify it early on. The reality is that we all know now that hiring processes are very random in many ways, and often biased in some ways. For me, it's very broad bucketing, like this person has deep experience in five or six buckets, like data engineering, ML, analytics, or data science. Tho se broad buckets are visible, from pretty early conversations, because the buckets are so broad, but it's not generally deeper than that? It's not usually, okay, I'm looking for depth in neural net ML models, right? Unless you have a very specific need, or you're in a very specific environment where you need those roles.

Kyle: I don't usually go to that second level of depth, it's more about which bucket. And just for context, a bit more than the audience about your background, but let’s add some context. Are we talking about a specialized data team within a 10,000 person company? What types of teams and what types of company sizes are you talking about when you answer that?

Justin: So a lot of my career has been spent building fairly broad-scope data teams at smaller organizations, usually 75 to a few hundred  people. In those roles, the types of data teams that I look to build are very broad scope covering analytics, data science, and ideally data engineering as well. Because of the size of the companies, there isn't a lot of that specialization. That's why I think that breadth is important, because data teams get all kinds of really interesting questions. And it's important to have good data professionals who have good breadth who can answer questions, maybe even if it's a little outside of their particular deep expertise or their comfort zone.

You’ll notice I didn't mention really specific backgrounds or specific skill sets. My background is economics and public policy. I sort of fell into data science and analytics accidentally, which I think is a pretty common story for a lot of people. And I've worked with really incredible data professionals from all kinds of backgrounds and with all kinds of skill sets. So for me, it's really about are people quantitative? And can they break down a problem thoughtfully? Rather than can they break down that problem thoughtfully in Java, or Ruby or Python, or something specific like that.

There's certainly a skills component. Does this person have the right skills, whether it’s across SQL, Python, Excel, or another great data analysis tool? Does this person have the skills to interact with the data and actually do the things they need to with it?  Beyond that, I don't have very strong preferences on backgrounds or skill sets. And I would encourage everyone to cast a broad net in their data hiring practices.

Kyle: And I talked about this before with other folks that have been on the show as well. I'm on the side of building tools for data teams, and even with shaping the vendor landscape, data is fairly unique because it's doing a lot of software engineering work that has to actually happen. But it's being done by people from physics backgrounds, statistics backgrounds, economics, etc. And that seems like it's a fundamentally different thing. I mean, not that those folks don't also exist in sort of more traditional software engineering roles. But that seems to be a defining characteristic of data teams.

Justin: A lot of the time. Absolutely. And I think it's frankly great.

Kyle: So let's say you're one of these folks that's looking to join a data team, what do you think they should be looking out for?

Justin: One of the things that's really fascinating to me is that data science has been a job title for years, probably for a decade at least. There's still not a lot of specificity around what that is . Now, data science is a very broad job title. It can include things like building visualizations, experimentation, building actual ML models, and symbol analysis in SQL or things like Excel. So it's a very broad job title.

One of the things I think it's really important is understanding what data science or analytics or data engineering mean to particular companies, and what the scope of the role is, and making sure that that scope aligns with what you're looking to do. I think sometimes people say, “I want to be a data scientist, I want to spend 90% of my time building ML models.” Great, that is absolutely a thing many data scientists do. But it’s less likely to be true at small companies where there isn't as much of a focus on ML modeling. So just making sure there's a good fit in terms of what you as a prospective data professional are looking to do, and what the company is asking of its data team.

The other one for me, going back to the business impact Bingo card, is making sure that the team is structured and leveled in a way that it can have that business impact. This is a rant that I have given to several people. And I'll give it to you as well.

I'm constantly really surprised at larger organizations hiring data teams that are siloed or under-leveled. I think that's really detrimental to data teams having a business impact. So my advice for people going into new data roles is, make sure that data is placed in a way that it can align with the organization.

Oftentimes you see data siloed in product ops, finance, really pretty much every team right? Sometimes you'll see a data team silo. And I think it's really important to understand what the through lines look like to higher level leaders or to other teams in the business in that situation. Ideally, data teams should be a top level team, which I'm happy to rant about more. Make sure that the team isn't so siloed that it's difficult to have that impact.

The other thing is to make sure that your own personal level and the level of the data person you’re working with are aligned with the size of the company.  I think you see a lot, 500 person companies with a VP or senior-VP level leaders for sales, marketing, engineering, product design, operations, etc.

And then they say, “Okay, data is really important to us, we're going to hire a senior data scientist.” I just find that very interesting. Because ultimately, what has happened in every instance where I've seen that is the senior data scientist a few years into their career is peers with 15 years of industry experience. So there's a leveling mismatch in those situations. And I think that can one make it hard to really get things done and have that business impact.

It feels like there's that mismatch there. And I think that can also mean that companies don't want to invest as much in their data teams as they should, because they say “we're not ready for a bigger team”  or “we're not ready to bring someone in at a higher level within the organization because of the cost or the complexity that that entails.”

And to me, that's not necessarily a red flag. But it’s an area where people should really ask questions to understand how the data team works with the rest of the organization, and what the vision for the data team is. Or if you don't, if you're the person that's being asked to provide that vision, making sure that you can execute on this.

Kyle: So if I could repeat a few of the things that I heard. One is understanding, especially as an individual practitioner, if you're coming in and you're saying “hey, I really want to do a bunch of analytics modeling,” making sure that that's actually something that the team is there to do and that that's on their roadmap. If I'm coming in to do some ML work, make sure that that is actually on the docket. Don't assume.  

The second one that I heard was, where is the data team within the organization? You mentioned through lines up to leadership. So does the data team eventually roll into the CTO? Does the data team eventually roll into the CFO? I think that sounds like there's a question there about where do we live within the org? Is that changing? And understanding sort of like where that placement is.

Justin: That also goes to alignment for the role, right? If you're really interested in product, data science, and the data or it rolls up to the CFO, oftentimes, there's a misalignment there.

Kyle: And then the third one you mentioned sounded like it had to do with funding. Which is like, where is the data team coming in, in terms of the company's lifespan? Is that appropriate, if the company has gotten quite far, and they're just now investing in data? The data team might be doing a lot of backlog work, or they might be having to implement cultural change within the company, because the company may not have had data at all up until this point. So understanding where the data team is, in terms of company maturity.

Justin: Absolutely. I think another way to phrase it might be, how is the company setting the data team up for success? Are they making sure it's the right size, the right composition, the right level of leadership, etc.?

Kyle: Okay. And then that's probably a good segue into one of the last questions that I had for you today. If you were to talk to founders, or anybody who's in that executive position, where they are deciding whether or not to invest in a data team, what would you tell them about how to identify when is the right time and how to go about it?

Justin: I love this question. My is always you should start a data team yesterday. And it should be a top level team reporting into one of the leaders of the organization, could be a CEO, could be a CFO, could be a COO. I think that does depend a little bit on the organization.

But to me, the ideal data team is one that is started early, is a high-level team that reports to a C-level executive, and also has representation at the staff level on an equal footing with other functional heads. So I think about it really as a separate function. And I think it's really important to start a data team or have a data professional, who either has some leadership experience or can grow into that early on.

Because I think there's just so much that having data people at your company can do early on, right? Whether it's establishing metrics and what the company should be measuring and thinking about, and computing those metrics. A big one for me is working with engineers on instrumentation and data collection, in an engineering-adjacent function to make sure as the company grows, that you have the right foundations for doing later analysis.

And then obviously, depending on how much advanced analytics work, or ML work, is based on the product, it is also really important to have people with that skillset early on. But I think it's really important to treat it as a separate function so that there's enough time and enough resourcing for it.

A lot of times you see data pipelines or data systems built by other quantitative people: engineers, finance people, marketers who are quantitative and do a great job, but are juggling that with some of their other roles. And frankly, I think it's really hard to juggle data roles and even sub-specializations of data: data science, data engineering, ML. It's hard to juggle multiples of those things, even in a small company.

Kyle: So if we had to draw a line in the sand. If you’re at 100 folks that are in software, engineering, finance, marketing, sales, legal, but you don't have a data team yet, you should probably be asking questions. Is that a fair statement?

Justin: Yes, I think absolutely. I think that is a little late to the game. It's not as if it's 200, then it's very late to the game. But I think earlier is better so that the data team can grow with the company. I think about data as any other function. If you talk to a 300-person company, and they said, “Well, we haven't hired a sales team yet.” People would have a lot of questions, and rightly so. But that's very common. I think it's changing, which is great. But I think that's still a very common thing where you have big companies who truly feel the data is important that wait to hire data teams, and it's really hard to play catch up in those situations.

Kyle: All right, so I want to turn it over to our rapidfire questions to wrap up, but that was a ton of valuable information for anybody who's getting started on a data team, or thinking about making one. But maybe we can do some fun stuff, too. So let's do it.

Question number one: favorite statistical technique?

Justin: So many good ones to pick from. I think I'm going to go with the good old T-test because it's easy, and there's a lot of value. It's available everywhere. And in a lot of formats.

My favorite thing about the T-test is it actually imposes organizational process around experimentation by adding some context around sample sizing, and so I love that you get not only a statistical test, but also organizational process pointers.

Kyle: All right. The good old T-test is a classic. Second question: iPhone or Android?

Justin: I personally am an Android user. Essentially every other device I have used, whether a computer tablet, mobile phone, computer or tablet, for the past 15 years has been a Mac right? I'm a long, long-time Mac user, even going back to before Apple was cool. And so I think it's great to have some competition in this space and great to have options for mobile devices and not just a single player.

Kyle: Okay, third, one last rapidfire question. “Analyst” as a job title, underrated or not?

Justin: I love this question. I spend a lot of time talking to people about this one. So I'll try to give the mostly rapidfire answer.

I'm a really big fan of data science job title to describe almost all work that quantitative professionals do. I think, frankly, you've seen a lot of bigger companies move towards using that job title as well. The reason that I really like it is because I think even in broad data roles, people are often asked to do data science work.

Going back to the first rapidfire, that could be things like T-tests, or Bayesian statistics. And I truly do consider those data science work even if they're not necessarily building ML models, right? Datasets for feature extraction for model: that's data science work, even if you're not necessarily building the model.

And so I think there's just so much. I think sometimes, data teams don't give themselves enough credit for the quantitative nature of the work that they're doing. And so I'm really a big fan of the data science title.

I also think it's really important for organizations to align on one or the other. Because I think in organizations where there's a mix of those job titles, it in some ways imposes some value judgment on the different types of work, which I don't know that that's intentional. But I think sometimes it can feel that way.

And often, even for things like recruiting for internal advancement and promotions, just brings up a lot of second order questions. So if you're talking to someone who's a data analyst, they say, “Well, I really want that data science job title. I think that reflects the work I do. Why am I in this other job title?”  And frankly, like, I don't think a lot of companies have great answers to that. So I'm in data science, by the way.

Kyle: All right. Well, Justin, thanks so much for being on The Observatory today and answering all these great questions, especially for folks that are either starting data analytics teams, or maybe they're managing or growing them right now.

Justin joins us from Nearside today. Nearside is a company that's building the world's best small business banking platform.

If you want to learn more about it, you can check out a link in the description below. Justin, thanks for being on The Observatory. This was a ton of fun.

Justin: Thank you very much for having me.

Kyle: All right. Thanks, everyone. See you next time!