#114 Sanity over vanity – focusing on metrics that matter (with Ole Bossdorf @ Project A)

[00:00:00] Dan: Welcome back to The Measure Pod. Happy new year, everyone. We’ve had a couple of weeks off. I didn’t really talk about it actually. Anyway, the point is that we’re back. We’re back for 2025, new episodes every fortnight. This is episode 114 and we’re back. And we had a great conversation with Ole, right?

[00:00:32] Dan: And this is someone that reached out to us directly on LinkedIn. More about that towards the end of the episode. If you’re. Interested in joining us because we require quite a substantial pitch deck, I think is what we got to, in that conversation. But it was really interesting speaking to someone that works in analytics at a VC with over 120 portfolio companies.

[00:00:48] Dan: So, I mean, the level of experience I thought I was working across lots of different projects at an agency and then, and then there’s that, right. It’s quite a different perspective on things. 

[00:00:57] Bhav: Yeah, it was really interesting because I think you and I, whilst we’re in a, probably a parallel world to others in the sense that we work with multiple clients across multiple verticals and industries, most of our clients are probably, they probably have some level of establishment in their industry, or, you know, the, the, the product or service that they’re selling and where all is coming from is coming from a background where actually.

[00:01:19] Bhav: He’s working with very early stage companies who haven’t really yet established themselves. They haven’t probably established a footprint. So it was really interesting to hear how his world was being similar to ours is also very different to us. You know, I think you and I probably work with an abundance of data and we talked about this on the episode, but I’ve always been fascinated by not having that much data and actually being creative when you, but when you need to do a piece of analysis or some exploratory research.

[00:01:48] Bhav: How do you do that when you don’t have that much? How do you size the market? How do you make forecasts? How do you make predictions? And if you start to realize that you need to cross the, you know, the boundaries of science and art and like blur them together so that you kind of have to take some, some level of like gut instinct approach.

[00:02:03] Bhav: So I don’t know. I, it was really interesting for me. I’ve, I’ve really enjoyed recording this episode. I’m grateful that Ola took the time to reach out and he actually reached out a while ago. It’s just our bad for how long it takes to kind of get these things off the ground. 

[00:02:16] Dan: Hey, we’re not professional podcasters. This is just a side hustle for us, right? someone should start paying us and then maybe we’ll, we’ll be on top of it more, right? There you go. 

[00:02:24] Bhav: How do people monetise these things? How do we make money off of this? I don’t, I have no idea.

[00:02:29] Dan: I tell you what, if someone works in analytics and podcasting, Let’s jump and have a conversation, right? okay. So, let’s, let’s let everyone jump into the episode before we do that. we’ve had a couple of weeks off. We’re starting 2025. anything to promote anything you’re planning for anything that’s coming up in the world of bath in the next couple of weeks. 

[00:02:47] Bhav: So I think the big thing that’s on the top of my list is getting crap talks back off the ground. If depending on when you’re listening to this, when this episode goes out, you’ve either just missed. Crap talks 27 in London, or you’re just about to go into crap or 27. And we’ve got into the diary, all five events this year. We’re going to be, I’m going to be certainly be really, really on it and trying to get these events off the ground.

[00:03:11] Bhav: Last year, we kind of got to July and then. We just, I, I personally just ran out of time and capacity to be able to like really see it through. So this year I have a mission to get five events done. So that’s my thing. Look out for crap talks is 2025 it we’re going to do five events this year in London. And then Dan, I’m sure you’re going to do some in Brighton now. 

[00:03:27] Dan: Yeah, absolutely. I mean, we had our inaugural, crap talks, Brighton last year, the back end of last year, and it was a success. Everyone seemed to like it and we got some good reviews. So. well, we’re going to keep going this year and I’m just working on that.

[00:03:39] Dan: the one thing I’m going to mention, hopefully, this episode is going to come out beforehand, but, I will be at super week, in the end of January. So if any of our listeners happen to be at super week in Hungary at the end of January, come and say hello to me. I plan to do some podcasting there and chat to people and, and yeah, and just meet a lot of our previous guests as well.

[00:03:57] Dan: I mean, it seems to be a bit of a Mecca of, people meeting up and hanging out. So I’m really looking forward to that and be sure to grab us and grab a beer or something like that. So, let’s jump straight in, without further ado, this is our conversation with Oli.

[00:04:13] Bhav: Hi, welcome to the show. It’s really great to have you. Thanks for reaching out on LinkedIn. I love it when people just message Dan and I and just say, Hey, we’d love to be a guest on your episode. And then we screen you secretly in the background to see if we think you’ll have something interesting to say.

[00:04:26] Bhav: And I, when I was stalking your LinkedIn profile, I loved what I read sort of like about your background in VCs. And I thought, Oh, this could be a really interesting topic. And Dan and I have never really discussed this. So welcome to the show. Before we kick off, we’d love to have Project a, your background, your origin story as Dan calls them.

[00:04:44] Bhav: So, you know, please feel free to go in as much or as little depth as you want about your background. 

[00:04:50] Ole: All right. Thanks. Thanks for kicking us off. And I’m very happy that I passed the screening test, so to say. So now my name is Ola. I work at, as you said, a company called Project A, which is first and foremost, an operational VC, a venture capital firm with over a billion management that we invest in promising early stage companies.

[00:05:13] Ole: But from the get go, when the business got started, we had sort of the strong hypothesis around also employing an operational team that actually helps The companies that we decided to invest in with the different operational challenges that they’re facing. And that’s quite a neat position to be in, to be honest, because you do have transparency on what truly matters for the company, because you do also sit on the board and own a share in the business.

[00:05:40] Ole: And then you basically figure out how to support on the different investment milestones or fundraising objectives and so on. And I’m responsible for the data domain within that operational team that I just described. So it’s a team of around 18 people, data analysts, data engineers, data scientists.

[00:06:01] Ole: And together we try to define The role data plays within a business that we invested. Sometimes we also do this now outside our portfolio. And, we basically try to define the role that data is supposed to play. And then we hopefully put that into action. So we define the different topics that truly matter, metrics that matter, predictions, automations, reports, and so on.

[00:06:25] Ole: And then we built sort of the necessary. wiring pipelines and so on, to make that happen and move the business forward. And pretty much early on, I decided that data is for me. and I recently found a notebook when I visited my grandparents. And apparently when I went on camping trips with them, I would count the number of cars passing us and break that down by color and also run stats around how many colors basically passed us on average per day, per trip.

[00:07:01] Ole: And that’s when I was like eight or nine running sort of stats on paper. she should have given me that book a lot earlier because then I would have sort of not, ventured into marketing first and other directions, but I knew right from the get go data is where it’s at for me. 

[00:07:18] Bhav: I love that. That’s a great story. And what were the results? What’s the most common color car? 

[00:07:25] Ole: To be honest, I did manipulate it a bit as any good data analysts would do. And she taught, she told me actually that sometimes blue or red would win. And then I would shift some cars towards to white or silver because they haven’t won in a while on a specific day or camping trip. 

[00:07:41] Dan: So, yeah, only trust analysis. You faked yourself. I guess you’ll be in an analyst and it’s inevitable. You would have been an analyst, right? Yeah. I have a, I have a slightly leading question. So I, I’ve, I’ve tentatively titled this episode, sanity over vanity, focusing on metrics that matter. So at Project A, it sounds like you’ve worked with lots of different portfolio companies, lots of different types of businesses, lots of different people within those businesses, different products and everything else.

[00:08:05] Dan: So how many, how many times have you ever, gone into a business and found that they were measuring what matters and then you didn’t need to do anything at all? 

[00:08:13] Ole: I think quite rarely, of course, but sometimes I find with founders that, they just have sort of this intricate feeling for data and know exactly what to track and what to not necessarily care about too much.

[00:08:27] Ole: And so I’d say 5 percent of the time, maybe if you tried to name it down on a specific number where everything was sort of in place. And they were also not too far in how much they ventured into sort of data warehousing and analysis, because sometimes you can actually overdo it as you two probably have seen before.

[00:08:45] Ole: And they were also not under investing in the domain. So I guess that’s also tricky to do for the maturity of your business, business, having sort of the right data set up for that specific time and not overall under investing in it. And rarely you really nail the timing, but sometimes. Sometimes they did, I was quite impressed, still found ways, I guess, to optimize further, but it does happen.

[00:09:12] Bhav: Like, Dan, I’m, I’m fascinated by the fact that actually you’ve probably been there. Done that, seen it all from companies, you know, like at the early stages of, of, of, of, of this kind of journey that the, the startups go on, when is the earliest that you’ve been involved with, with data? Do you wait for it until like companies reached a certain size, a certain number of users, a certain threshold before you kind of think, actually, you know what now?

[00:09:34] Bhav: Cause like one of the questions I get asked a lot, and I’ve, I’ve had this discussion so many times has been really around. When do you hire a data team? When do you bring in data professionals into an organization? Do you bring in research first? You do quant first. Do you go gut instinct and product management first? Like when’s the earliest you’ve been involved? 

[00:09:53] Ole: So since we also do incubation, so really, truly identifying promising business ideas and then finding teams and scaling those up, it’s super early. It’s like a slide deck early, basically. A lot of times there’s not a lot of operational things for us to do, but it is more of a mantra that you try to.

[00:10:13] Ole: It’s sort of basically establish early on in the business, collect data properly. So at some point you want to make decisions based on that. For now, your gut is fine, but later you need to move to some sort of a fact based decision making. so, and then outlining early on what kind of needs to be, needs to be tracked.

[00:10:34] Ole: But a lot of times. I actually gained credibility with companies that were supporting by holding them back and investing in data too early, simply because they’re all excited about the AI or back then it was data science and machine learning trends and want to get something going, but they don’t have the volume for it.

[00:10:53] Ole: They don’t necessarily have the right mindset of people for it. And then I tried to hold them back on these investments and say, look, the tooling that you have. today has some pretty cool built in reporting. Let’s run with this until you don’t get the insights from it anymore that you need, but your Hubs HubSpot reporting suite or your Magento reporting suite is actually quite powerful.

[00:11:18] Ole: Once we get to insights that require joining. data from different sources and running more complex modeling on top of it. Let’s talk again, but for now, I don’t see a need here to invest into data. So that’s been my observation, really, basically brainwashed by generally emerging trends. People tend to invest a bit too early into something that that needs to be maintained and needs to be overhauled because the business model changes early on very drastically. So I always try to hold them back a bit longer until sort of the right timing. 

[00:11:52] Dan: So, I mean, I mean, what’s the magic point at which something like, let’s take, let’s, let’s take something, like, like creating and centralizing your data in the data warehouse. So they become warehouse native in terms of reporting and analysis.

[00:12:05] Dan: This is something that everyone feels they should be doing. And it’s always that point. That is changing quite a lot of stuff. If everyone’s used to having reports out of Magento or HubSpot or Salesforce, and we’re saying, actually, let’s, let’s put something in the middle of that and actually centralize it over here.

[00:12:20] Dan: Is there, is it about the people in the team? Is it about the maturity of the business or the size in terms of revenue? Like it could be a combination of those, but what is that deciding factor where you would say to someone, Hey, now is the time to do this rather than actually let’s, let’s take a step back.

[00:12:35] Ole: So I already mentioned a bit, like. Insights that are truly crucial. For example, how do we spend our marketing relies on joining data together from multiple sources. That’s a good signal for us. It can also be the business would. Basically benefit tremendously from having an access to some external data source.

[00:12:58] Ole: And that’s maybe also the only exception where you want to invest in data a lot earlier, when it’s not about internal data, but actually you plug into some external data source. That is not necessarily available to everyone, but by you sourcing data from there and basically creating insights for your customers, that could be the only exception where it makes sense to invest earlier.

[00:13:20] Ole: But let’s run with sort of the mainstream, which is that companies should invest a bit later, but then properly. And when should they invest? One thing that causes a lot of frustration and costs companies tremendous amounts of time is discussions around the right value for a given metric. So that’s a pretty strong signal.

[00:13:42] Ole: You run through your business, marketing reports, that number of customers, sales reports, that number of customers. And in your own application database, you come up with an entirely different number. You are in trouble. It tends to happen at some point and it’s fine to a certain extent, but at some point you want to align.

[00:14:00] Ole: the company on metric definitions that sort of work for everyone and then always spit out the same value so that people don’t lose time in meetings, talking about which report they can trust and which one they can’t. So here is sort of a very strong signal. Do metric values align or not across different teams? Does that make sense? 

[00:14:24] Bhav: Yeah, I think so. And I think what’s really interesting actually is in, in a world where we’re living, where there’s an abundance of data, it’s actually kind of refreshing to hear about working in a space where there is an absence of data. And, and, and I’m going to lean to the conversation towards data strategy on this, that wants to pivot back to what we were just talking about, but you talked about like defining metrics.

[00:14:44] Bhav: And, and I think a lot of this ties into building a data strategy. And I had a conversation very recently with a, a, He’s a head of analytics for a, for a health company. And he was just asking me, he was like, Hey, Bob, how do I build a data strategy? Can you, can you like guide me? And I kind of gave him my views on it, but I realized that actually, there is this, gap in understanding of like, what is a data strategy and how do you build one?

[00:15:08] Bhav: So I’d love to hear, cause you’ve probably done it a countless times over what, what is, what encompasses a data strategy? And of course, there’s going to be an element of like metric definition, but if someone wants to ask you like, Hey, what is a data strategy? How’d you build one? How would you respond to something like that?

[00:15:24] Ole: Yeah. Yeah. Happy to get to that very small topic, which, which is data strategy. No, of course we’ve done some, or we gathered some experience there over the years and maybe two points where you don’t start is What is the data that I have available and from that, from their deriving data strategy, that’s pretty much a horrible thought because you will miss out on a lot of interesting data topics that you might pursue, but you don’t have the data for it.

[00:15:56] Ole: So you don’t even think about them. So let’s not start from the data, not a great idea. Let’s also not start from the tooling. Like, Oh, this is a snowflake. it seems like a powerful database. So we should probably structure our data strategy around a specific type of tooling that we want to procure and use.

[00:16:15] Ole: That’s neither a good start. It very much starts. And it’s super easy for me to say, because we do have transparency there, but it very much starts from the direction that the business is moving towards. A lot of times, we are lucky to own a share. Right. in the company that we are supporting and thus we sit on the board, thus we know exactly what needs to happen for the next investment round or, for specific financial milestones that need to be reached.

[00:16:44] Ole: And then comes the very tough part. That sounds very easy, which is deriving from the overall company direction, the role that data plays in order to basically. Yeah. Support, that direction that the company is heading towards sounds very simple is incredibly hard to do, to be honest, and that’s what we train our analysts to do over years.

[00:17:06] Ole: what are the different initiatives that contribute to the overall company direction, and that could be. Maybe we, we dive into a specific example, to make it more tangible to the audience, but that could be anything from certain predictions. We need to make certain analysis or automations or reports that we simply need to have in place in order to reach.

[00:17:30] Ole: the revenue target for 2025 or whatever a certain number of customer users, activity on the platform and so on. 

[00:17:39] Bhav: And I think in this instance, you have to be somewhat, sorry, I was going to say in, in this instance, you have to be somewhat accepting of the fact that you’re not going to have tons and tons of data, right?

[00:17:48] Bhav: So it becomes balancing. It becomes a balancing act between. Art and science. And because again, the other thing I get asked a lot, and I had a call just literally yesterday about someone who wants to transition into analytics as a, as a field and as a discipline. And she comes from a B2B marketing background.

[00:18:05] Bhav: And she was like, Hey, Bob, I’m thinking about doing a course in R or Python, or what do you think I should do? And I said, why start with the technology? Why start with the tools, right? Let’s start off with the you have and, and then work from that point onward. So with, With this world where you are working, you know, the setting that you’re in, where you don’t have lots of data, how do you train the analysts to make forecasts or make predictions or give advice in terms of like, look, this is where we want to go.

[00:18:33] Bhav: We don’t have that much data. What, what do you look for? I think this is a lost art by the way. And I’m, I’m really, really pleased that we’re talking about this topic. 

[00:18:40] Ole: I think that that’s something definitely to disentangle their path because. Like you can already get working on a data strategy without having the necessary data in place.

[00:18:49] Ole: Actually formulating that more in detail might help you to understand where you’re currently still lacking data volume or consistency data quality. So let’s disentangle that. The data strategy doesn’t necessarily mean, okay, we now necessarily have to execute. It could also be sort of a guidance.

[00:19:08] Ole: Towards, how you need to set up different tools properly in order to even get to the topics that you outline within your different data strategy. How do we handle not having sufficient data available? I think you can sort of circumvent that to a certain extent through more qualitative measures, and then you have to involve the product or research discovery team, and you work a bit more on a qualitative aspect because you have a lower sample size.

[00:19:35] Ole: And this still yields interesting information. but a lot of times, it’s probably more of a product and actually a data analyst task when we, when we talk about more of the qualitative work, how else do we sort of come up with recommendations when there’s limited data volume, I guess it comes back to the point that then it’s a bit.

[00:19:56] Ole: Too early to invest into a proper data setup, meaning infrastructure and team when there’s not even sufficient volume available to come up with real meaningful insights. And then the business will just be frustrated with the returns it gets from the data team when they come back and say, sorry, thanks for hiring us.

[00:20:14] Ole: But unfortunately we don’t have the data available to get to the business questions you asked us. So you have to be careful there. It’s probably, a pretty meaningful check to do beforehand. 

[00:20:26] Dan: Does this, you said something earlier, which is around, like you have a seat at the table, essentially a project a, because you’re lucky enough to kind of be a shareholder or be on the board level.

[00:20:36] Dan: And this is like a privilege, I think, to have some kind of data representation, or at least people that are aware of data at the board level. And so I think a lot of people, a lot of businesses aren’t going to have that kind of representation are trying to fight, fight for that kind of representation. Is there anything, I mean, have you ever gone into working with a.

[00:20:54] Dan: with one of your portfolio, businesses whereby it’s been more of a challenge to actually get that kind of representation or trust in a data practice, or maybe they’ve been burnt previously in terms of some bad analysis that have been done. Like, has there been any of that? Or has it always been relatively similar or smooth sailing?

[00:21:11] Ole: No, definitely, definitely no smooth sailing. especially in the very dynamic VC environment that we’re acting in. generally, I I would say if you lack transparency on what moves the business forward and you cannot easily just ask for it, then at least conduct a thought experiment of basically going to your senior stakeholders and saying, okay, what This is apparently what you want me to do.

[00:21:39] Ole: Can you be so kind to outline for me what you will do with this information I’m about to deliver? Apparently you’d like more insights into the sales pipeline, into sales velocity, deal cycles, and so forth. Just to do my work better. Can you outline what are the specific decision levers you have in place that you will pull?

[00:22:01] Ole: based on these insights. And you do that in a very naive way so that they can actually, feel sort of obliged to, to help you and that they don’t think that you don’t trust sort of their ability to figure out which analysis are needed. So you do it in sort of a, a bit of a naive, just curious mindset way.

[00:22:22] Ole: But that can help to run through this thought experiment once and maybe sometimes stakeholders figure out, oops, I don’t, I wouldn’t actually do anything differently based on these insights. A lot of times that happens. because I cannot just say, If we look into, let’s run with the sales insights. If we find out that our deal cycles are very long, it takes us 180 days to close a client.

[00:22:47] Ole: I cannot just go back to the potential clients and say, can you please sign earlier? It’s not an insight that necessarily is super actionable. So if you don’t have the transparency, you need to do that in a more iterative way. And maybe start with this thought experiment. What would you do differently?

[00:23:05] Ole: What are the decision levers you have available that you would pull based on the insights I’m about to deliver? 

[00:23:13] Bhav: I like that you have the option to be able to do that. I think in more established organizations, even with the best of intent and a naive approach to this question, just to be able to kind of say, look, what are you going to do with this information?

[00:23:25] Bhav: you, you know, there’s a likelihood, there’s a very strong likelihood that you’ll just get pushed back and with a, just go and do it, like you don’t need to know, or you, it’s kind of nice. 

[00:23:36] Dan: Or you catch them out because they actually don’t know, and they don’t like. In that kind of cognitive dissonance, I find you’re always challenging.

[00:23:42] Dan: And you’re saying, what are you going to do differently? What are you going to do with this information? And it’s like, how, how dare you in a sense, like, it’s that kind of reaction. How dare you challenge me? 

[00:23:51] Bhav: I call most of them the okay. Thanks. Questions that I’ve, I think I’ve shared, okay. Thanks analogy with you before, but most stakeholders end up in the situation where they ask you a question, you know, you go away, you got to find the information, you bring it back and they’re like, okay, great.

[00:24:04] Bhav: Thanks. And that’s it. And then the conversation ends that, so I kind of liked this approach of like this very naive, like, Hey, like, what are you going to do with this data? Just, I want to know, like, and I imagine if it comes, I mean, if it comes from someone like me, they’re probably going to be like, stop being a dick, just, just tell us you don’t want to do it.

[00:24:22] Ole: Seriously. Why are you avoiding this? No, but I mean, if you don’t feel comfortable doing that, then at least follow up. I think it’s something Adam Greco talked in your show before as well, but at least do a follow up two months later, you put, you put your heart and soul into this specific analysis, then maybe naively follow up to six to eight weeks later saying, Hey, whatever happened, based on this, did something improve?

[00:24:47] Ole: in the best case, you hear that there’s something did improve, which helps you to better evaluate. The overall ROI or impact that you as a data team bring to the table, worst case, nothing happens, but then maybe they are a bit more open next time they have a request to you asking what they would actually do with us.

[00:25:07] Ole: So, yeah, it sounds very simple. It’s quite hard to pull off. 

[00:25:12] Bhav: I think I’ve just been in the game for so long that I’ve lowered my expectations even further than that. And now I’m just like, Hey, that analysis I sent you three weeks ago. Did you read it? 

[00:25:23] Dan: You need a red receipt. did you open the file?

[00:25:27] Bhav: Actually, not even read it. 

[00:25:28] Dan: Did you open the file? Jeez. Google docs is good for that. 

[00:25:33] Ole: So what we do is self destructing dashboards. So basically we ship a dashboard and then it just goes offline again. Sometimes 28 days, sometimes 14 days later. So, and then it’s the classic scream test. And then you see, have they actually looked at it?

[00:25:49] Ole: or do they scream, have they looked at it once and not ever again? and as a little insight, 80 percent of the dashboards that we do ship tend to be not necessarily within the first cycle, but within a couple of cycles, we tend to destruct them again. and that leaves the company with a big problem.

[00:26:08] Ole: Sort of reporting landscape that is still pretty easy to navigate. So I kind of liked that approach, but you need to make sure the business understands that a lot of the dashboards you’re building, end up. In the trash bin over time. 

[00:26:23] Bhav: I, I think that’s great. It’s self destructive dashboards. That’s, there’s a feature I’m going to recommend like amplitude and Tableau and, you know, hopefully one, someone will implement it.

[00:26:32] Bhav: so we’ve been talking a bit about strategy and I want to, I want to talk about metrics and, and, and, and one of the things that I always think about is when you’ve got, I imagine you’re, you’re in a situation where you’re very, you know, Much the early person in the data space, entering a lot of these organizations.

[00:26:47] Bhav: I, I find it hard to imagine that they probably got an established data team. In many cases, you’re probably going to be the first or in like supporting someone who’s already there, maybe doing a little bit, how do you prevent a startup or someone who, you know, a team who, who aren’t maybe possibly data savvy, they’re very good at what they do clearly.

[00:27:06] Bhav: They’ve made a startup, they’ve been able to get funding. Maybe they’ve even got a sales pipeline now, how do you get them to focus on the right metrics and how do you steer them away from vanity metrics? And we know that this can happen a lot, right? Where you’ve got product teams who are looking at maybe daily active users is say, for example, the right metric in some instances, but in many instances, it’s the worst metric or it means nothing.

[00:27:28] Bhav: So how do you find, and how do you get the team to look at the right metrics? Like what’s your approach for finding and identifying correct metrics? 

[00:27:36] Ole: Maybe starting off with the fact that the metrics that matter to a business will evolve over time. So it’s pretty much an endless pursuit. It’s not something that is ever really done because the business does evolve.

[00:27:49] Ole: The business model might change, might be adapted with the internationalization or with the focus from growth towards cash efficiency and profitability, which has happened with a lot of our portfolio companies over the last years. And then suddenly the metrics that matter change dramatically over time.

[00:28:07] Ole: So it is something for us as data practitioners to always have on our radar. Sorry, unfortunately, the job’s never really done. and it does, it does evolve. If you think about marketplaces that are two sided in the beginning, one side is more important and later on another side is more important. So that’s maybe something to point out very much early on, that the metrics that matter for a business will evolve.

[00:28:34] Ole: over time will move actually more from sort of a product usage standpoint. This is where, especially a lot of the early stage SAS businesses will focus on initially, and they eventually move more towards which, I mean, what you can see with public companies, then it’s very much focused on the, financial metrics and EBITDA and so on, but that’s less important for, for early stage businesses.

[00:28:58] Ole: So it moves from sort of. product usage and, and market demand and so on more towards the actual, lowest items on, on your P and L so to say. so that’s the evolution of the metrics that matter. How do we find them? So of course we look at the stage of the business. We look at what’s crucial again, what are the.

[00:29:21] Ole: Things that the business needs to accomplish. And then we try to quickly figure out what are metrics that we currently are looking at that actually, like if we would remove them from our daily or weekly business reviews, we would not be worse off and maybe let’s make it tangible. we were into, we were invested in a company in the real estate space that basically helped you to sell your homes.

[00:29:48] Ole: Very efficiently. And their initial hypothesis was. We just built incredibly cool digital products that help homeowners to sell their properties very efficiently, super transparently, and you can track the value of your home and how many people are interested and so on. And then initially, they’re very much curious.

[00:30:10] Ole: and focused on basically how are homeowners using the different products that they’re working on, the different apps and property value trackers and so forth. And eventually figured out that homeowners don’t care about this. They care about selling their property fast. And for the highest price possible, no matter if they can look at a fancy dashboard to see how many agents are working on this or not.

[00:30:36] Ole: And that took them a bit of a time. So the metrics that truly mattered were more related to the actual sale price and sales velocity. And there was a lot less about. Logins into the real estate value tracker. so that was an interesting and valuable process to go through. It basically shifted the entire monthly business review, towards more sales velocity and less towards just product usage. Maybe that’s an example that, that is a bit more tangible. 

[00:31:06] Bhav: That’s a, I mean, I think that’s a really good example because you’re right there. Sometimes I think product managers can get lost in what they’re building to the point where they’re blinded by the fact that actually it’s adding zero value to the overall value proposition of the organization.

[00:31:21] Bhav: I used to work for a company where we had a product manager who was looking at recording video recordings. The product manager’s role was number of people who played back video recordings after the event had finished. And. They were really, really obsessed with this feature. And even though my team and I were like, look, no, one’s really using it.

[00:31:39] Bhav: They were like, literally asking us, can we make the numbers look better? And of course I’m, I’m a very, I’m a very ethical person when it comes to data. I’m unethical in many ways, but when it comes to data, I’m very ethical. I was like, no, we’re not changing these numbers, right? No, one’s using this feature and we need to deprecate it.

[00:31:54] Bhav: But it’s, it’s interesting that, and I really liked the story because you’re right, like a product team, there’s probably someone there who’s gone and maybe done some, some, maybe not, they’ve not even done some research, but they’ve gone and built this price tracker or these fancy things. And actually they, they’ve stopped.

[00:32:09] Bhav: They haven’t stopped to think about the fact that homeowners just want to sell, right. They don’t really care about anything else. I just want to sell. I don’t care who buys it. Right. I don’t care. Just how many, like how quickly can I sell it and what’s the maximum I can get for it. So that’s interesting.

[00:32:25] Dan: Oh, I mean, this, this, this is really. What I’m finding actually is I think you’re in actually in a very interesting position compared to even me and Bhav who work agency side is that I was just looking at Project A’s website and you’ve got over 120 back companies which means that whether you’re directly involved or not there’s a lot of businesses you’ve worked with in this kind of context so Being in a unique position of working across a lot of different businesses and some similar kind of verticals within that, how hard is it for someone like yourself to go into, let’s say the next SAS company that, that joins the portfolio and not have assumed perspective of what the metrics that matter are?

[00:33:02] Dan: Because I can imagine you’re like, I’ve just, I’ve worked on 10 SAS companies in the last six months. And I know what a SAS company needs to measure. I know what the metrics that matter are, but how do you. How do you take that experience but also kind of like take the business individually without kind of just giving them a list of saying, Hey, no, no, no, no, you don’t get it. Here’s the metrics. No matter. I promise you, we’ve done this before. 

[00:33:25] Ole: So that’s exactly a mistake that I’ve done in the past. And it led to sort of negative feedback from the founder side because of having been involved in many SaaS businesses or marketplaces or e commerce businesses, you do see patterns.

[00:33:42] Ole: And you can judge from the outside pretty quickly what the company should truly be focused on. But if you talk to an e commerce business and come up, come into the room and say, okay, it’s part of recommendations. It’s sort of customer life cycle. It’s a marketing budget allocation. It feels very cookie cutter to them.

[00:34:01] Ole: And they will say, well, you’re just doing the same thing you’ve been doing before, which might actually be the right thing to do. but instead, as you’re saying, you need to have a bit more of an open mind. And I try to be more careful and observing a bit and listening before coming up with a solution that feels a bit too cookie cutter for them.

[00:34:22] Ole: The good thing I have going for me is that our investment team can tends to cook up very interesting new investments in entirely new spaces, pursuing different business models. The whole, I mean, even looking at SAS, the whole seed based pricing is now overthrown with something around AI at the core of many.

[00:34:43] Ole: product offerings. And now we talk about digital labor and how we should charge for that. So things tend to change, luckily so we can draw a bit from experience. but I, I still have to be super observant and listening instead of jumping to conclusions quickly, but thanks for the point. I might still be doing that a bit too often.

[00:35:03] Bhav: I mean, I do this. I, so. Where I work, I’ve come from a background where I feel like I’ve been there, done that, seen it all, but I kind of always want to jump because I want to jump to the answer. But even if you arrive at the same answer, I think one of the things I’ve learned over the last year and a half, work I mean convert is you have to take people on that journey with you.

[00:35:21] Bhav: And we, we practice something called, process consulting, where the idea is that you actually don’t give solutions. You get People to talk and open up and bring their own challenger and assumptions and get, and you’re probably going to arrive at exactly where you want to get to. And it probably would have just been quicker if you just did it yourself.

[00:35:39] Bhav: But I feel like I’ve learned that there is actually a lot of value in getting people to arrive at the same conclusion you would have, because then they’re invested in the process. They come up with, they feel like they came up with the idea, even though you, like I said, you would have nailed it like 99 percent anyway, even if you just done it yourself but, it’s a, it’s a really good point. 

[00:35:57] Ole: Oh, and it’s a more sustainable way that you’re describing, right? Because I mean, you don’t suffer from the not, not invented here syndrome as much because people still very much feel like they own sort of the value creation with data. And they came up with this to a certain extent themselves and they will continue to basically.

[00:36:21] Ole: Yeah. Use this and do it this way long after we’ve gone and moved on to sort of the next company to support. So that sounds pretty sustainable. How, how, how does it work in practice? 

[00:36:32] Bhav: Pretty well, actually. I think sometimes you kind of take what they get and then you. You know, so like, let’s say for example, the way we would do is, we’ll do as a, a workshop exercise with sticky notes and, and whiteboards and walls.

[00:36:45] Bhav: So we’ll map out the user journey in that user journey. We’ll get them to sort of like group key stages. Once they’ve grouped key stages, we’ll get them to identify what is the success criteria for each one of those stages. And they’ll come up with a bunch of metrics and things like that. And then you take it away and you write it up and then you remove the stuff, you know, it’s just garbage.

[00:37:02] Bhav: so, and then you, then you feed them back saying, Hey, cause they’re not gonna remember everything they wrote on the sticky note wall, right? Like there’s like a hundred sticky notes. And I, I’m the one that unfortunately has the, The sad task of having to write it up, but once it’s all written up and you say, Hey, look, these are your ideas.

[00:37:15] Bhav: I can like doctorate a little bit and manipulate a little bit just to get it to where I think it should be. But by and large, they kind of walk away with that. It’s actually is their product. They created it. And maybe they were off slightly, but for the most part, it’s worked so far every time I’ve done it.

[00:37:32] Ole: I like it because it’s very much igniting sort of the process and providing the relevant spark. And for a lot of those teams, and we do come in a lot when data teams are sort of stuck and they lost trust from the business and they just work through sort of an endless amount of incoming requests, but don’t feel like they create a lot of value.

[00:37:52] Ole: A lot of times the magic we’re looking for is in the work that we’re avoiding. So getting them back to sort of. Yeah. I mean, an empty whiteboard and really thinking about the role of data within the business and how it can create actually a real difference. it’s the, it’s the spark, that can be provided by sort of an external perspective, but then they still run with this and they still feel like they own this to a large extent. So I’ll try. 

[00:38:22] Bhav: You get to ring fence it basically. So it’s, it’s done within a controlled environment, but they have the freedom to operate within that controlled environment. And it brings me joy as well, because I could admit as much as I like coming up with the solution, there is. some magic and being able to facilitate an environment where other people come up with a solution.

[00:38:39] Dan: This is, this, okay, this is, this is my little tip here to kind of like curate that experience on the right path, right? So, often when we do these same similar exercises with, post it notes, whether it’s digital or in real life, you go through the clustering exercise. Oh, there’s a pattern around these.

[00:38:53] Dan: And that’s where you can start joining things together under the things that you know, like, where we’re going, right? And so I think that’s a real kind of important step to say, everyone’s just thrown down a thousand different numbers they think is important. But if you can start grouping them together, okay, we’ve got our commercial ones, okay, this is our ROAS or ROI or CPA, but like you, you can kind of direct that with your experience and, and essentially using their own words, you can cluster them under the umbrella, which you can, you can direct and label.

[00:39:20] Dan: And so I just wanted to give that in there because I go through this process all the time, and sometimes I do know. where we need to get to or where they are going to naturally end up But i’m not going to just say let’s jump the step It has to be a collective experience, but you can kind of get the best of both worlds that way.

[00:39:37] Bhav: I was going to say i’m gonna Put you on the spot now ola because we talked about this before we started recording where we’re going to give you the chance to It’s a bit, we’ve not really done this before, so we’re going to give you a chance on the fly to come up with some vanity metrics and identify some sanity metrics and identify vanity metrics for a completely plucked out thinner organization.

[00:39:58] Bhav: So I don’t know all the companies you’ve worked with, but I’m going to hope I’ve taken a, I’m going to find something you’ve not worked in before. So I used to work for a company called Hopin Hopin are a B2B virtual conferencing platform or they were and the goal was basically, so the product proposition is that you can, whether you’re an Amazon and event organizer or something like that during the pandemic, you could bring your events virtually.

[00:40:23] Bhav: So. You’d create the event, put it live. People will come and blah. So, and then we sell this as a subscription for 12 months. So, you know, for X number of thousands of pounds per year, you get access to this platform to run as many events as you want. You, I, you as the, the VC head of data, who’s coming to help me revolutionize my product through data are going to help me build a data, a set of metrics that I’m going to use to track the performance of my product. How would you go about this? 

[00:40:54] Ole: Okay. Hop in virtual conference platform. We didn’t use this one before we use something similar during COVID times. It’s a bit of a time ago, but no investments in that space. Putting me on the spot here, it would have been. Much nicer to just, add vanity metrics and sanity metrics, but let’s, let’s look for the ones that they create real sanity.

[00:41:17] Bhav: And you have the added pressure of knowing that I already know the answers because I spent 18 months doing this. 

[00:41:22] Dan: You think, you think, you know, the answer. 

[00:41:27] Ole: Yeah, I mean, let me think for a second. All right, we’ll simply run this through by sort of the general life cycle and how businesses create or value is created for, the different clients of that business.

[00:41:43] Ole: And, I would go by the initial demand creation where there’s probably some marketing budget involved in creating initial demand and making companies aware that such a solution actually exists. where we don’t want to look for marketing budget spend or website traffic, because these are more vanity, but instead we rather look for sort of efficiency metrics around what you just mentioned, Dan, that could be something around, return on ad spend, for example, across the different marketing channels.

[00:42:17] Ole: If we are looking for a specific drill down that tells us a bit more Are we able to efficiently generate sufficient demand for such a solution, which is even strategically pretty important, because if we don’t, then we might be Be offering not a great USP to the overall market anyways. And especially we’re within COVID times where, the number of virtual conferences was fluctuating quite a lot.

[00:42:46] Ole: That’s probably a metric that we want to keep sort of in place. Because it resembles demand from the platform, let’s see where it’s sufficiently or efficiently are generating than meaningful traffic. Then it’s all about, do they get the value proposition? So are they able to create an event within the different software, the, the software solution that we’re providing?

[00:43:10] Ole: Are they able to create an event without much friction? That is directly tied to the product NPS. but it’s probably, more precise to look for sort of their ability to create virtual conferences. so that’s probably the second thing, related to the actual usage. You have different stakeholders in that regard because you have your clients that are paying for the solution.

[00:43:40] Ole: So basically the organizers of these different events, but then you actually have the attendees as well, which is an entirely different group to track. And eventually the. Clients only pay for the software if they’re able to create events, of course, but more importantly, if they’re able to engage their attendees sufficiently.

[00:44:02] Ole: So you’re probably also looking for the ease of use and the signup rate for the different attendees for conferences and how engaged they are within the different platform. Want to avoid the empty room problem. So there is probably some sort of engagement. Do they sign up for the different sessions within the conference?

[00:44:22] Ole: Are they active through the different chat functions and so on? that almost looks a bit like vanity. So we have to be careful here, but there’s hopefully some sort of a metric that alludes to how customers or how attendees are then actually. then actually engaged because if they are engaged, they will likely attend another conference and which directly relates to customer retention, because they will organize something else.

[00:44:53] Ole: so that’s probably, an interesting company, company health metric to track as well. And, yeah, I would leave this here for now and let me know if I’m on the right track. 

[00:45:08] Bhav: Yeah. I mean, you nailed it. Like, so a lot of the, obviously there’s more. And so asking you to do this in sort of like the space of two minutes was always ambitious, but I think you tapped, you touched on something that, the marketing team and the product team hadn’t really picked up on for the longest time.

[00:45:23] Bhav: They were ignoring the fact that our clients had clients and the value of the attendee experience was a massive factor in the renewal rates when the end of the subscription term came. So the more engaged attendees were. The more, the more the organizers for the event as a success and were more than, well, then more sort of like encouraged to run another event and another event and start to get value out of the platform, which meant that come the end of the, the subscription, they were more likely to renew.

[00:45:53] Bhav: So there you have it, folks who are listening, we’ve just had a master class in developing proper metrics and ignoring the vanity metrics. And, and it was done through a very good, it’s like thought process and critical thinking exercise. So good job. 

[00:46:11] Dan: All right. I’ve got that. I’ve got one for you. And then I think we’re going to have to start winding up just because of time, unfortunately. but what about a, nutrition and supplement company, that sells direct to consumer on their website? It’s digital only. So there’s no in store, aspect, but they also, support, medical and, sort of practitioners who are then got their own clients.

[00:46:37] Dan: So essentially they’ve got this wholesale business. where they work with, medical professionals, in terms of, suggesting supplements, but they also sell directly to consumers. In their perspective, what are the vanity and the sanity metrics? 

[00:46:52] Ole: Ooh, that’s an even more complex business model. let’s look at the different revenue streams this business has.

[00:46:59] Ole: If I understand it correctly, there’s on the one side, the D2C business, which requires almost its own team to generate demand, but then efficiently also converted through the through the website that you’re offering to the traffic you generated. But then on the other hand, you have these sort of, Medical practitioners that if I understand you correctly, Dan are then also offering recommendations for specific supplements and then directly basically distributing them through their specific client base.

[00:47:31] Ole: Is that correct? Yeah. So, A major strategic decision for that business is what’s the more efficient way in distributing our product. Because if you run both things at the same time, they do lead to some sort of operational expenses or OPEX, because the teams behind that is quite different. You have the sort of D2C marketing team, and you need to do a lot on the influencer marketing space and navigate all these different content initiatives.

[00:48:02] Ole: And then also run the website in a way that is truly engaging and helps to convert people. So then conversion rate necessarily is, I mean, you are the experts here, but, is of course a massive driver next to marketing efficiency. But on the other side, you have this interesting Sort of alternative sales channel, which is to the practitioners, which a lot of times, in my experience, actually can work better to scale a business model if you have a sufficient number of practitioners.

[00:48:33] Ole: So that’s probably an interesting metric to track. How many people are you actually working with on this sort of alternative sales channel? And then you have ample numbers on how much are they able to distribute in terms of Product and the kickbacks here that you distribute to these practitioners might be a lot less than what you would have to pay Google and Facebook in a very crowded market for something like supplements, where I imagine a click on a conversion being quite expensive in a very crowded space.

[00:49:10] Ole: So I would very much look into, Sort of what’s the more efficient revenue stream in terms of margin? And how much does it cost you for a new client, for example, for each revenue streams, and then you have the classic gross, gross profit breakdown, right? So you hold that against sort of your cogs, to get to sort of a margin.

[00:49:35] Ole: And there’s a lot you can optimize in that extent. So definitely a more challenging business to truly dissect, but I would very much focus on the revenue streams, optimizing the product portfolio because margins will differ drastically by supplement. And then looking at which products make the most sense for which kind of Go to market motion, and start from there.

[00:50:03] Bhav: I feel like I missed the trick. I should have asked Ola to like solve one of my existing problems that I hadn’t got around to solving yet. It’s just like, 

[00:50:09] Dan: Hey, how dare you throw those allegations at me? This was a completely made up example. It has no bearing on anyone I might be working with. 

[00:50:16] Ole: Felt very made up.

[00:50:18] Dan: Yeah, yeah, no, no, exactly. I just had it. I, you know, I was thinking about it. It just came to me. It just came to me. well, Ola, one, one last thing I like to do to wrap things up, before you’re off the hook is to ask you a couple of quickfire questions. and so they are quickfire. so the first one of these quick fire questions is, what’s one thing happening in analytics at the moment that you’re excited about?

[00:50:40] Ole: Ooh. so. I have to go with AI, but for a different reason, because it does create quite a lot of focus on the field that the three of us have been in for quite some time. Cause all this hype around these emerging technologies, basically lead companies to one conclusion, which is our data is not in order.

[00:51:03] Ole: And that, is. Yeah, very valuable to us because we are usually the plumbers that put in the fundamentals be that the data platform or certain analysis product tracking tools and so on and companies tend to realize how crucial these setups now are to even jump on on this emerging trend. So I kind of like what AI does for this, you just have to tap into.

[00:51:28] Ole: budgets that have now been redistributed towards AI and basically relabel the data foundational work that you do in contributing to the overall AI vision. But if you’re able to do that in a credible way, then us as practitioners, we can really benefit from, from sort of that increased attention. 

[00:51:53] Dan: I like it. Yeah. Essentially if I can paraphrase, it’s, keep doing some really fun, interesting work, but call it AI now because there’s more budget. 

[00:52:01] Bhav: Exactly. 

[00:52:02] Dan: Yeah, yeah, exactly. 

[00:52:03] Bhav: It’s the newest data science, right? 

[00:52:05] Dan: Yeah, exactly. Yeah all all of the machine learning we did before is now ai right, just called it AI. 

[00:52:10] Ole: And before it was big data.

[00:52:12] Dan: Yeah okay next question, is there a book tool or resource that you’ll recommend to someone that are looking into deepen their analytics knowledge? 

[00:52:21] Ole: The one publication I’m following mostly ’cause it very much, meets sort of my interest as Ben Stancil and a lot of people are using, are lead, reading this. But he’s really well suited to outline analytics in the broader scheme of things. 

[00:52:40] Dan: Okay, and last question and then you’re completely off the hook. what do you do outside of work when you’re not doing analytics? What do you do to wind down and escape from this world? 

[00:52:50] Ole: Yeah, I have two sons. So winding down is not necessarily an option. I think like the most dynamic part of the day starts when I come back home. But I truly, truly like spending time with them. And I’m so happy. The older one is now six and a half and he’s finally getting into Lego, which I was huge on as a kid. So I got all my Lego from my grandma, the one that has the notes with the car stats.

[00:53:15] Ole: And I mean, Lego is timeless. You’re like, you can still use what I played with 30 years ago. And we’re now basically spending our evenings and weekends on the, on the floor. scheming around what else we can build based on the pieces that we have and I really like that. 

[00:53:35] Bhav: I have two boys myself, so I know that, my, my eldest is four years ahead of you, of yours he’s, he’s nine soon to be 10 and he’s massively on the Lego, Lego bandwagon. I personally dislike Lego. 

[00:53:51] Dan: Well, I’ll let you off the hook, feel free to jump and do whatever or continue whatever we’ve been interrupting you doing today. And the last thing is if anyone wants to contact you or ask you questions, what are the ways that people can get in touch with you?

[00:54:04] Ole: I think the easiest way is to, find me on LinkedIn. Very happy to chat and dissect your business together with you. If that’s what you want to try out and, yeah, it was really fun joining. I was an avid listener for quite some time, so it’s kind of a full circle moment to come on the show. Thanks a lot for spending some time today.

[00:54:24] Dan: That’s it for this week. Thank you for listening. We’ll be back soon with another episode of the measure pod. You can subscribe on whatever platform you’re listening to this on to make sure you never miss an episode. You can also leave us a review if you can on any of these platforms. We’re also over on YouTube.

[00:54:38] Dan: If you want to see our lovely faces and our lovely guest faces while we do this as well, make sure to subscribe to the Measurelab channel to make sure you never miss an episode. Listen to the episodes as they come out. If you’ll leave us a review, that’ll be hugely appreciated. You can do that on most of the podcast applications, or there is a form in the show notes, you can leave feedback directly to me and Bhav. Thank you for listening and we’ll see you on the next one.

• Spotify
• YouTube
• RSS Feed