#84 The future of product analytics tools

Join Dan, Dara and Bhav discuss the future of product analytics tools and the intersection between product analytics and marketing analytics. Gain insights into the changing landscape of analytics tools, the challenges and complexities of product analytics, and the potential impact of Google Analytics 4 (GA4) on the market. This episode they also discuss the importance of data collection, the benefits of using one platform for all needs, and the potential future of analytics tools with advancements in machine learning and cloud integrations.

Read the Substack they discuss here from Timo Dechau, Mixpanel’s announcement of them entering marketing analytics here and the article from Airbnb about their ‘evolving’ PM roles here.

Follow Measurelab on LinkedIn and on Twitter/X.

Find out when the next CRAP Talks event is happening on LinkedIn.

Music composed by Confidential – check out their lo-fi beats on Spotify.

Master Google Analytics 4 with Daniel Perry-Reed on the next GA4 Immersion 6-week cohort training course. Early bird, charity and group discounts availible!

Quotes of the Episode:

1. “I think as an industry, we’re getting smarter as a collective, you know, marketers and analysts are getting smarter, but maybe budgets haven’t improved as much to have a data science team or a data engineer, for example, to manage all these pipelines.” – Dan
2. “But there’s something deep down inside me that believes that they’re probably going after the Amazon market share.” – Bhav
3. “I was always told that there’s no such thing as a funnel, it’s a pretzel, users don’t act in straight lines. But the funnels are still really useful” – Dan
4. “Sankey diagrams are horrible because they look horrible, they’re not very actionable” – Bhav

---

Let us know what you think and fill out the Feedback Form, or email podcast@measurelab.co.uk to drop Dan, Dara and Bhav a message directly.

---

Transcript

The full transcript is below, or you can view it in a Google Doc.

Intro | Topic | Outro

Intro

[00:00:15] Dara: So we’re recording this episode just a matter of days before the big date in the Google Analytics calendar, which is the sunset of Universal Analytics. So I’m sure Bhav and Dan, I’m sure that’s very much front and centre of both of your minds, or at least it will be on yours, Dan. But we had a really interesting chat on today’s episode. I really enjoyed it, actually I have to say, and I know I say that all the time, this one was particularly interesting, I thought, which is about the future of product analytics tools, but we obviously ended up talking quite a bit about GA as part of that. I learned a few bits, which is always good. But I think we also had quite interesting perspectives. Again, there wasn’t enough disagreement from me. I’m still going to make it my mission to try and get us to disagree on some of these things, but I guess we’re all coming from similar experiences. Maybe our thoughts won’t conflict too often, but there’s even a conspiracy theory at the end of this episode. But I’m not going to give any spoilers away.

[00:01:06] Daniel: We talk about product analytics in the future of the product analytics tools. It’s an amazing blog that I read and shared with both Dara and Bhav and I’ll link to it in the top of the show notes so go check that out. I would say pause this episode now, go read it and then come back. But it’s a bit of a long read, so maybe listen to this if you don’t have the time and we go through it pretty well.

[00:01:22] Bhav: Yeah, and I think what’s interesting about today’s episode is it’s a slightly different take from previous episodes where we have talked about product analytics. I think in this one we kind of start talking about the intersection of product analytics and marketing analytics, and we haven’t done that from the angle of the product analytics space. In the past we’ve talked about product analytics and why GA is moving into that space, or Google is moving into that space. In this one actually, we kind of start talking about how the other big players are coming into the marketing world, which is, you know a good time for them to capitalise and we unpick why that might be. Is it marketing conditions? Are they changes in tech? And so, yeah, so it’s an interesting one, listen out for it and I think you’ll enjoy it.

[00:02:00] Dara: Right, enjoy the episode.

Topic

[00:02:02] Daniel: So there’s a Substack that I follow called the hipster data stack, guy called Timo that writes it. And I’ve been reading this article that they shared the, well, it’s a good couple of weeks ago now, but it’s been one of those things that’s been on my reading list and I’ve been kind of stabbing away at that over the last couple of weeks. And I thought this would be a really interesting topic for us to kind of pick up and talk about. And this is the idea that the long and short of this whole article, it’s an amazing, interesting deep read, and I’ll link to it in the show notes. But the long and short of it is, is that there’s this change in the air around the product analytics space and they address the different tools like Amplitude and Mixpanel and things like Heap as well. And the idea there is that these kind of quote unquote traditional product analytics tools are starting to diversify or start to pivot and start to refocus maybe.

[00:02:45] Daniel: But for example, just a couple of weeks ago, Mixpanel came out with a post and they started introducing their marketing analytics spin on things. Heap have started introducing mouse cursor tracking and things like this. But these are all tools that I suppose classically we’re doing product analytics. So I just wanted to kind of bring that up. I thought it’d be an amazing conversation or topic for us to talk about on the podcast especially now that we have Bhav with us to talk about all the kind of product analytics space. This is slightly out of, out of my depth in terms of this, but I just thought this was a really interesting take on events and I’d love to get your thoughts on someone inside the product space. What is going on? Is there a change in the air? What’s happening with the product analytics tool set?

[00:03:20] Bhav: It’s interesting. I mean, first of all, I should clarify, I’ve only read about three quarters of this article, Dan, so most of what I say is not going to be a regurgitation of the article. It’s going to be my views and interpretations of what I’ve read so far of the article. And it’s a really interesting article I’ve got to admit, and the discussion around how, in the last episode, I believe we talked about GA4 and how Google are trying to move more into this event space and actually how this one, it’s kind of talking about this article that you referenced talked about how the three big players in the product analytics space are trying to move more into the marketing world.

[00:03:54] Bhav: You know, it’s a really interesting one because it’s kind of like both sets of those parties, they’re trying to eat each other’s lunch and yes in reality, Google has not fully captured the requirements and needs of the product teams. And at the same time, these product platforms haven’t captured the needs of the marketing team. So actually this transition that, you know, you’re talking about where Amplitude, Heap and Mixpanel going the other way, it’s going to create a really interesting ground. And actually, I know we’ve spoken about this before, is what does this mean for roles and responsibilities and the crossover of roles, and I think that’ll be an interesting thing to pick apart in this episode.

[00:04:28] Daniel: I think it’s probably worth starting right at the very beginning, Bhav, like just for my own benefit maybe for the listeners too, because we’re talking about these tools moving away from the classic product analytics space or product analytics tools. What is product analytics and why is it separate to you know, every other variation of the word analytics that we use, like what is it that they are starting with, that they’re moving away from? How would you explain product analytics?

[00:04:49] Bhav: It’s very clear when we talk about marketing analytics, we’re talking about the attribution and optimization and analysis of how marketing teams are spending their marketing budgets and trying to find channels and platforms which give them the best return on investment or the lowest cost per acquisition (CPA) or whatever it is they’re measuring. With product analytics it’s such a grey area. It’s not your classical marketing analytics, there isn’t a simple funnel that you would, you know, you want to try and measure and analyse. Product analytics is really trying to look at user behaviour on the website and we all know, like, you know, based on our own experiences, but based on sort of academic annotations from psychologists and all sorts of people out there, that human behaviour is rarely linear.

[00:05:32] Bhav: And what product analytics is trying to do is trying to capture and quantify that behaviour, and in most instances, even the most popular journey still won’t represent majority of your customers. It will only be a fraction of your customers, but we use this ideal journey in terms of your product to define how everyone might be using your product. So I think with product analytics, what you’re trying to do is you’re trying to take a very complex idea of human behaviour and distil it down into its most basic forms of going from step one, step two, or step three. And then you’re trying to, within that create segments and cohorts and then within that you’re trying to understand which of those segments and cohorts are your most valuable and how long they retain. So I think product analytics is just an encapsulation of all of these different things, which you know, which aren’t linear. When you’re doing things like financial analysis or marketing analytics or customer analytics, it’s all fairly straightforward.

[00:06:27] Bhav: You know, you’re not operating in this very grey space. I think why product analytics is so tricky and why it’s such an interesting space to be in, and I think why Google is trying to move into it, plus this article talked about it, is that it’s such a complex space and you rely on how well you’ve tagged your website to be able to capture all of that. So that’s a very long-winded answer, I hope that made sense.

[00:06:45] Dara: Bhav I suggested in a previous conversation or speculated or maybe tried to stir the pot and said, you know, it’s probably not coincidence that Mixpanel very recently, like about a month ago, I think maybe six weeks ago, said they were moving into marketing analytics. Amplitude did this about a year ago, and I said it’s probably not a coincidence that this is on the exact timeline of GA4 and Universal Analytics going away. Just kind of wondering or pondering more on that. I wonder if maybe it’s easy, at least for us to think that way because we work with Google Analytics, but actually maybe it’s nothing to do with the tech and more actually a change in the way businesses are focusing on growth. And I know a very favourite term of yours is product led growth, which we spoke about a little bit before.

[00:07:34] Dara: But I wonder if these technologies are actually responding to a change in what’s happening within businesses. So on the product analytics side, they’re realising that actually product teams are needing to get more and more involved in marketing. Hence they need to have that marketing attribution element within the tech they’re using. And then the other way around the same thing applies. So, you know, if you’re in marketing, you’re going to be suffering from a lack of event based data and you’re going to need that extra capability, especially in these days with websites and apps. So I wonder if it’s actually nothing to do with anybody being scared of GA, but actually a broader change in terms of how businesses are actually approaching growth.

[00:08:12] Bhav: Imagine you were the team at Amplitude, you know, or Heap or Mixpanel, and you know, you’ve seen GA make this big move. Now fundamentally I would be both scared if I was in their situation. I’d be scared as well as excited about the opportunity because from what I’ve seen and the way GA4 has been received by the community that uses it majority of the time, is that they feel a little bit abandoned by Google. They feel unsure about what the future would look like, and they probably to some extent realise that, you know, Google are trying to make this change for a specific reason, even if they don’t fully know why.

[00:08:48] Bhav: Now if you’re the exec, the fact that all of this stuff has happened with these companies and them shifting towards marketing attribution. In the space of a year. You know, I think GA4 was announced 18 months ago, so it’s probably not a coincidence, probably is some reaction to, to Google. It may not be a complete reaction, but there’s probably going to be some reaction to Google. I would be thinking the same things like, hey, how do we now take back some of that share that Google has always taken from us and bring those customers into our space. And the best way to do that is, you know, you’ve already got the trickier side of the puzzle solved, you’ve got the product analytics side of the world solved.

[00:09:25] Bhav: You just need to build the attribution side of things, you need to enable campaign tracking. You need to stitch all of that together because what these platforms do really well is that they allow, you know, you can do things like cross platform attribution, which typically hasn’t been Google’s strongest area. They’ve always over-indexed on the marketing attribution side of things. So I think there is going to be some reaction to Google. So going, you know, going back to your point, part of it’s going to be a tech reaction to Google, but the other part of it’s going to be market. So you’re absolutely right, there is going to be this, you know, everyone’s going to be looking at how do we optimize, how do we improve as a business? How do we scale, how do we grow our market share?

[00:10:00] Bhav: And when you’ve kind of, you know, if you are one of the three big product analytics players and even some of the smaller ones, You’re going to start thinking about going into other domains and territories, it’s only natural. If you think about the fact that Amplitude and Mixpanel, I believe both of them initially started off as app analytics platforms. They’ve naturally evolved to go into more web based product analytics as well. You know, this, I think this is probably just the next evolution and it just happens to have been triggered by all of these things, including marketing conditions as well I think it would be naive of us to rule out the fact that the overall economy and the need to continuously grow in a very tough market, they’re going to have that pressure and the best way to do that is to start going into other people’s domain and territory.

[00:10:41] Bhav: Especially if Google have now got their eyes fixed on building this new beast that they built. You know, they’re not going to be focusing on as much of. I mean, they’ll probably be doing acquisitions, but they’re going to be trying to get existing customers to move over onto the new one, train them up and actually Amplitude, you know, or Heap or Mixpanel or whoever could just go and say, yep, you know, I’ll take some of that if I can.

[00:11:01] Dara: I find it interesting, and maybe, Dan, this is maybe a good question to you and or an opportunity for you to correct me if I’m wrong, but it feels to me like Mixpanel and Amplitude are doing a better job of pushing this message of kind of unified analytics and Google Analytics with GA4 have almost built a much more product focused analytics tool. They’re not really pushing that, they’re not really going out there saying to marketing teams who they would’ve typically had as their customers, they’re not really saying, well, guess what? You can also use this as a product analytics tool.

[00:11:33] Daniel: But, I think they’re just an early player, right? I mean, the thing is that they are the newest player in this market. And because it’s Google, there’s this like, I don’t know, I always refer to it as the Google halo effect, where like everyone thinks it’s like, you know, God’s gift and you know, Google’s the, you know, industry standard, it’s not, it’s just what Google’s done and because it’s so widely used, it’s the most mass adopted tool in any of these spaces, so I’m in two minds about this. So first of all, like Google isn’t the industry, just because Google’s doing something, it could be a reaction to the industry. We talk about like where everyone’s going over to this kind of event schema and all that kind of stuff, because it makes more sense in a, in a hyperconnected world with everything’s connected to the internet, whereas classic and Google Universal Analytics, you know, went before the smartphone era, then sessions and things like that made make way more sense and so I think they are reacting to the market as well.

[00:12:18] Daniel: Google are late, always late to the game, but because they are able to release things for free and dominate so much of the space just because of that aspect of it being free and people have the kind of brand recognition or the brand awareness of Google that they end up becoming quite dominant in whatever space they decide to do this in. It reminds me this whole move right now of product analytics, I do lots of as you probably know, I do lots of training on Google Analytics, Google Analytics 4 now specifically. Google Tag Manager and those things. And everyone I train is often a marketer coming from Universal Analytics and for them it’s, they get this kind of whiplash coming from UA to GA4 because it’s so different because they’re going and I have to tell them to stop thinking in terms of sessions and think in terms of events and, and all of that’s really hard, but when I work with, you know, app developers or app marketers or product analytics people, it’s like, well, yeah obviously. Why wouldn’t I?

[00:13:04] Daniel: You know, this isn’t a new concept, it’s been around for a very long time. It’s just new for a subset of people, which is web marketing and analysts or web marketers or something like that. It’s a huge segment of the audience, right? It’s a huge group of people, but it’s just new for them. And it reminds me a lot of the time when, I don’t know if you guys remember when or any of our audience either, when Google Tag Manager came out and I, at the time I was working at a company, we got bought by Rakuten Marketing, but it was called DC Storm at the time, and we had our own tag management platform that we sold, tag management was not free.

[00:13:34] Daniel: But at the time we had our own relatively successful product management tool, and then Google came out with Google Tag Manager, released it for free and completely devastated the market. We had to pull our products, you know, make it free overnight, basically, because all of a sudden it was like a right, you know, human right to have tag management for free. And you notice that with Adobe tag management and the other tech management tools having to pivot, and you saw that pivot happening, they went into like DMPs and CDPs and they started going off into slightly different sort of tangential areas.

[00:14:01] Daniel: And I almost see this happening now from the product space. Google have done something, whether it’s good or bad, doesn’t matter. They might not be the biggest player in it right now, but they’ve done it and things are going to change because people are going to use it because like anything, you’d be like, oh, just start with a free version before you start paying for a Amplitude or a Mixpanel, just start with a free version, see if you like it, that’s Google Analytics 4. And then if you need to start paying for it or go beyond the capabilities of the free version, then pay for it. And I think Google is really good at cornering that side of the market. I just see this happening again here and again, we talk about this article, which a lot of our audience won’t have read at the moment, but it’s the same kind of themes that I’m seeing surfacing here with all of these tools saying, hey, look, this is a core thing, product analytics, that’s not changing, but let’s start dabbling and moving and having a backup plan. Because if Google comes along with GA4 and just wipes out that kind of like base level market of basic product analytics users, it could be quite devastating for a lot of those tools. Like it was for every sort of tag management platform back in the day.

[00:14:57] Dara: It could be, I wonder if it would be a slower burn. I’m just thinking the difference between when it happened with tag management. Tag management would’ve been something that would’ve been maybe handled within the same team. So it was a case of, well, instead of using Tealium or DC Storm or something else, we’ll use Google’s free product. Whereas with product analytics, it’s going to be a different team. So maybe that’ll still happen, but I wonder if it could be a slower, a slower realisation for people to think, oh, actually Google is now capable of offering this kind of functionality.

[00:15:28] Bhav: I think what’s interesting, you both kinda like touched on it is the free element of it is so critical, it’s going to be the make or break part of Google. Now the fact that the product is complex is probably not even a big deal. I think the free element and getting users to join for free is going to be the thing that decides whether or not, you know, if people want to move to this. But it’s not the only thing, you know, I think Amplitude, Heap, Mixpanel they already have a free element to it. I think one thing that people don’t realise yet, let’s say you’re an early stage startup, as you grow and as the company grows and as your users grow and as you add more events, that’s when your costs start to go up. And I don’t think people are fully aware of the cost models involved with event-based tracking. You know, yes, of course your users will have a big influence on how many events you process and how much you pay. But actually it’s also going to be down to how much you track. And I think this is where Google are super smart because by keeping the product free, and I don’t know what the barrier to becoming a paid product is going to be, is it going to be based on events or whatnot.

[00:16:27] Bhav: But if they keep the basic number of events really high, if people do make that connection and say, actually not we, we’re probably going to approach a million events per month. Now Google does up to 10 million events per month for free. Whereas one of the existing platforms may not, you know, you may not even get to 500,000 events per month.

[00:16:46] Bhav: And that’s going to be the point where people are going to start thinking and potentially thinking, oh, you know what, let’s just stick to Google. And this is one of the things that I think might allow Google to keep that domination of the market is if they keep the volume of events processed high, they definitely can out compete against, you know, and definitely win and maintain that market share. I think if their competitors can keep their freemium product advanced, if they can increase the number of free events that they’re giving away, they can still tap into that product-led growth market where people can go and try it. Because realistically, I think with, you know, whether you go Google or whoever you go, the key is getting that initial adoption and getting adoption with a high level of depth of implementation, because you want to keep that, yes, the barrier to entry is high, but you want to make the barrier to exit even higher.

[00:17:34] Bhav: Okay, it’s not easy to set up a new platform, but once you’ve got it all set up, you basically want to convince your customers that, hey, look, you know, you’ve got everything set up. You’ve got all your historical data, why move over to GA4 or why move over to Amplitude or Heap or whatever? You want to keep those barriers to entries, exits sorry as high as possible by encouraging full implementation of these things. I think, that’s my view on like where Google might still have the advantages, the event tracking, but then getting people to track it, develop properly will ensure that they keep that market dominance.

[00:18:04] Daniel: Well, the thing is with Google Analytics 4 they’ve done something really clever, which is they’ve made it completely unlimited. There’s no limit on the amount of events that you can track on a daily, weekly, monthly basis. So that was a UA thing that 10 million hits per month, they’ve completely scraped that. So you can now theoretically track as much shit as you like for free, within the free version of the tool, there are other elements that you would go into to kind of like find those edges to do the, to pay for it, basically but it’s not based on volume. What I’ve noticed with the other product changes as well, we’re seeing with like Heap and other things as well, because for me it’s like agency models, the marketing agency model of charging as a percent of spend is like, so old school, it only incentivizes one thing, which is spending more right? And it’s such a bad model as a advertiser to spend on because the agencies aren’t then incentivized to sell more, it’s just spend more.

[00:18:47] Daniel: And I think, you know, maybe we’re seeing a slight similar shift in this kinda space where paying on a volume of events or paying per event is so antiquated now. Because actually the game is the analysis of it and the interpretation of it. Actually, you can look at any product out there and it can collect events, right? You can look at GA, you can look at Snowplow, you can look at Amplitude or Heap, you know, does a bunch of automatic, you know, event based stuff, that’s it’s original kind of focus.

[00:19:11] Daniel: Event collection, that’s done. It’s easy now everyone does it. You can even do it yourself, push it into a data warehouse, and you’ve got it there. It’s no longer a challenge to collect events. What I’m noticing with these, like take Heap for example, is about the pattern matching. And if you’ve got a data science team and a bunch of analysts working on this full time, you probably don’t need this. You need data collection, you need a data warehouse and you can start identifying and building your own models on top of this. What I’m noticing with this is now, for example, attribution modelling is a really good one. That’s just a model you’ve applied on top of the event-based data. That’s not anything special, that’s not a new thing to track. That’s just a model you’ve applied on top of the data. Looking at things like Heap, you know, they’re now identifying propensities and patterns in the data. Like if this button is clicked, then they’ve got a higher propensity to do this next thing or to purchase here and this is like predictive modelling in a sense.

[00:19:56] Daniel: But all of this, like almost underlining all of these products, all these tools that we are using are the same sets of events it’s just what they’re now monetizing and using is like, how do we give some analysis or modelling or capabilities to a company that maybe don’t have a data science team? And I think that’s the target here. Like all of this you can do yourself, but do you have the know-how, the skills, the patience for this kind of stuff? No. Okay. Well, you can pay us, you know, free data collection, data collection’s easy and free now, in the new world order. But actually, do you want us to be your data analyst, do you want us to be your data scientist? And I think that’s where you might go through the product selection tools or the vendor selection and say that I need like Heap’s approach to this is actually what I need, or GA4’s approach to this is what I need. And I think that could be, I don’t know if there’s something interesting there.

[00:20:41] Dara: Just extending on from that then and kind of turning that into a question. You’re saying about what people need, it seems at least, and either of you jump in and tell me if I’m missing something here, but it seems to me like GA4, Mixpanel and Amplitude are all heading in a, you know, at least a vaguely similar direction. Whereas Heap at least have recognized that maybe that’s not the way to go for them. So maybe it’s not a case of, you know, just pick one. It could be GA4 or Amplitude or Mixpanel plus Heap or plus something else, obviously there’s loads of other tools out there. So I guess my question, and I’d be interested to hear what both of you think about this, is should we be striving towards a point in time where there is one tool that does all of this?

[00:21:21] Dara: Or actually, is it better to recognize that that’s never going to happen. There’s never going to be an analytics tool that suits every need of the business. And it would be better to recognize that and actually have specific analytics tools for different, either different teams or different needs or whatever.

[00:21:38] Bhav: It’s so inefficient, right? What you’re saying makes sense Dara. Like, it should be, in my view, it definitely should be one platform to do everything because otherwise it becomes so inefficient. Especially when you start talking about, you know, if you’ve got two or three platforms or even one or two platforms that do the same thing. You then have questions like which one is right? How accurate is the data? Blah, blah, blah. That distract people from the fact that actually it doesn’t matter if you don’t have, you know, one is 95% accurate and the other is 93% accurate. What you’re looking for is directional accuracy, and I think that can be more easily attained through the use of one platform rather than multiple platforms.

[00:22:11] Bhav: So I would prefer to see a world where people are just using one, you know, one platform because it’s more cost efficient, reduces conversations about things like accuracy and which one is right, and it allows people to just get on with shit, right? And like do their job as opposed to like faffing around trying to, you know, answer these like frankly unimportant questions, especially for the most like day-to-day type of things.

[00:22:34] Bhav: And I think the Google transition from UA to GA4 is a classic example of, you know, people have built their entire tech stack on Google Analytics and Google Optimize and you know, Google Optimize has been sunsetted and Universal Analytics is moving to GA4. So, you know, you’re effectively in the situation where it’s like, okay, you haven’t got a choice but to move. And none of that data was originally ever in your data warehouse. You know, you might potentially lose it if you don’t move. So I think there’s pros and cons, but I would prefer to see a world where everything is one. Dan, I don’t know what you think on this one, I’d love to hear your thoughts. 

[00:23:03] Daniel: I think we need to be even more specific. Having different tools is not the bad thing. It’s duplication, as you said Bhav. Like, if everyone’s trying to track their own events in their own ecosystem, that’s redundant and stupid. I think, you know, if you’ve got your own event collection mechanism, it shouldn’t matter. But all of these tools are, like I say, think of them as analysis layers. They’re reporting layers on top of the same raw data and each tool will be doing a different kind of skew here on top of your raw data. But the events are the events, if they’re important, they should be in all of them.

[00:23:31] Daniel: So like there shouldn’t be a way of going to GA4 and tracking all the events, but then also adding Amplitude on and tracking all of the events and doing a Snowplow instance and collecting the events through Segment, for example, and having all the events over there. In an ideal world, people can do their own event tracking into their own data warehouse and plug these tools on top. Having different tools reading from the same data is not a bad thing. I think there’s diversity there, you know, it kind of diversifies the market as well. And I think different people are going to have different requirements, like, you know, from a marketing analytics perspective, in a product analytics or maybe a form just, I just want to look at the forms, you know, that might be a different perspective there, but data collection is different to the reporting and the product and the tool that we’re using.

[00:24:11] Daniel: And I think I’d love to see a world where it becomes, and I think we are and give it five years and maybe we’ll be there, but this whole idea of doing server-side tracking and everything going server-side, whether we’re using a plug and play system, I said relatively plug and play, but where we’re using a maintain system like Segment to do that or whether we’re using something like server-side Google Tag Manager, we are hosting it, we are tracking it and we are owning it as an advertiser, as the business. Then we are sending these events and we are collecting these events into our own ecosystem. People might end up using something like GA4 and the BigQuery export there as the source of that because it’s a free mechanism to do that.

[00:24:43] Daniel: And it’s almost plug and play, how curious would it be if then you take like a heap and they can just plug on top of your GA4 data, export in BigQuery, for example, and give you the same analysis, give you the same interface for Heap, but just not doing the implementation. So you’re tracking something once and I think that’s the key thing. You’re only doing one thing tracking wise, and then you can plug these tools on.

[00:25:04] Dara: I like the idea of that. But one thing I just thought of, and I’m just going to try and throw a spanner in the works here, basically, but you mentioned about, you know, if it’s important, you should know about it and you should be tracking it already. But one of the things I’ve always liked about Heap is that you don’t need to know what’s important and with this AI that they’ve now layered on top, it can kind of give you these patterns that you wouldn’t necessarily know about otherwise. So I definitely agree that there’s no point having two similar data collection tools in place, but none of the other tools do what Heap does in that sense where it’s going to help you, and even if you just use that to then inform you of what you should then be tracking in GA4 or Amplitude or whatever. But I like that about Heap that you don’t have to know what you want to track. And if it tells you what’s important, you can then improve your tracking implementation with your kind of primary event tracking tool.

[00:25:53] Bhav: But I think this is one of the things where like Heap, you know, as a concept, I really like catch-all. I like the concept of catch-all. I don’t think strategically it’s the right, like decision. If I think of myself as how do I minimise engineering efforts, of course catch-all makes sense. But then if I think if I put my analytical hat on and the strategic hat of like answering business questions, a catch-all isn’t the right way to go certainly not in majority of the companies I’ve worked at because with the catch-all situation, when you’re, when you’re just capturing front-end defence. They don’t have the richness of data as a server event or, you know, something that you’ve very consciously architected and you’ve added on the additional parameters and properties and you know, all of the things that you, you need to make that event useful.

[00:26:33] Bhav: I think if it’s just picking up some, some signal from the front without any sort of context or who it was or any sort of underlying richness, that’s where it starts to fall down a little bit. But I think there’s value in that. And actually Dan, just to kind of like, on your point, I agree data collection should be a single, single line, but even if the underlying data is correct, you know, as someone who within the business you know, in an ideal situation, we’re talking about platforms in this like very grey crossover space, they should really be able to do all of these things and you know, it shouldn’t be a case of you need to have one platform to do form tracking another platform to do, you know, click tracking another platform to build funnels.

[00:27:09] Bhav: I think we’re in a situation where these platforms should be thinking about how do we bring all of these things together, the use of machine learning to spot patterns in your data that’s certainly very clever but it’s a unique selling point. Is it something that people need? I’m not quite sure if that’s still the case, like, because there will still be nuances in data that you need an analyst to pick apart. So I think it’s fantastic that they’re doing it because it does create a unique selling point for them.

[00:27:33] Bhav: Post Hoc does the exact same thing actually, I recently discovered where you can build a funnel and then there’ll be a little machine learning box at the bottom or, you know, whatever’s plugging it that says, Did you know that these are the key events that drive this as well? So I think there’s value in it, but I don’t know, I just, putting everything into like multiple platforms just becomes so ineffective from a cost perspective.

[00:27:55] Daniel: For sure, I want to jump into it very quickly, but I think this is the thing you said about earlier, about accuracy it’s about directional accuracy and having this, you know, is it 95% accurate? And this comes up a lot actually at the moment, for me, specifically when I’m talking about going from UA to GA4, and it’s another one of these Googleism’s where people, because it says Google, people assume it’s correct, and so it’s always compared against Google. And I’m having that moment with Universal, and I think a lot of people are assuming that Universal Analytics is correct. And so when they’re going into GA4, they’re like, well, how accurate is it? And I’m like, well that’s an unanswerable question because also it’s like it’s only tracking consented user data based on things that some browsers, the cookies are reset like Safari every seven days. You know, other browsers it doesn’t work, incognito might be different, they might have ad blockers.

[00:28:36] Daniel: And I said, actually, there’s no such thing as like, to work out accuracy, you have to have a source of truth to compare it against, to work out a percentage. So in a sense it’s unanswerable. I don’t know how accurate this data because we don’t know what we don’t know. And I think with all these tools, I think it’s kind of similar. It’s like they all have a different USP, they all have a different skew. And I get what you’re saying in terms of coming at it from a you know if there was a single product there, but I have a feeling then if there was a single product, it’s like this ecosystem where you can plug and play different models on top. But maybe again, that just comes back to if you’ve got a data warehouse, could you just go to basically a marketplace in the, you know, the Google Cloud platform or could you go to a marketplace in, well, not even a marketplace, let’s say I just went to GitHub and just found a bunch of repositories that I can copy and paste over is that not doing the same thing?

[00:29:17] Daniel: Because if you own your own data warehouse, there might not be a fancy UI on top, but you can kind of get to the same end result. And I think what’s going to happen is that what they’re doing is they’re preying on people that don’t have the time or the knowhow or the staff to do this kind of stuff. Basically, I think as an industry, we’re getting smarter as a collective, you know, marketers and analysts are getting smarter, but maybe budgets haven’t improved as much to have a data science team or a data engineer, for example, to manage all these pipelines. And so, you know, you take a Heap and stuff and it’s like, you know, it’s doing these funnels, it does some auto-tracking, as you were saying, Dara, it might not be everything, but it’s enough to kind of be like, shit, this is cool, I kind of need this, it’s going to help me do my job. But we don’t yet have the budget for a data engineering team or a science team to make this even better. So I’m just going to buy into that and I think there’s going to be different positions of kind of, it’s basically maturity.

[00:30:03] Daniel: What we’re talking about is maturity here. And I suppose on that path to maturity, I think people will go in and then maybe out of these products. Because I think maybe Google Analytics 4 and the stuff that it’s doing there, because that’s also doing some stuff like predictive modelling. But that’s almost like the starting block, right? Or at least it’s going to be, if not already. And then you might go through these tools like Heap or a Mixpanel, and then you might get to the other side and be like, fuck, we’re just going to do it ourselves and build all this stuff with a data team. And in a sense kind of replace the cost of these products, these escalating costs of these products because we are plugging loads in and just hire internally and build it from scratch. I’m wondering if there’s any permanence with them. I think they’re just capitalising on this change and these teams that don’t have the resources available to them maybe.

[00:30:40] Bhav: I’ve been thinking a lot about this whole event tracking and why Google are moving away from Universal Analytics and into event-based tracking into GA4. And initially my thought on this was really around taking market share from the Amplitudes and Heaps and things like that. And then I started thinking about the cost model and where the money is and if Google effectively keep it free. What was the benefit of them doing all of this and going through this, all this like, hoo-ha and I realised we’re sitting here talking about Google Analytics and Mixpanel and Heap and Amplitude and direct competitors. There is something like inside of me that says that there is a bigger picture at play here, do you guys want to hear it? My theory, do you know who the real competitor is? It’s Amazon, and I’ll tell you why this is my theory.

[00:31:23] Bhav: Amazon’s most profitable service is AWS. So it’s not their primary source of revenue, but in terms of their primary source of operating profits, it that comes directly from AWS. Jeff Bezos has said it a million times, AWS is their most profitable product. And Google are probably losing market share to AWS on BigQuery. So in order to bridge that gap, what you do is you create this product that’s based on event tracking, and you put it behind a very rudimentary platform like GA4. Now, GA4 will do the basic things and it’ll probably do them good enough but Google wants to sell BigQuery. They want their users to take all of that raw click-stream, event-stream, server-side stream data and pump it into BigQuery so that, that becomes the place where you actually access your data.

[00:32:12] Bhav: You then build very clever machine learning models, your data science team can use it. You’ve given something that’s good enough for your marketing teams to use, but you productize the event-based tracking via BigQuery. So this is my theory, right? I said it first here. But there’s something deep down inside me that believes that they’re probably going after the Amazon market share, would love to hear your thoughts on this.

[00:32:34] Daniel: I don’t even think that’s a conspiracy. I think that’s almost explicit in their actions and things like that. I mean, unless I’m wrapped up in this too, Bhav. The thing with that, I think it’s completely true because Google Cloud Platform is their big focus and actually some interesting kind of wording they’ve been using over the last sort of year or so is rather than using the GCP or the Google Cloud Platform products, they’ve called it their BigQuery suite of tools. And I think the term BigQuery is kind of in a sense, quite interchangeable to people, to the Google Cloud Platform. But what they’re going to do, If you imagine you would talk about this event collection, Google’s doing it for free and you’ve got free export to BigQuery. And even in BigQuery, unless you’re a big company and querying the data all the time, it’s actually free. You can actually store a lot of your data for free within BigQuery and who knows, there might even, and I wouldn’t be surprised if over time they’re going to make exporting your data to BigQuery a dependency of using Google Analytics 4, so it’s a mandatory part of setting this up at the moment, it’s optional, but I think they’re just like phasing it in.

[00:33:23] Daniel: But if you’ve got all of your web and app data in BigQuery, why not then build out your data warehouse in BigQuery or the Google Cloud Platform? Because it’s a good starting point. The amount of sort of data warehouses, or maybe a better term for that, it’s like a marketing data warehouse or a marketing datamart, I think all these words are interchangeable. We’re not building like business operational stuff here, but you know, from a marketing perspective or product perspective, most of the data collection is using Google Analytics 4, most of that is in BigQuery.

[00:33:49] Daniel: So starting there is a good point, why not then connect Google Ads and search console into BigQuery, which you can now do super easily and for free, right? And so now, okay, well I’ve got my biggest marketing channel, my biggest organic search data, and I’ve got my web and app data, all within the Google Cloud Platform or within BigQuery. And I think that’s where they’re going to start monetizing it. Because even if you just want to send it out to AWS or read it from there, then you’re going to pay for that. You are going to pay for your long-term data storage. The longer you track, let’s say it’s 10 years later, you’ve got your historical data in there too, and that’s where they’re going to monetize you. And this is where they can do the loss leader stuff like GA4’s collection methodology, like their process and aggregation system, and they’re just going to improve that.

[00:34:24] Daniel: And I don’t think it’s so far-fetched because that is the future of it. And also that’s the future of like, being able to utilise that data for yourself. I mean, we often have this different approach to anything Google-based. The Google word like makes it sort of like, oh, it’s fine it’s Google. It’s like a standard, it’s normal. It’s the default, it’s the source of truth. Actually, we forget that Google Analytics is a marketing tool. It’s biggest thing is Google Ads, it’s biggest money maker. And so when we are let in Google Analytics, which is a marketing tool, give us a tool for free. You know, the classic thing, if you’re not paying for it, you’re the product. And they’re doing marketing attribution, I wonder how they’re going to approach this. They’ve definitely got skin in the game, right?

[00:34:59] Daniel: So I think there’s always this thing with that. And the same thing they can give GA4 for free out, you know, with the BigQuery stuff because they’re going to monetize you elsewhere. I think this is just an assumed thing and they get in early, or I say early, but early in a lot of companies maturities or at least their maturity journey. So it’s like a company might not even think about a data warehouse, and so they’re going to get in with a BigQuery and then they’re going to start using that. And also there’s this big change at the moment is moving over to first party data.

[00:35:24] Daniel: So Google is still Google, right? It’s not your company and it’s all third party data. So whatever Google Analytics is tracking, it’s third party data. If you want it, if you want first party data, if you want to do something with your data, for example, activate it through a non-Google product, or you know, even just own your own data, then you have to store it somewhere. And this is the free, quote unquote, free way of doing that. So first party data bandwagon slash the kind of data warehouse stuff. Get in early, give it for free to everyone using Google Analytics, which is most of the websites and apps out there nowadays. I think it’s, yeah, I don’t think it’s farfetched at all Bhav.

[00:35:57] Bhav: Oh glad to hear it.

[00:35:57] Dara: We’re a cynical bunch, I also agree with all of that.

[00:36:00] Bhav: I love it. I mean, it’s kind of nice to be watching it from the sidelines, right? Like, who’s going to win? Will they, won’t they, you know that, what’s going to happen, type thing. You know, Google are, they try and draw our attention onto GA4. They got us thinking about this event based tracking stuff and against these other players, but actually they’re taking like subtle shots at AWS and Amazon here and there. I want to, if you guys are open to pivoting the conversation a little bit talk about the users and actually, I read something recently about, I think it’s Airbnb who are, I don’t know if they’re closing down, but they’re certainly moving away from the concept of product managers.

[00:36:36] Bhav: And it got me thinking about Amplitude and Heap and Mixpanel and things like that. That article, if we go back to the start of the conversation, it talks about how Amplitude, you know, they don’t say product analytics anymore, they actually published an article last year where they talk about digital analytics and Heap have produced their own marketing analytics and Mixpanel have done the same thing. And the author of that article he actually says something I couldn’t agree with more. He talks about the fact that actually product managers are very difficult stakeholders to start using you know, to get using the data, quantitative data to make decisions.

[00:37:09] Bhav: They’re very good with qualitative data, and in fact they probably could do do most of their job with qualitative data if you let them. So getting them to start thinking about quantitative data has always been a challenge. And I agree, I’ve worked with plenty of product managers in my time, and some of them are great, some of them very like data savvy, but others, you know, they just don’t want to touch data with a 10 foot barge pole. And I wonder if this transition from event-based tracking and moving towards marketing attribution. And Dara, you talked about it, you know is this tech based or is this a business decision? I wonder if this is purely because of the fact that actually there is a ceiling to infiltrating the product space.

[00:37:47] Bhav: And you know, I think I’ve said in the past before, like product teams they typically like to do their own things. You know, they don’t want to outsource to consultancies or agencies or SaaS platforms or anything like that. Most of them believe they can do it in-house, I think if you let most engineers, they probably build their own in-house experimentation platform, whereas marketing teams, and again, it’s called out in this article, the rise of the growth teams are a better ideal customer profile. And again, we talked about the fact that actually Amplitude and Mixpanel both pivoted from mobile analytics to, you know, include product analytics within the web space. Now to move into this marketing attribution, I wonder if this merger of roles is one of those things that could potentially be driving this. I’d love to hear your guys’ thoughts on what you think might be driving this.

[00:38:31] Dara: I think it is because that’s kind of back to the question I asked earlier where I kind of flipped from last time saying, you know, is this all down to GA4 and fear in the market? But I got the same interpretation as you did Bhav, I think it’s also changing within businesses and they’re starting to realise that they need to be. You know, if you’re trying to, if you’re trying to grow subscriptions or you’re trying to grow accounts, then you’ve got to be looking at marketing attribution as well as kind of classic product analytics. Dan we need you to disagree now, just to be contrary.

[00:38:59] Daniel: I’m on board with all of this. It comes back to this idea of they’re all moving in the same direction, it’s a market change rather than a product change. Like GA, Google and whatever is not driving the market here. This is a reaction to the market and I think, you know, changing job roles, it happens. You know, we’ve seen the rise and fall of many job titles, you know, in our space over the years and I think this may be the latest trend and maybe it’s a fad. Maybe they’ll flip flop, they’ll do this as a kind of like, let’s shake things up and then realize it doesn’t work as well when it’s an experiment and go back. So we don’t know, we don’t know.

[00:39:25] Daniel: Going back to the idea of all these products and I think the same that you could apply to the different jobs, it’s a Venn diagram. And there’s overlap between them all right? And it’s that overlap, that is either going sort of further together or further apart. And I think everyone wants to differentiate themselves so much. They see the Venn diagram overlapping more and more and more as things kind of blend into one and they’re trying to act and pull that apart and maybe Airbnb and this example are trying to do the same, and they’re saying, well let’s pull it apart, let’s differentiate this, or let’s overlap them completely. I mean, look, I think going to end that there because I’m in danger of just keep rambling in circles.

[00:39:58] Dara: Well, one more just to change direction at least once more for from me. One other thought that I’d like to hear from you both on is, so we’re seeing more and more machine learning within these tools in terms of laying kind of patterns or insights on top. So GA4 we’ve talked about on this podcast many times before. It’s got a much bigger reliance on modelled data. We talked a little bit today about how Heap has introduced a layer of AI on top of its pretty plentiful data because of course, it is tracking everything that happens in the, in the browser. What we haven’t really talked about is, although you hinted at it, Bhav, you talked about this, I can’t remember which tool it’s in, but where it will tell you or advise if there’s an extra funnel step that you’re not aware of.

[00:40:41] Dara: So that made me think about this concept of self-building funnels or self implementing analytics, and I wonder how far that might go in the future where all of this modelling is actually identifying maybe gaps in the implementation gaps in the data collection, and then is actually integrated into the tag manager or whatever server-side tag management, and it’s actually able to both maybe introduce additional tags, but then also configure or suggest configurations within the analytics platform to say, ooh, did you know that you’re, you know, you’re tracking this funnel, which is five steps, but we actually think we, sorry, we being the collective AI saying, you know, we think there should be six steps. Or actually three because one of them is irrelevant and never happens. So I wonder how far that is likely to go in terms of kind of self building or self implementing analytics.

[00:41:33] Bhav: I love the idea. I think you’re onto something here, like a multi-million pound idea here, Dara, multi-billion pound maybe I don’t even know.

[00:41:40] Dara: Let’s build it.

[00:41:40] Bhav: Yeah, let’s do it. I do like that concept and I think with these tools, it depends on how efficient the tool is and how clever the tool is and how good the machine learning is. But I think there’s also a part of me that thinks that most people intuitively will get the core funnel correct and there’ll be offshoots of it. Where the challenge will be is how much do you let those offshoots and those additional steps, and actually if you, you know there’s a internal mechanism here where actually users who do this thing that’s not in your funnel will then do this. It’s going to come down to the decision making and what you prioritise and what you do with that information.

[00:42:16] Bhav: So, I think I like the idea of not having to think about my own funnels. I think that’s my favourite thing is like, actually, you know what, Mr. or Miss or whoever, machine learning robot, please can you just show me my top five funnels, however many steps they are. And I think that’s a nice idea of not having to like manually build those funnels because I think, you know what, you’re right. Funnels have always been built with a very conscious train of thought around what step one, step two, step three and step four looks like, and what it is. Actually it should just be, hey, show me my funnels, and you just do them.

[00:42:48] Bhav: Now, sankey diagrams do kind of do this, but you need to pick a starting point and you need to explore the roots. And actually, sankey diagrams are horrible because they look horrible, they’re not very actionable. You know, you might think there is some value in them, but I don’t really like sankey diagrams. Only because they give you too much information, if I could just say to the tool, show me my top five funnels, that’s all I need. That is for me, Dara, where it would be amazing to get to.

[00:43:12] Daniel: I like that view of things Bhav. And I have the same thing with the sankey stuff. When I do the GA4 training, we’re looking through the explore workspace, there’s two visualisation techniques. There’s the funnel and then there’s the flow, right? Which is the sankey diagram. And I always start with the funnel because everyone likes a funnel just because it’s simplified, even though it’s not reality, right? Users don’t behave linearly, they don’t do things that way, but you can quantify it with a whole number and you can measure that number over time, and it’s really actionable that way, even if it’s not the reality.

[00:43:37] Daniel: Whereas I think the sankey diagram is more reality-based, but less actionable because like, you pick a moment in time and it’s going to be different to the next moment in time it’s just completely uncomparable over time and seeing kind of relative change. And I think that’s why we always come back to funnels, although I was always told that there’s no such thing as a funnel, it’s a pretzel, users don’t act in straight lines. But the funnels are still really useful, it always comes back down to funnels just to look at this is what we want and this is how well it did, you know, this is how many users did the thing we wanted them to do and you can’t argue with that.

[00:44:04] Daniel: I think, you know, with the rise of things like the GPT world especially ChatGPT and the large language models, I think we’re going to get to a point where it’s just going to be, plug it into your data warehouse, ask of it questions. And then, you know, especially with its coding generative language stuff, especially around the plugin into something like GTM. There’s no reason why you couldn’t be like, okay, we want to do this. Okay, would you like me to publish this tag? Yes you know, in a chat window and it would just publish this tag. I mean, there’s very few and far between times where it needs to be manually customly developed. You know what I mean? It’s all pretty much like, I need to track this form click, you know? And it’s like, okay, cool we can do that. Generate some code, plug it into the API, deploy that away. So yeah let’s not give too much away actually, because we need to trademark and copyright this idea before we give too much of it away.

[00:44:49] Dara: But it’s got to be, I mean, that’s got to be on the long-term roadmap for tag management tools, that there would be a chat interface that you could say, I want to track successful purchases on my site, which happens on this URL or whatever. And I want it to feed this, this, and this into my analytics tool of choice. And the AI in Google Tag manager just goes away and does it for you.

[00:45:10] Daniel: Well, maybe, maybe. I mean, I think if you’re thinking specifically about Google, I think they’ve got bigger fish to fry with their search engines right now, and I don’t think a chat interface to Google Tag Manager is going to be on their priority list for now, but down the line, maybe, maybe a third party script or a third party app that can do that for sure. But that’s why I said we should get in there early, Dara, we should capitalise on this. Five pound a month fee, why not? And then when Google completely wipes it out in about five years time, we would’ve made a quick buck right?

[00:45:35] Dara: We’ll have retired.

[00:45:36] Daniel: We’ll link off to all the resources in the show notes. So feel free to have a look through that blog, that amazing blog that we’ve been reading through and discussing a lot today. Before we part ways, is there anything you’d like to share Bhav, Dara? Any like parting thoughts?

Outro

[00:45:47] Bhav: I guess I’ll just maybe summarise the things I’ve really thought about today, and I’ve taken away. One is event-based tracking yes, it’ll be free for majority of platforms, if you are thinking about event tracking don’t forget to take in the growth of your event tracking and the growth of your users because of inevitably that future bloat of, you know, what you track is where the costs are going to come from. The other thing I kind of was interested in is like actually the crossover of roles, I think that’s going to determine where the market moves and how that will take things. And finally, the secret, secret player in all of this that we’ve been talking about for this episode and the previous episode is Amazon, so we shouldn’t lose sight of that. That’s my takeaway.

[00:46:27] Dara: Mine is, I mean I’m usually a bit of a contrarian and I’m usually complaining about how things aren’t as good as they should be. But actually I feel quite, after this conversation I feel like it’s quite a fascinating time in the space. All of these tools even if they’re taking a slightly different direction, they’re all getting better and they’re all becoming broader, they’re all catering for more needs and becoming more powerful with things like machine learning, with things like cloud integrations. It’s a fascinating space and I think the end users of these products are going to have more and more choice as time goes on, which can only really be a good thing.

[00:47:02] Daniel: All I’ll say is that Google is a pretty powerful force in the market, and although It’s not the driving force, it has a big tidal wave behind it. It’s got a big wake, and I think a lot of the reaction we’re seeing might be caused by the evolution of Google Analytics 4, or the adaptation into the kind of product space. And I think that the reaction of the market is still not fully, fully seen yet, but I think over the next six, maybe even to 12 months we are going to see this kind of cement itself in a little bit and we’re going to start to see what this actually looks like and how reactive these products are actually being to the places like Google. I think that’s the things we’re going to see over the next couple of months, but more to come on that.

[00:47:38] Daniel: I think that’s it. Thanks Dara, thanks Bhav. Thanks for having this chat. Thanks for humouring me by reading through the article I shared and chatting about it, and thanks for listening.

[00:47:47] Dara: That’s it for this week, to hear more from me and Dan on GA4 and other analytics related topics, all our previous episodes are available in our archive at measurelab.co.uk/podcast. Or you can simply use whatever app you’re using right now to listen to this, to go back and listen to previous episodes.

[00:48:05] Daniel: And if you want to suggest a topic for something me and Dara should be talking about, or if you want to suggest a guest who we should be talking to, there’s a Google Form in the show notes that you can fill out and leave us a note. Or alternatively, you can just email us at podcast@measurelab.co.uk to get in touch with us both directly.

[00:48:22] Dara: Our theme is from Confidential, you can find a link to their music in the show notes. So on behalf of Dan and I, thanks for listening, see you next time.