Measured Opinions #9: Is attribution modelling important?

The Measure Pod

Measured Opinions #9: Is attribution modelling important?

Play Episode Pause Episode

Mute/Unmute Episode Rewind 10 Seconds 1x Fast Forward 30 seconds

00:00 / 25:36

Apple Podcasts RSS Spotify YouTube

RSS Feed

This week Dan and Dara chat attribution modelling and how/where/if/when it’s useful and important to invest in.

Documentation on GA4 attribution models excluding Direct traffic – https://bit.ly/3AhMQEA. “Note: All attribution models exclude direct visits from receiving attribution credit, unless the path to conversion consists entirely of direct visit(s).”

In other news, Dan gets a new skateboard and Dara goes out singing!

Leave a rating and review in the places one leaves ratings and reviews, or suggest a new topic by emailing Dan and Dara at hello@measurelab.co.uk.

Transcript

[00:00:17] Dara: Hello, and thanks for joining us in The Measure Pod, a podcast for analytics enthusiasts where we try and make sense of the analytics industry and try and have a little bit of fun along the way. I’m Dara, MD at Measurelab. I’m joined as always by Measurelab’s longest serving Analytics Consultant Dan. Hey Dan, how are you doing?

[00:00:35] Dan: Yeah, not bad thank you. Not bad.

[00:00:38] Dara: Good good. What have you been up to this last week?

[00:00:41] Dan: This last week. Well, I attended BrightonSEO last Friday and it was awesome. Yeah, I really enjoyed it. I suppose the thing I should talk about is the content that was talked about. But what I loved most I think was just the fact of being back in a conference with real people, walking around stools, chatting to people. It was just such a nice, experience that I hope there’s many more of these to come. I can’t wait to do more meetups and conferences, I very much miss them. So how about you Dara, what have you been up to?

[00:01:07] Dara: Well I’m just back from holiday, so I haven’t been doing any learning or any work for the last two weeks, which was really nice. Also something that felt like a reminder of the pre COVID days. So it’s the first overseas holiday I’ve had in a couple of years. So it was really needed, really enjoyed it, nice and relaxing recharging. And I’m, believe it or not, glad to be back to the real world.

[00:01:29] Dan: It’s good to have you back.

[00:01:30] Dara: Thanks, Dan. It’s nice to be missed.

[00:01:33] Dan: I didn’t say missed.

[00:01:37] Dara: All right. What’s our topic this week?

[00:01:39] Dan: So this week Dara, we’re going to discuss attribution modeling and whether or not it’s relevant, and in which circumstances if so. So attribution modeling is been around for quite some time, and I feel like it’s coming back into the conversation. So let’s talk about it, let’s see where we come down on it. ” Is attribution modeling important?” That’s the topic.

[00:02:01] Dara: Big topic, and probably one that we could spend a lot of time talking about. I’ll start, my background background’s in attribution’s a bit less specific, I’d say than yours. So mine comes out of purely from a, like a broader analytics point of view. It’s something I’ve always been really interested in and I’ve always been fairly opinionated about it. And I have spoken about it actually at BrightonSEO. I think that was the first time I spoke about attribution specifically in relation to Google Analytics and the tool set available within that to look at attribution. And that was probably the best part of 10 years ago?

[00:02:36] Dan: It was Urchin Analytics back then wasn’t it?

[00:02:38] Dara: Not quite, it was GA just about. So it’s something that comes up a lot as well. When I used to do GA training, it was an area of GA that most people hadn’t used before. So I always enjoyed talking about it. I always enjoyed teaching people about the intricacies of using Multi-Channel Funnels and the attribution modeling tool within GA. Because it was always something that people didn’t really know much about. Or they knew a little bit, but there are a lot of misconceptions around it where people were making mistakes in terms of how they used it. It’s quite a nuanced set of reports. But I think your experience is more specific and you’ve worked very closely in the attribution space.

[00:03:19] Dan: Yeah, I entered the analytics space back in 2012, in an attribution platform. So my first job getting into this industry was in an attribution platform called DC Storm think then got bought by Rakuten Marketing. And yeah, I spent four years there before joining you Dara at Measurelab, and spent the last five years here. I definitely had a very formative experience in that world and we call it an attribution platform, but it was an analytics tool. They had a GA equivalent where they tracked every page view and event on a website and apps that had an SDK too. But the focus there was not necessarily using it to do dashboards and reporting. But then they had the data modeling behind the scenes to try and work out the value of each touch point in that user journey. Just kind of going back on ourselves one second, we probably should define what we mean by attribution modeling before we jump into the relevance and how we’ve used it. Attribution modeling whether we like it or not is always present in pretty much every report or dashboard I’ve ever used, specifically when we talk about marketing analytics. It’s the process of assigning value to a marketing channel when a conversion occurs. So a good example is let’s say you have a customer who comes to your website via a social media post. They sign up to the email newsletter, then they get an email. Two days later, they click the link and they purchase a product from you for, let’s say a hundred pounds. The question then is, what drove that value, who gets the revenue? Is it the social media platform? Is it the email platform? Is it your social marketers, your email marketers? And there’s this tug of war of I got that revenue, or maybe even more likely is to happen is that both are going to claim that purchase and both be reporting that ROI. And that’s what we mean when we talk about attribution modeling, it’s talking about different approaches to that process of assigning value, right.

[00:05:04] Dara: Yeah, and the default is last non-direct click. Other platforms might have a different default, but there is always going to be a default attribution model in there because there has to be. If you’re going to assign credit for a conversion, whether it’s an actual e-commerce conversion or some monetary value assigned to a conversion event or a goal, then there has to be a way of assigning that back to a marketing source.

[00:05:30] Dan: That’s really interesting you said Google Analytics there because that’s Google Analytics’ default, Universal specifically. GA4 has a very similar approach, although they’ve now introduced a feature where you can change the default model in the property settings. I suppose just more broadly when you talk about attribution modeling, you can only ever attribute across the data you collect. So if you take something like Google Ads, for example, it only tracks Google Ads interactions. So it can never attribute anything back to an email click or your Bing paid search click for example. It can’t do it, it has no visibility. So even then, if we talk about Google Ads attribution, even from a last non-direct click, it’s only ever going to go back to the last ad click that occurred. And GA is doing exactly the same thing, but the only difference with a platform like Google Analytics is the fact that it has a full view of all digital marketing channels, right. Or at least they should do if we’re doing things like the UTM trackers correctly.

[00:06:22] Dara: Yeah, And the only slight exception to that I gues is with direct traffic. And you did say it will cover all digital marketing, but there is an argument to say that, that decision to down weight direct traffic could be negatively affecting offline channels.

[00:06:38] Dan: Yeah, my bugbear with any digital attribution product is generally speaking they end up down waiting direct for one reason or another. And again, take Google Analytics as the example, it’s always the last non-direct click, or down waiting direct. And all attribution models in GA4 now, slightly differently to Universal, all exclude direct. So we have no choice. We cannot change that, direct has been down weighted to a point where it’s only ever going to get a value if it’s the only channel used. But you’re absolutely right, that’s great if you happen to be a digital retailer, don’t have any offline marketing. There’s no word of mouth. There’s no nothing in the real world. And what happens is you only have digital traffic to attribute across in which case, then maybe you could justify that. But the issue is, as you said, is things like offline media. If you have a TV ad running, or if you have a, uh, don’t use the classic example of a billboard and someone drives past and sees it. The point is, is that someone could see or hear about your brand a different way and go directly to your website. That then is a marketing channel, which is offline, so it can’t be tracked. But then that gets swept up into this down weightedness of direct and it just kind of loses all sense of value. And the opposite of down waiting the performance of that campaign is also up waiting another one. So you might be attributing that back to let’s say email, having a skew perspective of how much value your email campaigns are driving. However, what generally happens is all of a sudden, if you switch off another channel like TV, then you see email start performing worse. And then we kind of scratch our heads and ask these questions of why.

[00:08:09] Dara: And that problems made worse if you have a channel like email where you’re not tracking it correctly with UTMs and it can appear as direct traffic in GA. So it’s going to have that effect you’re talking about where not only is it going to down weight the value of that email channel, it’s actually going to give that credit incorrectly to other channels, which are going to look like they’re performing better than they actually are. So it stresses the importance of having all of your channel activity tagged up correctly.

[00:08:36] Dan: Yeah, so knowing the blind spots of a product like Google Analytics and its attribution capabilities, you still want to do the best job it can do, right. And that still put all digital channels that can be tracked into it. Understanding there’s going to be blind spots with the offline world, but try and get those in as much as we can. Suppose it’s probably worth mentioning a couple of other blind spots, that are going to be things that happen off the website or app too right. So things like impressions from your paid search ads or display campaigns or your email opens. All of these things are never going to be tracked within Google Analytics. Only ever, if the user lands on the website from a click. That’s where the tracking starts, these are the touch points that Google Analytics can attribute over. It goes back to that same analogy I had with Google Ads, you can only attribute over the data you collect. Google Analytics has a maybe slightly wider view. But it’s still quite narrow in terms of the data it does collect. So I suppose we were always understanding these blind spots, but I do think going back to the point of whether it’s important, I do think there is an importance there. There is an importance of getting better quality data, even if it’s limited, even if it’s got blind spots. We’re always going to be chasing this idea of perfection, I don’t think we’re ever going to get there. But it’s just making the best of what you’ve got and making these kinds of incremental steps to making better informed decisions with data right.

[00:09:48] Dara: One of my opinions, I guess I’ll call it in the past and something that I spoke to people a lot about was often attribution can be used as a bit of a tug of war, or a case of trying to fight for a bigger piece of the pie. So when you’ve got different agencies or different stakeholders or different people internally managing different channels. It can be used as a way to try and take a bigger piece of the pie for the channel that you’re responsible for. So it can lead to lots of debate or even argument about which attribution model to use and different people will favor a different attribution model because it’s gonna make their channel or channels look better. Whereas I’ve always taken the view that you can learn more maybe by using some of the other tools like Multi-Channel Funnels within GA, where you can learn about the different interactions channels have together where certain channels tend to fit in the conversion journey. So you can glean all this insight by looking at the conversion paths and the assisted conversions reports within GA without actually having to change your attribution model that you’re using. And of course, within Universal GA, you had no way of actually changing the default attribution model. Anyway, you could use the modelling tool, but you couldn’t actually change the default that’s used within the standard reports. Which often means the majority of users are going to continue to use the standard reports and therefore continue to use last non-direct as the attribution model for kind of standardized reporting.

[00:11:13] Dan: Let’s maybe bring it back to some of the models that we actually have at our disposal, right. So I always like to categorize the different attribution models into two types, we have the associative models and the distributive models. The associative models takes the conversion and the value that happens and associates it to a single campaign. The distributive takes that one conversion and that revenue and it breaks it apart and distributes the value across multiple potentially multiple touch points, multiple sessions, multiple interactions with your brand. And that’s really where the complexity comes in to some people because all of a sudden you’re looking at a report and you say, okay well I spent 15 pounds on this campaign over the last two days and I’ve driven 3.65 conversions. It starts playing with your head and understanding, well how can I driven a fractional conversions? And I think that’s where maybe there’s a resistance there in terms of adoption for things like attribution modeling, because you have to kind of explain the whole thing before you start looking at our report. Last click is quite intuitive, it is quite visually easy to understand. Something, drove a click that drove a conversion there and then, right. We’re talking about direct response, someone clicked this campaign and bought from us, great. But it’s harder to explain that someone clicked on this campaign and came back two weeks later and purchased, I feel like that might’ve been something to do with my campaign. And so where attribution modeling comes in is just a way to try and help answer that. And just, cards it on the table, I do like attribution modeling. I do think it can be useful. I’m not saying it’s the answer to every single question. Exactly as you said Dara, like using those Multi-Channel Funnels reports and Universal Analytics, or the advertising workspace in GA4, is the same difference. That’s where the interesting stuff is, is understanding the user journey. I don’t care if we’re applying models or not. It’s just understanding the user journey and what attribution modeling does. It just puts a lens on it. It just gives us an aggregate view of it. So what I love to do is see them side by side. I’d never necessarily just advise, picking one model and say, that’s it. You know, you mentioned this tug of war. For sure there’s a tug of war going, it’s like a musical instrument. It depends who plays it to what unit plays. So you could have a PPC manager create an attribution model that upweights PPC and downweights everything else, and the vice versa with other marketing channels. But that’s already happening within the ad platforms that’s already happening. Do you not think that Facebook will choose an attribution model that the values Facebook more than anything else? And the same with Google Ads and the same with Microsoft Ads. And this is just what happenes. So where I like using attribution modeling is not necessarily in those tools. But where I like to do this is within something like Google Analytics or Adobe Analytics or wherever. Where it’s, within reason trying to track all marketing channels or digital, at least marketing channels. And then I have multiple different lenses I can apply to that data to say, well how about an a first click world, what in a last click world, in a linear distribution world, what in a data-driven world using the kind of black box algorithms from Google Analytics or whatever. Having these all side by sides gives you that visibility into understanding where channels play a role and where they may be don’t. The reason why attribution modeling becomes a bit important, not important, where it becomes relevant is classically. You know, you do your quality reports or the board needs a quarterly reports and they need a pie chart of revenue by marketing channel. And then you just have to pick one, you have to do one, right? You have to pick a model. You can’t have 15 pie charts side by side and said, there they are. You know, sometimes they just need to know what’s my return on ad spend. And so, yes, in which case then probably at that level, you need to pick one. Maybe pick a last click or maybe linear because it’s quite easy to explain, or some other model like that. That’s going to be different for each company.

[00:14:45] Dara: You mentioned the data-driven attribution there, and I think people can get very absorbed into arguing between last click, first click, linear, time decay. The reality is they’re quite, arbitrary is the wrong way to put it, but you’re selecting a single model and assuming that that’s going to continue to work and you could go back and review it and maybe change it. But it’s not a very adaptive, it’s not an adaptive model at all. So data-driven attribution avoids that issue. But if you’re going to do it within the GA360 functionality, there is a major drawback which is the fact that it’s quite black box. So you don’t get to see exactly what it’s doing. And there is some documentation around the modeling, but it’s fairly limited, you don’t really get to see what’s going on within the models. And also it relies that some thresholds as well, you need to be getting through quite a lot of conversions for it to work. But there are other approaches outside of that, so you could create your own machine learning based attribution models. But then you’re moving into a space where you need to be fairly mature in your analytics, and that’s not going to be for every company. That would be really jumping kind of light years ahead if you’re currently using last non-direct or you’re just taking data straight from GA, to suddenly jump to creating your own machine learning models. It’s a fairly advanced jump ahead. Um.

[00:16:07] Dan: So it’s just that difference isn’t it, between rules-based and algorithm based modeling. And I think it’s at least in my experience where I’ve had to kind of come up against this before is you’ve got rules-based models. They’re easy to explain and to validate, but they might be quite limited in the output. Whereas you’ve got your algorithmic or machine learning models that adapt and evolve and change with the marketing campaigns or the marketing mix that you’re working with, but you can’t really easy to explain it or validate it right. So you have to kind of weigh this thing up. And I think that as exactly as you said Dara, it comes back to the maturity. We’re not saying that if you are mature, you do attribution modeling. Of course that’s not always saying, what we’re saying is that if you’re a point in your organization where you need to explain things super clearly it’s a point of change and you’re moving maybe away from just using last click or the default attribution model in these platforms. And maybe starting with some of these rules-based models is going to be more preferable because you don’t want to jump too far advance where you’ve got a bunch of people that have no clue what you’re talking about. What we need to focus on is around that kind of value get from having different lenses on your data first. And then you can talk about improving those lenses or adding more lenses on as you become more mature, as you get more used to using that data, optimizing your campaigns off the back of it, and so on and so forth. But just pulling back a sec, I think I just want to drill in a bit more around optimization. If someone comes to me and says Dan, I want to talk about attribution modeling, I want an attribution model. Generally there’s red flags of like, okay, well maybe we need to start right at the beginning you’re already using an attribution model, what’s wrong with the one you’ve got, what’s wrong with last click? How is it not answering the questions that you want to answer? And then we can talk about moving into different rules-based models, even data-driven eventually those kinds of things. But the thing I always go to is, well, are you optimizing your marketing campaigns or your ad spend based on last-click? Because if they’re not currently optimizing anything based on the data, then it’s just a vanity project or a dashboarding project of like, I can spend hundreds or thousands or tens of thousands of pounds working with someone to build better and more full attribution models, better view on the user journey, understanding machine learning models applied on top. But if you’re not currently using the data to make informed decisions, then this isn’t going to change anything. You’re almost kind of chasing after this holy grail you never quite get to. We said that there’s always going to be limitations with every step of the way, but it’s about making the most of every step. So are you optimizing your campaigns to anything? If so, then maybe we can get better at doing that. I would never start with attribution modeling. Attribution modeling is a means to an end. It’s not the end itself.

[00:18:43] Dara: Yeah, completely. And I think it’s the usual case of walking before you run. Another way to think about it as well as going back to the question of this topic, is attribution modeling important? Yes, of course it is. But what attribution modelling is, is going to be different depending on different circumstances or where you’re at in that journey. So it could be as simple as using the assisted conversions reports in GA to try and prove or disprove if a channel is under performing based on a last click model. So often what you hear is social media marketing isn’t delivering or display advertising isn’t delivering for us based on last-click reporting. You can very quickly jump into the assisted conversions reports in GA to try and see if those channels are performing a different role, higher up the funnel and then you can validate one way or the other. To somebody else, attribution modeling might be a fully custom built machine learning model integrated with a data warehouse. It could be very sophisticated, could have a team of data scientists and data engineers supporting that. It’s not a black and white situation, it’s not like do attribution modeling or don’t do it. It’s something that can be looked at as you go on that journey from maybe fairly straightforward reporting through to much more comprehensive and sophisticated reporting. And you’re absolutely right about your optimization. It’s like, there’s no point spending time, money, resource on something complicated that isn’t going to be used.

[00:20:10] Dan: Yeah, absolutely. Absolutely. This is it with attribution, again going back to one of the first things I said actually is that you’re already using it. Whether you’ve labeled it as attribution modeling, you are already using it. Anyone that’s got a dashboard from Google Analytics has been using attribution modeling. It’s just the default one is last non-direct click. So we talk about is attribution modeling important, of course it’s important, without it we wouldn’t have any campaign performance data. But the concept of going beyond a last click view or beyond the defaults, that’s the thing of question here. So, is attribution modeling important? If it’s just Google Analytics data, if your not even tracking all of your marketing channels with UTM parameters, if you’re not even uploading cost into it, or if you’re not looking at optimizing your campaigns based on the data that’s already there, then probably no, right. Especially if your brand relies heavily on interactions that happen outside of the session. GA is a session based analytics product or at least Universal is. And if all you’re doing is attributing sessions, is that enough? We refer to them as touch points when you talk about user journeys and attribution modeling. Is only looking at the touch points of sessions enough or do you need to start thinking about the other side of this, which is as you said Dara, pulling into a data warehouse, blending multiple data sources, having a full view of your customer, understanding beyond the cookies and sessions. Is there other touch points, you know, calls to the call center, or do you have sales reps if it’s B2B making calls back to the leads. So if you’re that kind of second example where you’re putting the data into a data warehouse, blending it, if you’re already optimizing your campaigns or using that to inform decisions or strategies, then yes, attribution modeling is still very important. Because you will still need a clear way of understanding value assigned to your efforts and understanding where to push and pull those leavers in terms of ad spend or effort or energy or time. As you said, it’s just not about saying attribution modeling is important, you need to have a better model. It’s not always about jumping to data-driven attribution modeling as you said Dara. It’s about walking before you can run, and just really understanding that it can just give you different lenses on your data to help you make better informed decisions. And if you think of it from that perspective, it is important.

[00:22:17] Dara: I think you’ve given me the conclusion as well which is, yes it is important, but how important it is to you is going to depend on how sophisticated your requirements are, where you’re at in your analytics, maturity, what you’re spending on media, how you’re optimizing that, and who you’ve got to make use of the data as well. So what kind of team you have and who’s going to support the more complicated end of attribution modeling versus just the simple fact that everybody has to care to some degree about it, even if it’s just understanding what last click attribution means.

[00:22:50] Dan: Yeah, exactly.

[00:22:52] Dara: We’re undoubtedly going to come back to this topic again, so I think that’s probably enough for now. What have you been doing outside of work Dan to relax and unwind lately?

[00:23:01] Dan: Outside of work? Well,

[00:23:03] Dara: Or have you just been working that you just work all the time?

[00:23:06] Dan: Well, you know, if the MD takes two weeks off, we’ve all got to start working evenings to cover the gap you see. Um, no, this, this week, I say this week, over the last couple of weeks I’ve really been getting back into skateboarding. And I just love it, it’s just such a really good excuse, especially with the nice weather we’ve been having recently, just get outside. You know, meet up with some friends, travel to different parks and escape. And I’ve recently bought myself. I said recently, yesterday I got a new skateboard and I put it together. And where is it, here it is, it’s a one not great for the listeners, but,

[00:23:36] Dara: You’re going to have to, everyone will be jealous and this is probably wasted on me anyway. So you’re going to have to add a picture of it. It looks cool, it looks like a skateboard.

[00:23:46] Dan: It’s a yellow skateboard with pink wheels and multi multicolored, uh, bolts. So I’m well happy. Um, haven’t even taken outside yet. I’ve got a go into a skate jam on Sunday, so we’re going to go down there and christen it I suppose, and hopefully not hurt myself. It might sound like a city question Dara, but what have you been doing to wind down from being on holiday?

[00:24:07] Dara: Well, I mean, I had an easy get out here. I could have just talked about being on holiday, but actually, after getting back from holiday, we went to Lucky Voice, the karaoke place. Me and my partner Hannah and we, she’s a singer, so she has no excuse, but I, I was terrible. But I’m allowed to be terrible because I’m not a, not a singer. But it was good fun. It was different, and I’m not sure I’m going to be rushing back to do it again, or, or rushing out to quit my job at Measurelab and become full-time singer that that ship has definitely sailed. Uh, but a few drinks, badly singing a few songs. It was, it was really good fun. I think that’s a wrap for this episode. As usual, you can find out more about us at measurelab.co.uk. Or you can get in touch via email at hello@measurelab.co.uk, or you can find us on LinkedIn and ask us any questions you might have or even if you want to suggest a topic for us to discuss. Otherwise, join us next time for more analytics, chit-chat, I’ve been Dara joined by Dan. So it’s bye from me.

[00:25:09] Dan: And bye from me.

[00:25:10] Dara: See you next time.