#117 Marketing incrementality, attribution & ROI optimisation (with Jorge Roberto Ahumada Garcia @ Coppel)

[00:00:00] Dan: Welcome back to the Measure Pod. I’m Daniel Perry-Reed. I am an analytics consultant at Measurelab, and of course, I’m always joined by the wonderful Bhav Patel, director of Analytics and Experimentation at LeanConvert. Now Bhav, we had a guest on this episode. Jorge, worked at Coppel, which is a huge online and physical retailer and bank, and it seems everything in Mexico.

[00:00:36] Dan: exploring things like MMM and multi-touch attributions and everything else with a nice little surprising, nod and appreciation for GA4 in there, which was unprovoked for me. I might add, what’s your initial thought? Just coming off the back of it, 

[00:00:50] Bhav: I, this reminds me. What we do is it ebbs and flows between new ways of working and then always having to go back to old ways of working to solve new problems.

[00:01:04] Bhav: And then we get new problems and we find new methods, but then we have to go back and like find old ways to solve those new problems as well. So I, I dunno, I really enjoyed this episode. I, I actually really liked the way that Jorge talked about the fact that. And, and I think it was in swings of roundabouts we got there, but there was this emergence in the reliance of technology to do things that historically, you know, people were really expected to do themselves.

[00:01:29] Bhav: And it’s, you know, they’re very smart people, obviously. And I think there is this underlying belief that you no longer need to understand these things because the technology will do it for you. And I think this episode served as a. Yes, the technology will do it for you, and I think it’s okay to use the technology to do it, provided that you know what the technology is doing.

[00:01:50] Bhav: If you are blindly trusting these models, MMNs, MTAs, attributions, whatever, and you don’t really understand what’s going on, you kind of leave yourself exposed to difficult questions. Worse, best case. Like negative impacts on, on commercial performance, worst case. So yeah, I think all in all, a very interesting episode.

[00:02:11] Bhav: I think, I think Jorge’s got a ton of experience here, and actually it was a real pleasure listening to him talk about his experience and share it with, because I, I, I think I took away a lot from it. 

[00:02:21] Dan: Yeah. And, and, and I’ll just add onto that, before we jump in, but one, one of the, one of the first things that, they mentioned is the, the kind of beauty in that we’re all in this industry together having similar challenges.

[00:02:31] Dan: So, Jorge is in a, a completely different part of the world to us with different, economics, different legislation, different cookie banner policies, even, everything like that. But we’re still trying to solve the same problems and, and talking about it in the same way and sharing ideas. And I find that fascinating that, you know, as a community we all kind of get together and.

[00:02:50] Dan: Before we jump in, Bab, is there anything you wanna, say, plug or share before we let everyone listen to the rest of the episode? 

[00:02:58] Bhav: I have one thing I find I wanna plug, and it’s only ’cause I spent all of like last night working on it. Keynote, as you guys know, is this, platform that I built to connect speakers to.

[00:03:09] Bhav: Event organizers, podcasts, hosts, webinar organizers, companies who want to run training, anything like that. I’ve, I’ve given a massive facelift, and over the course of the coming weeks, I’m gonna do some more work to just get it back up again and running. So please do if you, you know, if you are a, if you are a speaker or if you’d like to speak at events or join podcasts and you are trying to find a way to be discovered, do create a speaker profile on Keynote.

[00:03:33] Bhav: So that’s the thing I’m plugging. How about you, Dan? 

[00:03:36] Dan: I’ll second that. Actually, we use that to find people to speak to on this, this show. So if you ever fancy yourself, potentially want to come on this show, then sign yourself up at Keynote and you are on our radar. for me, I, I think, something I, I plugged last time, other than my wonderful time at Super Week, which I’ve just come back from at the time of recording, is going to be, crap Talks Brighton, which will be coming up early.

[00:03:58] Dan: And there’ll be a link in the show notes once that is there. And it’s all on Meetup. It’s all free to attend. And we’ve got a cool new venue and some really interesting stuff going on. So, I’ll put that in the show notes, but, I don’t wanna hark on about it too much ’cause this isn’t about us. This is about Jorge.

[00:04:11] Dan: So without further ado, enjoy the episode.

[00:04:17] Dan: Okay, here we go. And we have Jorge joining us, from Capel all the way from Mexico. Right. how are you? Tell us a bit about yourselves and tell us how, I don’t wanna say how you got into analytics, ’cause that’s, something we’ll find out in a moment. It’s not quite exactly the right way to phrase it, but how did you get to a point of meeting people like us and get into, talk to us on a podcast about analytics? Jorge, 

[00:04:37] Jorge: Thank you. As you said, I’m, I’ve been working in COP for the last four years now. Coppel is a, is the largest private retailer in Mexico. We, we have, more than 18. So. Main purpose is, to be the omnichannel preferred option of the Mexican families to come and buy goods, to come, and have their personal loans. And, group of which is a group, it’s a. Also holds a bank. So we both, let’s say, well, let’s say facilitate to the Mexican families, both, goods and financial services.

[00:05:46] Jorge: So, I’m very glad to been here because, for the last five, five years, like, researching and analyzing things like attribution and how to give credit. A specific strategy that drives the consumer into the conversion. it’s very, very interesting. So, yeah. Thank you very much for having me. 

[00:06:13] Bhav: No, it’s our pleasure to have you and I guess. One, one maybe just for my clarity, just ’cause I’m not familiar with the e-commerce landscape in Mexico, how much of what Copal does is install based transactions versus how much of it is online? 

[00:06:30] Jorge: Yeah. in every. eCommerce business represents the 10% of the whole business. So for every, let’s say.

[00:06:48] Dan: Amazing. Well, look, this is, a fascinating, topic conversation. If anyone that knows my background, I, I, I cut my teeth in the world of analytics in the attribution modeling space. And, I’ve just come back from a week long analytics event called Super Week, whereby I was talking to a couple of people about attribution modeling.

[00:07:05] Dan: So even up to, you know, 10 to 15 years later, we’re still talking about the same thing. And I think this, this, Jorge is something that, that, that drew me to our conversation, that when we connected on LinkedIn, that we are. Debating it, we’re still talking about the same thing. Now. Other things have entered the conversation, like, uplift and incrementality, media mix modeling, marketing mix modeling and those kind of things.

[00:07:27] Dan: But I would be just really interested from, from your perspective, what’s your, what’s your background like? How did you find yourself dealing with things like attribution modeling and, and media mix modeling and, and what, what kind of part of that system are you playing at cop? 

[00:07:40] Jorge: Yeah. First of all, my academic background is being an economist. So let’s say like, with a bachelor’s degree and a master’s in, in, in economics, like, let’s say in, in me, like inside me there’s this, curiosity about finding out what happened. If I do a certain thing and what doesn’t happen if I don’t do the, any, let’s say, project, public policy, it’s what I, what I’m academically formed in.

[00:08:17] Jorge: So when I came in, they, we, we were kind of discovering the problem because in first view, the problem we had was. The voice saying, digital marketing budgets, it’s so high and we don’t know what is happening with the digital marketing budget. Like where is it going? What is happening? Like, it’s, is the website, receiving so much traffic? Very well? how are we, shifting, to the app era? is it happening? Is it.

[00:08:57] Jorge: So we were like, step by step when I was at this, researching stage where I found that there’s a thing that, it’s called attribution. That the whole industry is having the same problem that I am having, as an analyst. that was, a relief and I was like, oh man, well, I’m not alone. You know, that’s, it’s, it’s when I started like looking for people, like in the real, like in the outside world, like with, with the same problem like me, it’s when I found you guys is when I found, Measure lab and uplift, I think is, is is the other American agency that, that I’ve been following, for, for a couple of years now. They’re these German guys, they’re called, funnel, I guess that they, they just changed the name. it’s very interesting what, what they do, as well. So we, I was, I saw a post of you, Dan, I, I, and, and I don’t remember what the post was about, like specifically, but, but I replied to the post and it, it, it’s how it happened. We, we shared emails and here we are.

[00:10:17] Bhav: I love the analytics community. I think one of the best things about it is that when you do find yourself in a problem, you realize that actually it’s a problem shared with many people and the world feels a little bit less lonely. And maybe a little bit brighter.

[00:10:31] Bhav: That’s my, that’s my positivity for the day. Now then back to back to the negativity. I was gonna say huray, what’s, what are the, one of the, some of the, I, I wanna start from the top and kind of like work our way down because I feel like this topic is quite broad and you coming into it trying, you know, discovering that actually it’s a problem shared across the industry.

[00:10:52] Bhav: How did you first go about tackling in the absence of knowing that there were other people doing this type of, problem solving? How, you know, what was your approach? Did you go straight into your economics background and, and apply, this kinda like economical thinking where you’re building models and trying to determine uplifts in terms of impact of marketing spend? Like how, you know, how did you first approach this? 

[00:11:13] Jorge: The first thing, we did as a team was the first decision we made was. Stop listening what providers said about the impact of your strategies in your business. In, in, in our business. Let’s focus on the data we have and let’s analyze what the consumer does in our.

[00:11:48] Jorge: Let’s see how the attribution look like, you know, and, and, and stop hearing. Like, you should be doing this model. You should be buying this new thing. You should be listening to this new firm. Like, people in other countries are doing this. People in other firms are doing that. what we first did was, okay. The people want these specific things for.

[00:12:22] Jorge: Best personal loans givers in Mexico. So one of our main, products, our financial products are personal loans like, and, and we partner with Mexican families in order to pursue, to, to pursue certain, life objectives. So let’s say.

[00:12:45] Jorge: People’s life better. It’s, it’s our, our motto. So that was our first approach. Stop listening, and maybe this will sound very, like, close minded, but when we first understand what was happening inside. We could start, let’s say, playing with models, with rules, with analytics, with dashboards, and with stuff in order to go to the findings stage and start making decisions.

[00:13:24] Jorge: So, yeah, like, it, it’s, when I say like, in this, 20, 25 world full of technology data and, artificial intelligence like human understanding is more important than ever because it, what I think is that we as understand what. Never be able to use any technological tool. I dunno if that makes sense for you. 

[00:14:00] Bhav: No, it does. And it’s actually, the timing of what you just said is, is really important. It’s really, perfect actually. ’cause I was talking to some of the members of my team today about actually how the adoption of things like large language models is okay.

[00:14:14] Bhav: If you know what you’re doing in the first place, it’s using things like chatt cha t. Is fine when you just need to automate something you know how to do yourself. I think the risk becomes when you are doing, asking things like chat, GPT or whatever, to do something that you don’t understand yourself, because then how can you ever.

[00:14:37] Bhav: Even begin to QA or understand if what the output is is correct. So I think that for me is, is so important that, the human understanding element of what you are doing, because you’re right, I think attribution modeling can very easily become a black box and I. Your first point, stop listening to what the vendors are saying.

[00:14:56] Bhav: I think that’s gonna be my new life motto. I’m just gonna, I’m just gonna adopt it and embrace it fully. So stop listening to what people say and look at the data fuels, which I do anyway, but it’s kind of actually nice to phrase it like that. So, no, I think that made perfect sense to me. How about you, Dan?

[00:15:09] Jorge: Well, maybe it will sound, maybe it sounds kind of. 

[00:15:19] Bhav: Sounds great, is how I would say it. 

[00:15:22] Jorge: But, but you know what, but you know what, like, whenever you go to the, to the market, let’s say not, not not the analytics community, but when you go to the market, let’s say I have this problem, the answer is buy this thing, use this thing. And, and you. Oh, well, I found, I, I found myself in 2020 in the middle of the pandemic, surrounded by excellent, excellent salesperson by trying me to buy things that I was not understanding, the use of them.

[00:15:57] Jorge: I, I was not understanding what was happening. On my data, on my business, the data was not organized in business. Every minute. This the impact of your.

[00:16:16] Jorge: Okay. In a company that has four cell channels, that is, brick and mortar cells, we have, digital kiosk in, in, most of the stores, and we have the app and we have the website. The first thing we needed and we wanted to know is what of all the digital messages, brick.

[00:16:46] Jorge: The characteristics of our clients are not the most digital. They wanna go to our stores, but they go to our stores to buy cell phones, to buy computers, to buy tablets. So they are looking to our digital messages. They’re looking to our digital media campaign. They’re, they’re looking for our, so how are going to give economic value to those messages we’re sending by those buyers. 

[00:17:23] Dan: So I’ve got, I’ve got two, two things I wanna ask you, Jorge, and I’m gonna let you pick which one you want to go down. so this is the, the first one is that, that is an awful lot. Fragmented data, from lots of different data sources, that is very difficult to join together.

[00:17:40] Dan: So my, my first point you can pick is how the hell do you even begin to process that, when it’s so vast across 1800 stores across all these different systems, to be able. To do the things like attribution modeling. and the second, the second one, which you can optionally choose to take if you want, is that 1800 stores and Mexico’s biggest retailer and, and loan provider is huge, right?

[00:18:03] Dan: We are not talking about an individual shop that’s just set up. Five employees and they can do whatever they want and they can change their mind later. Implementing something like this and getting the li people listening and acting on data at that scale is, is a huge undertaking. So how did you even begin to approach that when actually the risk is higher and there’s more people to convince?

[00:18:25] Bhav: This is like one of those, did you ever read those? Choose your own adventure books. So, this is the first episode. Ever let it be known That’s given you a choose your own adventure question. So, alright. You get to choose your own adventure with Dan’s questions. 

[00:18:39] Jorge: Yeah. Yeah. Thank you. Very interesting questions. And, first I wanna answer the first question B asked, during the, this conversation that was the representation of e-commerce in, in the, in the business. And my response was 10%. But if you look to the channel cells. If you go to the website and some the application and some, the kiosk is around the 10%, but if you ask me the impact of e-commerce in the business, the number is really, really high.

[00:19:24] Jorge: The growth boost that the company is having this year is by eCommerce. So we might not be what the e-commerce or the digital side of the business might not be, the biggest in numbers, but is definitely moving the whole business. So that is very interesting, and that’s what we found in our, research, online purchase, offline, analysis, when we first, decided that we wanted to go.

[00:20:06] Jorge: By last click, non-direct, direct attribution, strategy. Of course, there was, a bunch of difficult conversations we had j just to one click. That’s unfair. To to, that’s unfair to us that we do, higher funnel strategies and we are never going to, be seen in the p and l finance analysis, and we are never going to, you know, the point was to make all teams align and tell them, okay, if you wanna be in the.

[00:20:52] Jorge: You gotta go to that last click, non-direct to that. You, you need to pursue that click. If that is gonna be your business goal. You, you know what you gotta do? If you wanna have a healthy higher funnel in order for the performance strategies, come and just convert the people you are working with. That’s another business object objective.

[00:21:20] Jorge: You know, so what, what do you wanna do? You know, this is, this is a measure, this is an analysis, you know, and this, and, and we, and if we have the right conversations, all the roles, of the different, let’s say, sales strategies will align and get people’s attention. Make customers convert, you know?

[00:21:49] Jorge: And so, convincing people, convincing people took us a really, really long time. We had to do, and I think we’re not done yet. we, we needed to go very, very strategically. We needed to go with certain stakeholders first.

[00:22:12] Jorge: Maybe you are not, getting the credit because the people you impact go to the stores and convert there. And it’s fine. I mean, whenever, whenever you don’t purchase in other retailer, for me it’s okay. You. You know, and, but there was the belief that the digital business and the physical business were parallelly, going, you know, but I think that it’s been, a process.

[00:22:57] Jorge: When the company, which is an 80 years old company, needed to understand that the new way to make business was omnichannel like. And, and I think that was, the best thing that could ever happen to the, to the business to.

[00:23:20] Jorge: Evangelization, if I may say, like, to people to understand, like, look, the, these physical cells, I can tell you the names and the, and the data of the people that, that were, browsing in our app or in our website. And they were looking to clothes, they were looking for refrigerators, electronics, you name it.

[00:23:48] Jorge: And then they came to the store and bought something related to it. So I’m sure there’s an impact why I wouldn’t give it to it, you know? 

[00:24:02] Bhav: So it was like, I think the challenge there is that middle step. I think you’re right. Like you can easily quantify. What people are doing on the site and what they’re doing, and what they’re looking at.

[00:24:12] Bhav: And you can easily quantify what’s happened in store and what’s been purchased. I think it’s that link crossing over the digital behavior to the in-store behavior. Now, I think if you have some type of identifier which is carried over, like for example, you do click and collect, right? That’s an easy one, or.

[00:24:36] Bhav: Or something, you know, something along those lines. I’m not sure what else, right? Like if you get a policy id, if you’re doing a, looking for a financial loan or something like that, you can then go and complete the, the, the, the, you know, the claim or whatever in store. I think in those instances, it’s, it’s fine.

[00:24:50] Bhav: I think where the challenge is, and I’d love to hear your thoughts about this, is on one hand you are pushing a last click, last click, a last direct click model that says you guys need to focus on the last direct transaction, the last direct click transaction. But on the other hand, you’ve also got a narrative that it’s very much like.

[00:25:10] Bhav: Well, people might be going in in store and if you’ve got like leadership team who are putting pressure on your marketing team, so the, there’s an affiliate team, there’s a PPC team, there’s an offline team. You know, not all marketing spend is then treated equally because the one that you can directly measure will always get an equal waiting.

[00:25:31] Bhav: So. To the people who, who maybe are a bit more top of the funnel and it’s, it is harder to get them to quantify the impact of their work. Like how do you approach this now? I’m, I’m sure we’re gonna now go into like the depths of like econometric modeling, but you know, like what do you say to those people?

[00:25:48] Bhav: ’cause that’s the first thing. Second thing you said was you are an 80-year-old business, right? Trying to convince people who are at the top, who are probably institutionalized by old school marketing methods of this new age digital technology and this new, new age digital analytics approach. It’s a hard sell.

[00:26:08] Bhav: Right? So what has your experience been trying to close that offline online gap, and then also to try and sell this new way and ensure that people do get the. Credit because the numbers don’t always speak in favor of certain channels or marketing teams. 

[00:26:25] Jorge: Well, I wanna talk the, let’s say. And, go back again to what we were saying, that in this technological data-driven and artificial intelligence world, there’s a fundamental need of human communication, of human alignment on human business making.

[00:26:53] Jorge: You know, because, let, let’s, and, and I have this, use case very fresh because there was, the social commerce team came and they were like, oh, your model is just, bias. It doesn’t look the, the, the whole funnel. And we are making the best campaigns that we are making the best creativity, and we have the best.

[00:27:24] Jorge: In our profiles, we have, conversations with them, et cetera, et cetera, et cetera. And I tell them, okay, you’re right. The model has a narrow view and is just one click. Your strategy will never be outstanding here, don’t we?

[00:27:50] Jorge: Understand the impact of your specific strategy into business. In, in, in, in what channels? are you better performing? What goods are you better selling in? Maybe people want to see digital creators recommending, but not, maybe, maybe not. And we came with these test agenda, geo-based agenda when, for example, we, excluded certain Mexican entities and performed.

[00:28:36] Jorge: UGC campaigns there and the people, the, the social commerce team, were talking to the brand ambassadors and tell them, okay, there’s this technology campaign and you need to give me the videos, the material, the creativity, and I’ll make my paid media campaigns with them. Okay. And so they, it needed to be like much of paperwork that I’m, I’m, I’m, I’m not, I’m not an expert about that paperwork, but I, the only thing I know is that was a ton of paperwork done and we came with that campaign and we could measure with, With the, with the geo leave, library, developed by meta that the, the impact of that certain campaign. And I told the team, okay, this is your, value in sales and you can sell and, and you can sell better in these channels and you can sell better these goods. Okay, so next time. What we are doing now, actually, it’s a bigger test now.

[00:29:56] Jorge: The group of entities is bigger, other categories are, are participating there and let’s see how, how it goes. and stakeholders and, and as you say, like, people in, in boardrooms, they are very, willing to. Because it’s been a process, a cultural process that have lasted four, four years now. You know?

[00:30:24] Jorge: So, and it’s, again, it’s patience. It’s, to be persuasive if that, if, if that’s correct, say and, And clear. Make them easy to understand. 

[00:30:39] Bhav: I have a challenge for you, a pushback here, Jorge. Hope you’ll indulge me. So you use matters geo lift model to understand the impact of the user generated content, UGC campaigns, at a geo level, right?

[00:30:53] Bhav: But then you are relying on matters model to give you that input. We started this entire conversation by saying, ignore what the vendors are telling you. Of course if you are using Meta’s model, it’s gonna try and bias, I don’t dunno about that. I’m not trying to be a, a meta conspiracy theorist. I am just curious as to how do you circumnavigate that in its defense?

[00:31:19] Dan: It is open source. I think 

[00:31:20] Bhav: it is open source. It’s not, it is open source. Yeah, you’re right. It is open. 

[00:31:23] Jorge: I think Bhav works with our stakeholders in I this and I. The answer is that we, when, when Meta first came and tell them like, test with our thing, we said, no thank you. And they had all the time and all, and, and they, they shared a lot of documentation papers with us and we find out that yes.

[00:31:58] Jorge: Free an open source library in R developed by Meta, but it performs a synthetic control statistical experimentation. And if I, and if, if me and my team go to say, okay, I don’t wanna juice geo. I wanna juice geo coppel, like the result is going be the. Behind that. because behind that open source library is just code is just, operations that, as we already said earlier in the conversation, I can tell you we understand.

[00:32:45] Bhav: is this part of the, the Robin MM model that they put No, no, no. 

[00:32:49] Jorge: It’s a, no. It’s, let’s say slightly different. It’s a broader Okay. It’s, it’s, it’s a different, it’s different. And what we like about Geo Live, it’s that we do, you know, what we did the first time we read the papers and we in Excel perform every operations.

[00:33:09] Jorge: Every operation and, and we were step by step. Step by step. Okay. And sometimes we were like, where is this going to take me? Like why? Why are we investing so much time doing Excel that the computer already does? Okay. And I was like, do you know what it’s gonna give to us? Security? Security to go to any stakeholders and tell them your budget?

[00:33:37] Jorge: Your budgets impact is well measured, you know, and, and, and it gives me the confidence to tell you I’m not using, I’m not doing what meta said. I’m using an open source, good or asset.

[00:34:02] Jorge: That result.

[00:34:10] Bhav: I think this big, and again, this circles back 90 to what your second point you made at the start of this call, this, this episode, which was human understanding is more important than ever. And I think you’ve just very elegantly demonstrated how something like, you know, using a, a, a model that’s been built by, you know, one of the biggest companies in the world can still be first.

[00:34:29] Bhav: Sense check before it can be applied. So, and, and my final thing is, the reason why I sound like your stakeholders is because I have spent so to so much time being scrutinized myself that, you get to learn. You learn to understand and preempt the pushback your, your stakeholders are gonna give you. So what’s that saying?

[00:34:47] Bhav: Keep your enemies, keep your friends close, your enemies closer. I sleep with my stakeholders. 

[00:34:53] Dan: Yeah. Yeah. Jorge, we’ve, we’ve, we’ve touched on attribution. We’ve, we’ve touched on, experimentation and incrementality and I, I think it’s only fair if we talk about the last part of this, this, this trifecta, which is the media mix modeling or the econometric modeling side of things.

[00:35:09] Dan: So, you’ve, you’ve mentioned how attribution was that first step of looking, not just at lower value, lower funnel activity, sorry. and then you’ve got the, the kind of controlled experimentation to understand the uplift and the kind of, what if we didn’t. what about joining all this data that can’t be joined?

[00:35:26] Dan: What about the effect of the pandemic? What about the effect of, you know, very timely but like, trade deals, for example, between neighboring nations or other things like this? But there’s a lot of, economic factors that aren’t digital, that aren’t marketing, that can influence what’s going on. and that’s really where the.

[00:35:43] Dan: Econometric modeling or the media mix modeling comes in now. Is that, that’s something you mentioned to me on LinkedIn, that, that, that this kind of like the, the kind of the experimentation and the play in that area of things now, how have you kind of used that and what kind of angle or problem is that solved that the others haven’t?

[00:36:01] Jorge: We, I think it was 2022 or maybe three when one of our stakeholders came and. When are you going to be done with this attribution matter? And we were like, okay, can we answer tomorrow? And we were like, and we were like thinking and like, we need to finish this. You know, this project’s been open for I know, two years now and you’re not done yet. You know? So, and I. I need my strategy and I need it to be it, and I need not you being changing it, you know, because we were like, you know, numbers change and behavior change. So, attritions change, you know, and we were like, okay. 

[00:36:55] Jorge: I think, well, I. And, and, and we said that, and in that conversation and that night, we, we had a very long night that time. And the answer we came back with was, you know what? We are never going to be done with the attribution thing because it’s always changing. The business is not the same. When we first had this problem, like digitally, the website was the main source of income.

[00:37:28] Jorge: And now is the application and how people use a website and a desktop device is way too different than how we use the cell phone, you know? So the attribution has to change. The calibration of the models need to change. So what we understood by then was that.

[00:37:56] Jorge: In an ecosystem of numbers is how we call it. And we have attribution and we have experimentation, and we needed to have like a, a north star to follow. And that’s. We found, our, mm, we, I, we right now do not, operate with the mm m number, but every, few months we go to our MM and ask like, Hey, what do you see now?

[00:38:35] Jorge: Like what, what is changing dramatically? Like, and, and, and it’s when, social phenomenon comes in, it’s when, the pandemic, came in. we have, offline marketing strategies going and us.

[00:39:00] Jorge: You are performing here, offline strategies are performing there. You need to, collaborate, in these two in order to do that and, and, and have again, human understanding about waterfalls that tell us about numbers and ROIs that we are looking in, in the m&m numbers, you know, so we have now attribution models. we have and we working on in order to. What it tells to us now that we are a different business, that we’re an application based business, now we’re not a website anymore. 

[00:39:56] Dan: Hmm. So, so this is, this is interesting. So you go back to B’S point from earlier, an 80-year-old company and changing minds. and ideas is the hardest thing in an established organization such as yours.

[00:40:08] Dan: That’s so big. And so I can now imagine a conversation you would’ve had with your paid search marketing team, and they’ve gone from, Hey, look, look, look at our massive ROI, and then you go, no, no, no, no. That’s just last non-direct click. If you were to look at the whole journey, that’s actually a lot less than you thought.

[00:40:23] Dan: And then you go, actually, when we do this geo lift or this kind of randomized control trial, it’s actually a lot less because some of that would’ve happened. Anyway, and then you’ve gone around and said, no, no, no, actually, let’s take some more away because there’s some economic factors and some market conditions that actually would’ve changed that.

[00:40:39] Dan: Anyway, so. I can imagine. Now, if you take that first conversation or the first position from that paid search marketer to where you are now, that’s a long way from what they were operating at before. Now, how, how much of that realistically has changed their approach, their methodology, and how much of this ’cause what I, what I find fascinating about this conversation, and I love this, this is my area that I love to talk about and prac practice in a, a lot.

[00:41:05] Dan: but there is a, there is a. There is a genuine risk of it being academic, an academic pursuit, rather than a practical implication or practical pursuit. So let’s take that, that example of the search marketers, like how much has the work that’s gone in through these different measurement options, how much of that has really affected their.

[00:41:24] Dan: Their day-to-day or their perceived value in the organization because there’s not just numbers here. These are like jobs and agencies and, and employment statuses based on this. So it’s a lot cha more challenging than just we have a better way of measurement. I’d love to hear from you if you can share around capel, like what has that this, this pursuit changed?

[00:41:42] Jorge: Do you know what really help us? GA4. 

[00:41:45] Dan: This is, sorry. This is the first time I think we’ve had this. This sentiment come up so quickly. It’s helped you. 

[00:41:51] Bhav: I was not expecting that. That was a curve ball if ever I heard one. Come on, 

[00:41:56] Dan: Sorry. We’re interrupting. You crack on 

[00:41:57] Jorge: And you know what? You know why? Because we couldn’t handle it and we were so good. Well. We could make again, excels and go to the 360 dashboard and we knew the number that it’s gonna come up if we look for example, and steals in a certain day. You know, and we were, we were, we, we had our, query documents are SP documents and we knew what the, platform was doing. With our numbers. Then GA4 came and we didn’t know, and we were doing like, okay, so no problem.

[00:42:46] Jorge: We, we went through this, we, we can do it again. You know, maybe, this platform is doing certain different things and we need to find them out. Find them out. We didn’t, no, long story short, we didn’t, so what we raw. This is, these are our attribution models. These are your lifts. Okay? So what we did is like we are sending that data to the team and tell them, okay, this is what are you going to.

[00:43:21] Jorge: Okay. And you know what? Now they are request requesting our data. At the beginning it was, you know what, what? Why are you telling me something different than my platform? How come? But if my platform does this certain attribution model, how come if doing, if you doing the same, how come.

[00:43:50] Jorge: I’m doing this and, and I’m doing this. Step one, step two, step three, step one 50. Okay. And these are the results. Okay? The platform is doing these 150, steps, and these are the platform’s results. Okay? We need to work with our results and with our ways of doing things. Okay. And at the beginning I was having difficulties with the paid search team, and when ga, when GA4 came, I received the call and they were like, I need the data every, every four hours.

[00:44:32] Jorge: So in a regular day, every four hours we send the data. Okay? And when we have like Black Friday or any certain thing, every half an hour, I’m sending the data and they’re looking the data and they’re optimizing their campaigns. With our attribution. Not any. 

[00:44:53] Bhav: I have a funny theory on that one. I wonder if that’s just because they could no longer do it themselves in GA4, because they were always like, oh, we can do this in ua.

[00:45:02] Bhav: We can do this in uae. And then suddenly you, you remove the interface of ua and they’re like, oh, crap, we can’t do this. Jorge, we’re gonna have to trust you now. We don’t wanna trust you. We’re gonna have to trust you. Well, 

[00:45:13] Jorge: That’s why I tell you GA4 killed me a lot. 

[00:45:15] Dan: Yeah, yeah. It’s never made, it made analysts, is more valuable, right? So, so you’ve got this, right? You, you’ve given, you’ve given the data to the marketing team or the product team or whatever, and now essentially you’ve given them three, at least three versions of the results, right? You’ve got your, your geo-based experiments, you’ve got your attribution, numbers, and now you’ve got your media mix models, that are telling different stories.

[00:45:39] Dan: How do you. Where do you start to help them interpret that? It’s easy to say you’ve got three things, go look at that and do better, you know, you know, move the needle differently or do that. How, how, how is that process of understanding that, or do, or do you kind of work to combine them into a single source of truth or a, a single pie chart as it were, you know, looking at that, that breakdown. How does that work? 

[00:46:01] Jorge: Well, the source of truth is the attribution. The last click, that’s the source of truth. The other numbers, as I tell to them, are like tips, like advice, like, directions, like, you know. But at the end of the year we go to the attribution models table and ask who did the cells? That table responds and we, let’s say, that’s our KPIs, you know, if, if that’s and the other, for example, the understand in a bigger, in higher view, you know, or in a general per.

[00:47:02] Jorge: performing between them and how the budget will be allocated. Like if it’s going to be the same, if it’s gonna be different, if the certain strategy needs to double budget, if we need to cut in half, like, you know, that’s the conversation with the MM. Okay. And for, and, and the lift, the question we ask to experimentation is.

[00:47:31] Jorge: Are we attributing well, are we over, over, over attributing or are we under attributing? You know, and it’s like, it’s how we calibrate. If, if, if tests are telling us, you know what this is, this is, the paid, paid.

[00:47:59] Jorge: This, this is the, the impact. And in the attribution model, this is the impact. So I need to cut this. And if it’s, and if it’s the other way around, if the lift is higher than the attribution, percentage, I need to make this better, bigger, you know, and that’s how we work. 

[00:48:22] Dan: I, I love this. I think we could possibly talk about this for an awful long time, and I’m appreciative of your time you’ve given us, and it’s the evening for us in the morning for you.

[00:48:31] Dan: So, before we, we, we draw abruptly to a close with our little wind down questions. Is there anything else, any final words, Jorge, that you’d like to add in, in terms of, you know, this multi-touch attribution, this medium mix modeling or any, anything along those lines? 

[00:48:45] Jorge: Well, I would like to say that for us, we have the most interesting job in the world because there’s a new question every day and there’s a new way to answer all questions every day, you know?

[00:49:01] Jorge: And, our main, A confident firm about the impact of their work on sales, the analytical credibility or the analytical authority that we’re building. it’s, our main pursue in, let’s say, the game, you know, because, we need to have confident stakeholders making decisions in our numbers. And that’s what we have been pursuing for the last four years, and it’s been a very, very interesting, journey.

[00:49:49] Jorge: Now we are. It all started as a group of two people. We are now eight. And, whenever I have, a new people coming in the team and they ask me like, Hey, and, and what do you do? And we are like, okay, do you have time? Like I, I I’ll go, I, I, I’ll tell you, a quick brief about our activities because, it’s, it’s, it’s also difficult to tell like family, friends, like for them to understand what, what, what I do.

[00:50:25] Jorge: Like, I always, I always tell my mom like. I, I just do numbers. I just do numbers, and I think she, she thinks I’m an accountant or something like that because it’s a very, it’s a very difficult thing to, to, to explain, but I, I really, really enjoy it. I, I really, really, really, really enjoy it. 

[00:50:46] Bhav: I think next time someone asks you what you do, just go with something like really abstract. Like, I make dreams come true. Right? And then just, yeah, and then just like disappear off into the sunset. 

[00:50:57] Jorge: Yeah, no, 

[00:50:57] Dan: I think it’s, that would be a good one. I, I, I, I always, I said this before, but I’ll continue saying it. My, my wife explains what I do as a job, as Chandler Bing’s job from friends, a trans sponsor because no one will understand anyway, so I just make up a word.

[00:51:12] Dan: it’s all good. And also that the reality is if you do start to explain what you do. People still don’t care. Like that’s the reality. So it’s only, it’s only really for people like us. And, one, one last plug for super week that I just come back from last week. It’s the one place where you literally get to talk about this stuff in front of everyone, and everyone kind of gets it. It’s a beautiful place to be. 

[00:51:31] Bhav: just to add on to Dan’s sentiment though, it’s not like, like you said, Dan, it’s not so much, what do you do? It’s more, they care more about what can you do for me? Right. So it’s, I think that’s, that’s the, the crux of it. So maybe on that note, Dan, we can move on to the, The quickfire round. 

[00:51:46] Dan: Yeah, absolutely. So Jorge, we have some quickfire questions for you. These are things you have not been shared beforehand and they have been AI generated based on a, semi Okay. Prompt that I do in chat GPT beforehand. we’ve got two questions for you, quickfire, so answer ’em as quickly as you can.

[00:52:01] Dan: We’re not looking for the perfect answer here. but what is the most overrated metric in your opinion, in marketing measurement. 

[00:52:10] Jorge: ROI. 

[00:52:13] Bhav: Okay. I like it. I wasn’t expecting that. 

[00:52:16] Dan: I like it. and, if you could only use one methodology going forward to measure your marketing efforts, which one would you pick?

[00:52:25] Jorge: Wow, that’s a difficult one. No, I, I go, you know what, with a rule waste model. 

[00:52:31] Dan: Nice. I love it. last question, Jorge, what do you do outside of work to wind down when you are not being a marketing economist and attribution person? 

[00:52:41] Jorge: Well, I have a, 15 months old daughter. so, I am, I am a father. And, when I’m not at work, I love like, spending time, with her.

[00:52:56] Jorge: We, we go to the park like nowadays, like my, hobbies are, Let’s say sleeping for, for some, for some time. Like, I need to be there for, for, for her, for my, and for my family, for my wife. In, in, in the free time when, but I love, just hanging around. I think I’m, Sophisticated taste or something like that. Very, very regular stuff. 

[00:53:37] Dan: Awesome. Well, thank you so much for humoring my questions, and for your time today on the, on the episode. the last thing I just wanna ask is there, if people wanna contact you or get in touch or learn more about Cabell or, or what you’ve been talking about, what’s the best way to contact you or find out a bit more about, about you or, or what we’ve been talking about?

[00:53:54] Jorge: I can, I can share with you guys my, my, my email and always, I’m, I’m around in LinkedIn as well. 

[00:54:03] Dan: Okay. Well, I, I’m gonna make a decision there. I’m not gonna put your email out on the internet for other people to find, but I will put your LinkedIn, account and, and then people can contact you that way.

[00:54:12] Dan: And if you want to give them the email address you can. Yeah. Yeah. 

[00:54:15] Bhav: But your angry stakeholders are probably just like, oh, let’s sign him up to some dodgy websites.

[00:54:21] Dan: That’s it for this week. Thank you for listening. We’ll be back soon with another episode of the Measure Pod. You can subscribe on whatever platform you are listening to this on to make sure you never miss an episode. You can also leave us a review if you can on any of these platforms. We’re also over on YouTube if you wanna.

[00:54:35] Dan: See our lovely faces and our lovely guest faces while we do this as well. Make sure to subscribe to the Mesh Lab channel to make sure you never miss an episode as they come out. If you’ll leave us a review, that’ll be hugely appreciated. You can do that on most of the podcast applications or that is a form in the show notes.[00:54:51] Dan: You can leave feedback directly to me and Bhav. Thank you for listening, and we’ll see you on the next one.

• Spotify
• YouTube
• RSS Feed