#95 Analytics on form (with Alun Lucas @ Zuko Analytics)

In this week’s episode of The Measure Pod we spoke with Alun Lucas, the Managing Director at Zuko Analytics. We spoke specifically about webforms, and the optimisation tool Zuko has built to help businesses get a better understanding of when, where and why visitors abandon forms. We also discuss the potential impact of AI on forms, and the benefits of using a dedicated tool as opposed to GA4.

Show note links:

• Find Alun over on LinkedIn
• Check out Zuko’s eBook titled “The Big Guide to Form Optimization and Analytics”
• Access the CAUSL analytics tool mentioned
• Share your thoughts and suggestions about The Measure Pod in our Feedback Form.

---

🎥 The podcast is now available in vodcast (video) format! Watch the episode below, or over on YouTube.

---

Let us know what you think and fill out the Feedback Form, or email podcast@measurelab.co.uk to drop Dan, Dara and Bhav a message directly.

Follow Measurelab on LinkedIn and on Twitter/X, and join the CRAP Talks Slack community.

Find out when the next CRAP Talks event is happening on LinkedIn.

Music composed by Confidential – check out their lo-fi beats on Spotify.

Master Google Analytics 4 with Daniel Perry-Reed on the next GA4 Immersion 6-week cohort training course. Early bird, charity and group discounts available!

---

---

Quotes of the Episode:

1. “…What we tend to look at is more a difference in behaviour between cohorts and the you know, the base cohort we look at is the cohort of people who are successfully completing a form and the cohort of people who abandon the form. And if you see a significant difference in behaviour between those two, and I mean statistically significant, then you’ve got an indicator there’s a problem.” – Alun
2. “…I think for me, it would be quite important to be able to view the sub metrics that are being used to make up that quasi-metric, because if you just get that frustration score and you don’t know what’s contributing to it, you could then make the wrong conclusion.” – Dara

---

Transcript

The full transcript is below, or you can view it in a Google Doc.

Intro | Topic | Rapid fire

Intro

[00:00:00] Dara: So we’re joined today by Alun Lucas from Zuko Analytics. So we chat to him about, I’m going to call it niche, a niche area of form analytics or why you might need a specialist form analytics provider rather than just trying to hack your way to a solution using something like our beloved GA4. What did you think guys? What were your kind of key takeaways from the chat with Alun? 

[00:00:37] Bhav: I found it quite interesting. I went into the call with a healthy level of scepticism, like I do with pretty much most topics that we discuss. And I try to use the session to challenge any conceptions I’ve had in my head. And I thinkI did a pretty good job actually of making me see the other side.

[00:00:59] Bhav: Because I think I’ve always been one of the people where I feel like one analytical solution is, you know, is the right way to go about, I think, you know, I’ve always felt like too much starts to create disparate data sources and actually, if you can keep it to one source of truth, you know, obviously do so. There was a moment where I kind of went through this episode and was like, actually, yeah, you know what? I can see use cases of this in quite a number of fields. 

[00:01:26] Bhav: I wouldn’t say it’s for every field, but actually having dedicated form analytics when you don’t have analysts, when you don’t have engineers, I think, you know, that was some key insights for me there. And I think I’m glad I approached as always with my healthy level of scepticism. And Alun, thank you for answering all my questions and I hope you don’t take it too badly. You know, I always try to challenge where I can, I think the last time I was this hard was on David and my preconceived notions of personalisation. 

[00:01:53] Dara: Which we’d set up slightly. We stirred the pot a little bit on that one. 

[00:01:56] Dan: Yeah, I liked that one actually, shake the bag. While you were talking there Bhav, I’ve been really trying to scrape the barrel to find some kind of joke when you’re saying about finding some fields for this to work in when his whole thing is about tracking all form fields. So I think this kind of analytics is for every field of a form. So yeah, that was rubbish, I know. We had a great chat with Alun, listen to the whole thing. There’s links in the show notes to all the stuff, Bhav even managed to plug CAUSL analytics and some of the tools he’s built in there as well.

[00:02:19] Dan: So we’ll put some links in there, which is, which is awesome. Remember to catch us on the CRAP Slack channel. If you want to chat to me, Bhav, Dara, and anyone else in that community and join the growing hustle and bustle over there. We’re now on video as well, so you can see us and talk to Alun and in all full HD glory over on YouTube. And we’ve got some links in the show notes as well for that as well. And yeah, as always see you next week and I hope you enjoy this one. 

[00:02:45] Dan: So today on the show, we’ve got Alun, who is here because I stuck a card up at MeasureCamp London only about a few months ago. And I put a tentative card up on the wall to say, hey look, I do a small analytics podcast, would anyone like to come on and talk? And here you are, Alun, you finally made it on after those many weeks of conversations. 

[00:03:06] Alun: Yeah, it’s great to see some opportunism pays off. 

[00:03:11] Dan: Well, thank you. Well, let’s do a quick introduction Alun. As our listeners may already be aware that we don’t basically because we don’t want to get it wrong. We don’t like introducing our guests that come on the show. So this is your opportunity to introduce who you are and how you found yourself in the world of analytics and talking to people like us.

[00:03:26] Alun: I’m Alun Lucas, I’m managing director of Zuko Analytics. We’re a form analytics tool, which basically gives you data on behaviour on forms, et cetera, so for use in optimization. Now, so I suppose my journey is not a typical analytics one. I worked for 12 years in media marketing agencies, so it was kind of analytics adjacent because you were using that for direct response and various other things, but it wasn’t hardcore analytics and after 12 years in that I got a bit bored and so via an MBA in a short stint at Google doing an MBA project, I ended up in venture capital and from there I moved into tech. 

[00:04:05] Alun: So a couple of startup scale ups before the opportunity to come to Zuko which kind of fit my skill set for marketing optimization, as well as the venture capital piece. So I ended up here in Zuko where I’ve been immersed. So I’m not traditionally a CRO or even an analytics person, but I’ve obviously had to get up to speed very quickly in the sphere, which is where people like yourselves and the sort of stuff you put out has been immensely useful for me. 

[00:04:33] Bhav: Amazing. Well, welcome to the show, Alun. I’m going to start right, you know, right off the bat with why form analytics? It’s such a niche area, I’d love to understand kind of like, you know, what, you know, why pursue this particular path?

Topic

[00:04:45] Alun: Yes well, I suppose, The short answer is someone’s got to do it. But I, if you’re talking about Zuko as a whole, we started almost a decade ago Formisimo is essentially the original name and still is the name of the company. Evolved into Zuko as we did more enterprise stuff and we evolved to products. There’s a long story there, but I won’t get into that. 

[00:05:05] Alun: I suppose the reason that we’ve managed to exist this long is that it is an extremely important part of a marketing funnel. The form is the sharp end of pretty much many lead or sign up processes, and if you include checkouts, virtually all of them. So getting it right is really important. And really there’s been a dearth of specialist tools out there. Now, obviously people do use things like Google Analytics to try and hack together something to get some insight. And for some people that’s good enough, but for certain sectors, you know, financial services being our biggest, it’s super important when you’re asking difficult questions to people to really understand where the points of friction are, where people are struggling with, where they’re dropping off, all that sort of stuff that you can’t necessarily get without a specialist form analytics provider.

[00:05:53] Bhav: Okay and I guess as someone who’s always done form analytics when needed through kind of like the Google Analytics interface, Adobe interfaces, even through sort of like just, just event firing. You know, you mentioned there’s a specific need and apologies I don’t know the platform that well. So would love to understand what is it that you think has been missing in traditional implementations of form analysis like Google Analytics or Adobe that, you know, Zuko bridges. I’m conscious of going too much down the sales path for you here. But I am very keen to understand like what I’m missing.

[00:06:28] Alun: So, okay. So yeah, without getting too salesy, but there’s probably two elements that a specialist package like ourselves gives. First one is time. So obviously, you know, first thing objection we always get from everyone. Oh Google Analytics is free, why should we use it? Well, yeah it’s free, but how much do you developers cost to tag up each and every form element?

[00:06:49] Alun: Zuko obviously has been designed to automatically track, so it’s got two tags and they’ll pull in everything for you. So on that side, it’s a time saving because we’ve invested our development to make that work. The second thing is kind of the depth and flexibility of the analytics that you can get, obviously if you spend some time tagging up in GA, you can get a little bit of interactions where people drop off, which is great because that’s, you know, that’s always your starting point.

[00:07:14] Alun: But if you want to start building in stuff, such as, you know, hesitation time, where people are returning to, behavioural differences between different audience groups, behaviour around failed submissions, all that sort of stuff. While you might be able to get something out of it with a, you know, a generalist package, it’s much better to invest a relatively modest amount of money on a analytics package such as Zuko that will essentially do most of that for you, do the heavy lifting. So you’ve got the report straight away, you don’t have to do it. 

[00:07:46] Dara: Is most of that configured with inside the platform itself. I know it’s kind of one of many solutions that kind of pitch this idea of like reducing the amount of time you need your developers to work on the site. So if you just put the standard Zuko code on the website is the rest of it all customizable through the platform, or is it actually the case that if you wanted to do a much more custom implementation, you would have to kind of custom tag up the website. 

[00:08:13] Alun: No, you shouldn’t need to, so the only time you need to custom tag is if the developers have built a form, that’s not a form, if you know what I mean, if they’re building things with a lot of divs. You just need to add a couple of override tags, again, not too difficult. Also, if the developers have called the form fields, you know, field one, two, three, four, five, X, you just need to spend a bit of time and we’ve got a point and click interface, which is just like a point on your form, click it and just rename it. So there’s a small amount but you know, most standard setups, it’ll start pulling things through immediately. 

[00:08:43] Dan: I’d like to highlight that I wasn’t the first one to mention Google Analytics in this episode, so I just wanted put that in for the record. 

[00:08:50] Alun: We’ve not started on GA4 yet. 

[00:08:53] Dan: Exactly, and the reason I say that is because I wanted to kind of take it there, if that’s all right. So talking GA4 specifically, it’s kind of moving into that world of auto event tracking, wherever possible one of those or two of those events are the form starts and form submits and it has some level of detail. Again, assuming it’s a real form and assuming it can scrape it automatically. But it has, for example, the element that it started in the form submits. 

[00:09:15] Dan: Where I think this is interesting is going to be that kind of differentiation between like a dedicated software or a dedicated package and say, maybe where these existing technologies are moving into, you mentioned something around things like the failed submission analysis and stuff. And I suppose I was just trying to get your perspective of like the future of I suppose this kind of form analytics game when we’ve got these kind of other tools that are kind of like creeping behind in terms of trying to automate and get there slowly but surely. Is there something that you see maybe they could do?

[00:09:46] Dan: Let’s take Google Analytics as an example. But could they implement a bunch of event tags that could automatically capture a lot of this stuff? Or do you see the kind of value is in the kind of the analysis around kind of why the hell did it fail and all this other stuff, this stuff that these tools would never be able to do?

[00:10:02] Alun: Anything’s possible obviously, Google’s a big beast. If it wanted to do something that basically did similar things to Zuko, then it could do it. I think sort of two aspects well, first one with GA4, they’ve obviously got issues around their submissions. He talks about the events, obviously a submission to them is someone clicking the button doesn’t necessarily mean a successful submission, which obviously for an analyst is you know, problematic because you really want to understand not just someone’s click, but whether they succeeded or not. Could they do something? 

[00:10:30] Alun: It’s not so much in the tagging per se, obviously well, let’s assume that they create something that allows auto tagging and all that sort of stuff along the lines of Zuko, it’s okay, it’s setting up the dedicated reports for it. Which again, I’d say anything is possible if Google wanted to do that, but it’s like okay well, let’s look at the reports that show you what happens after a failed submission. What do people do next? What do they jump back to? Which error messages are triggering those sort of metrics that give you the insights?

[00:10:56] Alun: You know, from a semi manual perspective to start with, but you know, some of the things we’re trying to get to is okay how do you cut out the time to insight. So, okay, well, what are your biggest friction points? And some of that is around the failed submissions. So it’s building the intelligence around that. Basically to highlight and identify and potentially quantify those pieces which like I say, anything’s possible for Google, but I think it’s not really their focus at the moment. It’s kind of the way on the analytics side, that’s kind of where we’re looking to move. And it’s where, you know, the biggest challenge probably for any analytics products is, is just cutting out that time to insight. 

[00:11:32] Bhav: I guess my, I don’t want to say pessimism. I guess it’s more curiosity than anything. Like how much of a problem is this? For example, I completely get your point that you made earlier about most marketing leads start with a form submission, even to some extent completing your checkout process is a form submission exercise where users will go through, they enter their personal details, their address details, their payment details, so I get that but once you’ve kind of locked down the core components of what a good form should have, how does this area drive longevity of, you know, for needing it, you know what I mean? Like in my head, optimising a form is a finite problem. 

[00:12:16] Alun: It’s an interesting business dynamic for us. And speaking frankly, we kind of have two kinds of customers. The first one is probably as you describe someone comes in, they probably don’t have a huge amount of you know analysis time or CRO time. And they’ll basically use it, find the biggest issues in the form, fix it, and then go our problem is solved and they’ll move on. The second one is obviously our favourite clients are the ones that stick with you. They’ve probably got a number of forms, they probably got a number of complex forms.

[00:12:43] Alun: So again, I’m using financial services as the key example. They’ll also have a number of regulatory concerns that they have to balance so much stuff internally. So it becomes, okay, this is critical for us as we move through our evolutionary journey. So it’s the classic test, learn, test, learn, test, learn, test, learn. And so they stay, you know, for years with us because it’s never finished because there’s always a change in regulatory or internal, or so they’re always looking to us, should we change the copy?

[00:13:05] Alun: Should we change the messaging? Should we change the validation? All that sort of stuff. So I get your point, for a certain sector of the market, we get them on, they use Zuko for a time and then they move on. But obviously for us, there’s also a big sector, which is again, almost a niche within a niche if you want to go there, that are extremely loyal because forms are their lifeblood if that makes sense.

[00:13:31] Bhav: Yeah, it does. And for the record, I don’t mean specifically Zuko. I mean, the forms, you know, the form optimization, opportunity to optimise as a whole, as opposed to, and by that I mean through either using Zuko or Google Analytics or anything like that. You know, from my point of view, once you’ve, you’ve implemented the tracking on it.

[00:13:50] Bhav: Okay yep, you’ve got it, it’s there. You might even identify there are some areas where that, you know, you’re seeing some, some proper problems. You kind of fix those and, and then you kind of, you know, you move on to the next thing. So this, it was more around, it wasn’t specifically around the longevity of Zuko as a product, I actually meant longevity of like form optimization. To me, there’s a very low ceiling in terms of incremental gains to be had from form optimization. That’s, I think that’s what I was going to, as opposed to specifically talking about Zuko or anything. 

[00:14:19] Alun: Yeah, it depends. I suppose you do essentially, typically any engagement has two stages. One is the fix the broken shit stuff. So, you know, the stuff on there, you see really bad stuff that you can get. And sometimes the you know, you can, you can identify them just by looking at the form and maybe even the customer knows it, but they need the data to back up, to go to internally and say, we need to change this stuff and generate the hypotheses.

[00:14:43] Alun: Yeah and then it is from there, it’s the test and learn and the gradual improvement. And I say that the bigger, the more complex your form, the longer that process will take and the bigger the opportunity. 

[00:14:54] Bhav: Okay yeah I guess for those law firms that makes a lot of sense where you’ve kind of, or maybe not law firms, whenever you’re dealing with sort of like regulatory issues, like financial institutions and things like that. I want to take it up a notch, Dan, but I’m conscious you probably want, there’s something you wanted to say. Do you mind if I take it up a notch or did you want to get your question in? 

[00:15:10] Dan: Well, I feel like my question is a bit up a notch, so you kick it off and I’ll jump in and see if it’s the same.

[00:15:16] Bhav: So you mentioned, Alun, that when you’re, when users start to think about form optimisation, they’re looking at, you know, they’re looking at submission, failed submission metrics, but there’s also some frustration points that there may be. And you, you know, you quantify and measure those through things like time spent on a form or time spent between sections of the form.

[00:15:37] Bhav: I know there are, you know, product analytics platforms who could try to quantify this from a frustration metrics. But, you know, who decides what the optimal time looks like? You know, and I’d love to hear from your point of view and your experience. You know, is that even a metric that would potentially drive down form completion rates?

[00:15:56] Alun: So two questions there. One is about the metric itself. And I suppose there’s a bigger picture: one is like a frustration or a friction metric valuable in itself. I’ll come on to that one in terms of the metrics themselves. I think what we look at is, is not necessarily benchmarking a metric. So how many times people make a correction? How much time people spend completing it, because in and of itself, it may be an indicator of frustration. It may not, you may have a question that naturally people go back to correct. If you’ve got a 1000 word essay for a university, people are going to be going back and making corrections a lot.

[00:16:29] Alun: What we tend to look at is more a difference in behaviour between cohorts and the you know, the base cohort we look at is the cohort of people who are successfully completing a form and the cohort of people who abandon the form. And if you see a significant difference in behaviour between those two, and I mean statistically significant, then you’ve got an indicator there’s a problem.

[00:16:50] Alun: So we often see it. So for example, if you have a phone number field, you might see that people who successfully complete the form go back on average 1.1 times to make a correction. Whereas it’s maybe 1.9 times or even higher for, for people who abandon. So that once you, you look at the numbers, you’re like, okay, so there’s a difference in behaviour between these two people who abandon people who complete and actually okay. This gives you an indicator to start looking at here for your hypotheses for improvement, if that makes sense. It’s not a benchmarking, it’s actually looking at your own cohorts where they differ. That’s critical and that can feed into, I guess what we talked about before a friction score. 

[00:17:33] Alun: Taking the bigger question on friction scores. This is a consistent product conversation we have. For the record, we don’t currently use a quasi analytic metric like friction score, which typically would pull together lots of different metrics and combine in some sort of algorithm to give a score. But should we? You know, when we speak to customers and potential customers, we tend to have two groups.

[00:17:59] Alun: One is the kind of more serious analysts like no smoke and mirrors. Give me the numbers, I’ll work it out myself. And that’s typically the side we’ve gone because they’re our best customers. But then we do have customers just say, oh, what does this all mean? Just tell me where the problems are quickly, I don’t want to have to work it out myself.

[00:18:15] Alun: And while there are, you know, indicators, you know, abandonment points, you know, what happens after failed submission, those sort of things we try to bring together and group. We’ve not created an individual metric. So we’ve always erred on the side of it’s a bit smoke and mirrors, you’re kind of making it up, but it actually, you know, as professional analysts yourselves, it would be good to get your input as to how important you know a quasi analytic metric, like a frustration score would be for you and the, and the community as a whole.

[00:18:49] Dan: I think it really depends for me, at least on the volume of data we’re looking at because someone explained it to me in the, in the concept of someone explained to me a concept once in terms of job applications and reading CVs, right. And I always wondered why for example, there was this job that required a degree of a certain level of a 2:1 and above, for example. And I was just like, but what’s the point? We work with lots of people that don’t have degrees and this 2:1 and above feels kind of arbitrary. And he said, well if you’ve got like 10,000 CVs applying for a job, this is a way to cut those in half. So that you only have 5,000 to read. And then if you put a 2:1 and above that cuts that in half again.

[00:19:23] Dan: And so what you’re doing is you’re kind of taking something quite qualitative, this data set where you have to read and go through them one by one and you’re kind of reducing the sample size so that you can actually tackle it. And I think when I’ve done work with things like hot jar and clarity and all this kind of qualitative data sets, where we’re looking at things like rage clicks and screen recordings and session session recordings, things like that, where you have to literally go through them one by one and sit there and watch them.

[00:19:48] Dan: It’s just a good way of saying, show me all users that have got a rage score of above seven, for example. And it’s just starts you off and it kind of reduces that sample set. So for our website, it’s probably not that hard to watch every session recording that’s consented to be recorded right. But for a lot of people like we work with, that’s just not possible. And just being able to kind of put the line in the sand and say, hey look, we’re going to look at this data set here, or this part of it is the deciding factor for me. 

[00:20:13] Dara: I think for me, it would be quite important to be able to also view the sub metrics that are being used to make up that quasi analytics metric, because if you, if you just get that frustration score and you don’t know what’s contributing to it, you could then make the wrong conclusion. Let’s say one of the, one of the elements going into that is time to completion or something like that for certain types of forms, maybe someone taking longer means they’re going to have less errors and it’s going to have a higher successful submission rate.

[00:20:40] Dara: So you wouldn’t necessarily want that to be skew in your frustration metric. So if you’ve got it as a high level, okay, I’m lazy, I just want to know if my form is working or not shouldn’t say lazy, but you get what I mean, you can look at that high level metric and get a rough idea of whether it’s, you’ve got a problem or not, but without being able to then look at the individual metrics feeding into that, you might not be able to add that nuance context of your own particular website or your own business. 

[00:21:08] Dara: Because you might look at it and say, okay, the reason why the frustration score is so high is actually because it’s thinking that people taking a long time to fill in this form is a bad thing. But in my particular case, I don’t think it’s a bad thing. And I need to look at those other metrics instead, like error, you know, number of errors, failed submission, rage clicks, that kind of thing.

[00:21:26] Bhav: Yeah, and I think from my point of view, I’ve always found this to be a really, it’s an interesting one, because there is an intrinsic motivation for someone to complete the form. And going backwards and forwards on getting the form right or not is, it’s kind of a, it’s in some cases a mood point, right? Because if there is a reason that you have to complete the form.

[00:21:45] Bhav: I think as, and as humans, we will endure a little bit of pain to get that thing done. And if there is a high level of motivation for me to complete it, for example, if I’ve got a government form I have to fill in, I know government forms are quite possibly the worst forms to fill in the world, but they are legal requirement and I’m convinced, sorry, that the government has designed the worst forms possible because they know that I have to complete it. 

[00:22:14] Bhav: And then I wonder if you, if the barrier to completing the form is it’s okay, it’s not too tough provided there is that intrinsic motivation to complete that purchase, make that sale, download that PDF where you might need to complete a form. I always try to think about the marginal gains and if in my, in my opinion, the form isn’t terrible but the marginal gain or the value added to the person who’s filling it in is higher than that bit of pain, I generally consider it as a net positive experience. 

[00:22:45] Alun: I think interestingly you mentioned government forms because we obviously have a lot of forms on our database. So we, you know, we’ve published stuff on our website with conversion averages, et cetera. Interestingly enough, government forms have the highest completion rates, but they also have the highest number of fields on average, because they tend to ask a lot of questions. So, you know, people often say cut down your fields to increase your conversion rates, it’s a bit of a myth, can work obviously, but you know, you can see the intrinsic motivation behind it.

[00:23:16] Alun: That said, just taking your point there, obviously, even in the worst form, generally government forms are perhaps an exception because they’re a monopoly, but generally speaking, there’s usually incentive for the business to make the user experience as smooth as possible. So even if there’s a higher motivation to complete on a high completion rate, you can still improve the form user experience.

[00:23:36] Alun: And if you identify via metrics, either, you know, concrete metrics that we currently do, or, or a quasi metric enables you to improve them, obviously as I still would contend that actually it’s very valuable for you to know the worst points. And from there, there’s also the, there’s almost a segmentation piece as well. 

[00:23:54] Alun: So once, you know, in general, but actually, do you know that on, you know, a particular browser on iPhone, you know, you’ve got a significant drop off on a particular drop down, you go, I didn’t realise that you go in, you test it out, you get your hypotheses that the drop down experience is absolutely appalling. People are dropping off because they can’t find the relevant answer or whatever. So you can use analytics again. And it kind of goes back to your question before who stays with it actually that’s where you dig down and you get these analytics pieces and these little nuggets that you, you know, you wouldn’t get if you were looking at the averages.

[00:24:29] Alun: It’s okay, drill down. And I think one of the things we are obviously always looking at is, okay, again, time to insight. Can you surface those insights? Automate them, that sort of thing. Because I think that’s where, you know, value of a specialist product comes in because we understand the experience more than GA does on forms. And therefore we can surface the insights that drive improvement hypotheses in theory much better. 

[00:24:54] Bhav: Sorry, I know, I know I’m being difficult on this, on this particular topic. I’m trying to basically figure out in my head, is form analytics a modularized component that needs a plugin through some third party, or is it good enough to be adapted through whatever analytics platform you have. That’s kind of like where my line of questioning is coming from. Is that trying to solve that problem? Because I think you’re right, if you haven’t got an analyst in place, if you don’t have engineers who are in place and forms are a big part of your business model, then having something that can do easy implementation, time to insights, you know, down to, you know, as little as possible is going to be critical.

[00:25:39] Bhav: But if you’ve got an analyst in place and you have the tools in place, you’ve got the engineers in place, is it that much more of a time saving. Speaking as an analyst, it would take me all of like five minutes to probably pull together a funnel in the form. I think GA4 now also has time to the next step right Dan? Like the time between steps. 

[00:25:59] Dan: It can do yeah. But let me jump in on this actually, because this is interesting. So I think I started from a similar place, but the more I’ve been thinking about it through this chat with you Alun, it’s like, I suppose you could argue the same for if you had an analyst, you don’t need GA4, you can use like a segment, do your event collection pipeline directly into a database and do it that way. The less dedicated headcount or resource you’ve got for specialty in that kind of side, the more you need to lean on a tool like Google Analytics for marketing analytics or Amplitude for product analytics or you know, insert your type of analytics and tool here. 

[00:26:30] Dan: And I’m thinking it’s exactly the same for this as well. Like I think it was Hotjar that used to do some form funnels. I think they deprecated that feature some time ago now I’m probably dating myself. But the idea there is that it is a plug and play, you can click on the form fields that you can map them out and visualise it. And for those people that don’t have the skills, even if it is, let’s say technically more proficient or, or better, I suppose, from our analysts perspective to do it another way, it’s just not possible. And it’s just not an option to them. So, you know, you either spend the money on the tech or the people right. And I think this is, this is exactly the same for any of those.

[00:27:03] Dan: And also like GA4 could do this maybe, but you’d have to really fucking hack it right? You’d have to really get in there with the GTM. You’d have to really get in there with these products and you’re spending the money one way or another. So I think, I think I’ve come around to, I think, you know, having this kind of modularized data ecosystem for forms. If it’s a big part of your business, then it’s important to have that speciality there, either through a human or through technology right. The same for any of these, sorry, Alun, I jumped in on your thing there. 

[00:27:27] Alun: No, that’s fine. I’m not disagreeing with you. There are times where analysts want to do it themselves and pull together their own stuff. But there’s enough of a market out there that doesn’t have the time or expertise, so therefore they need something that tells them where the issues are, especially if they are form reliant, which many businesses are.

[00:27:50] Dan: Well, some of these analysts think they can do it in five minutes as well, so. 

[00:27:53] Bhav: We can do it in five minutes, provided the event’s already in place. 

[00:27:59] Dan: Plus the three weeks. Yeah, exactly. Alun, one slightly left turn. And I did have a question around this so there’s two things I’ve been kind of itching to ask since since I knew that we were talking to you around form analytics. I’ll save the best for last, but the first one is just in a world where tracking users is just becoming increasingly hard and impossible and latched into things like consent management platforms, and you’re relying on the client to actually know what that means and how to have done it properly. 

[00:28:25] Dan: How is things like form analytics and obviously speaking as a marketing analyst and Bhav from product analytics, the technology and tools we work with approach these slightly differently, but how does the world of form analytics deal with not been able to track as much entire things together, especially when you’re looking at user data, such as time to complete, number of times doing per user. So how does that look in the world of form analytics? 

[00:28:47] Alun: It’s probably pretty similar to most others in terms of it’s a volume to get the insights. So depending on the territory that you’re in and obviously each territory is different. And we wrote an article on this for GDPR recently on the blog, but you know, cookie consent, you know, may reduce your volume. So it’s probably pretty similar for GA that you need to get a certain amount of traffic through to get insights. Obviously the point of Zuko Analytics, it’s more an ideation tool rather than the verification tool. So at that point, you just need to get enough to basically drive a hypothesis.

[00:29:24] Alun: You know, a month’s data might be enough to get you the initial hypothesis that you’ve got a terrible phone number field and you might need to fix it, or you might need to change the validation or whatever it may be. So we typically don’t have as much of an issue with that because the objective is just to come to the hypothesis that’s backed with data, if that makes sense, rather than being a catch all and end all and getting lots more for once you do the test, obviously, you’re going to be much more concerned on the, you know, how much traffic you get and making sure that’s significant. 

[00:29:57] Dan: And is that the same for product? And I suppose web and app then Bhav, from an experimentation perspective, is it the same kind of approach where it’s like volume kind of, even though you’re technically missing or could be potentially missing like 50 percent of your audience, it doesn’t matter because you’re looking at objectives and kind of trends.

[00:30:14] Bhav: Yeah so this is, this is an interesting one because I’m a firm believer that you don’t need tonnes and tonnes and tonnes of traffic to gain insights. So I genuinely believe that if you have enough, like, you know, even, let’s say for example, let’s take the example of the form. If 500 people complete the form and you’re only able to successfully track 250 of them, that 250, the insights you get from those 250, it’s probably going to be a representative sample for that, you know, of your entire population. And, you know, I’m talking about a really small number here.

[00:30:40] Bhav: I imagine for most forms, most companies have forms that need to be filled in. You’re probably going to, you know, easily get into the low thousands you know, at the very lowest from, you know, from a very low perspective. And If you, even if you capture, say, 50 percent of them, 40 percent of them, I’d like to think that the information you get is representative of your entire sample base.

[00:31:07] Bhav: So you know, I’m not one of those people that says, hey, you need to have billions and billions of users going through anything to get meaningful insights. I think if you’ve got 500 people and you can only capture half of them, you know, you’re probably going to get all the insights you need. 

[00:31:22] Bhav: Like, you can do and so on CAUSL analytics, you know that, that thing I’ve been playing around with, there’s a feature on there that does a calculation for you of how many responses you need. So if you know, on average, you’re getting X number of people going through, it will, you can statistically calculate how many people need to complete the action for it to represent your overall population. So if Alun, you know, you’re tracking a thousand people came to your website or 2000 through some internal analytics, but only 500 people completed, that might be all you need to get a representative sample.

[00:31:55] Dan: Okay well, we’ll stick a link to that in the show notes. Thanks, Bhav, I think that’d be really useful, these calculators. So, this is my last question, Alun. And like I said, I wanted to save the best for last, and it’s more of just a kind of what do you reckon is going to happen?

[00:32:01] Dan: But obviously with the, the dominance of generative AI, especially over the last 12 months there’s been lots of chitchat around how we don’t need web forms anymore, right? The future of that could go into a kind of natural language, sort of text chat interface to know how to get the data you need and to process that in the right way. I was just wondering if you see, if you are inclined to agree, or if you see that happening, or if you see that actually the web form is never going to die. 

[00:32:31] Alun: Well you know, I suppose our broader mission is not necessarily about analytics, it’s about making the web less frustrating. And I actually do see the state of forms today and for the past 20 years should not have been what it is. It is what it is because all we’ve done is taken the old paper forms and stuck them online. And it’s taken us 20 years to get to a stage where the forms are generally a lot better than they were.

[00:32:53] Alun: I can see a day where a lot of forms are gone. I see a day tied in with data privacy more than anything else, more so than generative AI, but that might be part of the interface where you just basically say, okay, this form needs to you know, understand, you know, lots of things, my passport number, my driver’s licence number, my date of birth, et cetera.

[00:33:14] Alun: Do you give consent? The issue will be privacy, and who’s controlling that? And I think there’s probably got to be something, probably local rather than even central, because no one’s going to trust Amazon or a government necessarily with that. And I can see in theory, that is the way things should go. Trust issues notwithstanding, and there’s probably an intermediate stage where you say like, like, through generative AI, people will ask stuff, although to be honest, that will basically be a form, someone asking you a form. So I don’t see it being that different, it’d just be a different way of interacting with the form.

[00:33:46] Alun: And I think, I think it’s still a fair bit of time off because as I say, it’s taken us 20 years, we’re still in this because people are used to filling forms that they filled in for, you know, a century. And people are kind of comfortable with it being a bad experience, so it takes a longer time to evolve that. And I think, you know, with AI, it’s still going to be a few years off doing that, but I see ultimately, yes, it will go that way. 

[00:34:13] Dara: There’s a bit of a frustrating halfway house situation at the moment. And This is a question for you Alun, and at the end of this, I might ramble a little bit, but we know when you use like a password manager or when your browser stores, some of your payment information, for example, but not every site handles that brilliantly. 

[00:34:30] Dara: So I love the idea of having this kind of central database where you’ve got all your, you know, your passport number and your expiry date of all your cards, everything else, but that can lead to frustration when a form presents as accepting that, but then when you use it, it either only pulls in half the information or it then has a validation error. I guess that’s another use for you know, a form analytics tool where if you can identify that some of that information is being pre populated, but it’s not doing it correctly, then that could lead to frustration for users is one of those things where if it works well, it’s brilliant, but if it doesn’t work, it’s almost more annoying than not having it in the first place.

[00:35:08] Alun: Yeah, I think there’s two, two things you touch upon there. One is from a form analytics perspective, it’s kind of hypothesis generation. So you can see this frustration around that. The question is why is the frustration? So can you detect issues that it sets up incorrectly. The second one is around actually how you do it better. And I think that’s where you need some sort of, you know, we’ve looked at doing this you know, something I would absolutely love to do if we had the budget. So if some VC wants to give us millions of pounds, maybe we’ll go for it. 

[00:35:37] Alun: But you basically, you act as the central store of information. You have a contract with the retailer or the form filler. They don’t touch the data, which is a brilliant data privacy thing. They basically just, you give them the form technology, they poke in, they do it. It gets pulled through automatically and pre populated. So there’s no, there’s no mix up because they have to explicitly link a passport number to a passport number, you know, credit card number to a credit card number. I say there’s a lot of, you know, work to do before someone gets there and it’ll probably be done by one of the big boys. But I think that would be a brilliant way to do it. 

[00:36:13] Bhav: What I find fascinating about forms is, it’s the opportunity to collect a lot of information. It’s no surprise, I’m a data person, I love data. And when you’ve got, when you’ve got fields that are fairly straightforward to complete and build a funnel out, so, you know, your title, your first name, your surname, blah, blah, blah, you know, all of that, it’s quite straightforward to build that funnel.

[00:36:31] Bhav: I think the thing that fascinates me most is when you have open text fields. And the, sort of like, the growth in unstructured data has been something that’s really caught my eye recently. I don’t know if you guys saw actually, Snowflake reported a 17 percent year over year growth in unstructured data. And I’m wondering, Alun, from your point of view, for anyone who is collecting unstructured data through the film, you know, through the fields of like open text fields, like, how do you best utilise that data?

[00:37:02] Alun: So it depends. If you’re using the form for a fairly standard, you know, tell me what your name is. Tell me a little bit of stuff then, then great. That would probably be interpreted by a customer service representative so it’s not too much of an issue. I think where it’s really useful is where forms have become surveys. So like you say, you’ve got the big instruction stuff and I’m no expert on this, but AI can give you a synopsis of all that information super quick.

I think that’s potentially one of the, the reasons it’s fuelling because you’re now not having to sit there and categorise everything manually, is you can just feed it into a large language model and get it to tell you what the biggest themes are for customer feedback being a classic example. I mean the technology’s already there to do that. So I can see that’s what’s driving it and that’s what will continue to drive it for the unstructured data. 

[00:37:52] Bhav: Okay and yeah, and actually you touched on the second thing I want to talk about was form analysis and form completions from a, from the perspective of a product person right. So I think we’ve covered quite extensively most forms generated are for the purpose of lead generation, capturing information, completing some very static fields that are necessary to get to that next step. But actually one of the most underutilised bits of analysis that product teams can can make use of. And I think the big reason why sometimes it’s maybe it’s not so much underutilised, it’s probably used wrong in most instances, is that it’s using forms to collect qualitative data from your customers. 

[00:38:32] Bhav: And I think I touched on, not on any of the podcasts we’ve covered off guys, but in a talk, and there’s usually a few different points of forms that you could complete when dealing with qualitative data. You have your NPS score, which is quite straightforward. You have your CSAT score, which is, you know, how useful you know, how satisfied are you with your experience during X, Y, and Z? And then there’s the most underutilised one in my experience, which is a CEX one, which is a customer customer experience score sorry, CES scores, customer effort score.

[00:39:03] Bhav: And it’s in those customer efforts schools where people just don’t use this and specifically product teams. So do you have any advice for anyone who is trying to collect qualitative data to understand how their products thing and how they might build a form for capturing as much information as possible from a product perspective. Sorry, that was a very long winded way for me to get to my questions. 

[00:39:24] Alun: So, but do you mean that in terms of capturing feedback or capturing actual user data because you’ve got a form within your product? 

[00:39:33] Bhav: I think feedback more specifically, because I think for most product people, they’re interested in how users are using their product. And I think if you can capture that qualitative feedback, then it’s a little bit easier for you to understand, you know, what people feel, how people feel about. 

[00:39:47] Alun: Well, I think we’re getting more into user research than analytics for my opinion, but from a form perspective, typically I would say it would be the open unstructured piece because in theory you should have analytics on this product anyway, so you should be able to get your quant stuff from that, so just trying to get those nuggets. And I say, I’m no expert on user research, but when I’ve spoken to them, it’s like, if you have five people, you can probably get a theme. And once you’ve got your theme that you can build your hypothesis, you can then quantify afterwards. So it’s keeping it loose and unstructured so big, long text fields. 

Rapid Fire/Outro

[00:40:27] Dara: Okay, Alun, on to our rapid fire questions. So you can be as strict or loose with these. You can keep it to form analytics or you can go a bit wider it’s up to you, if you want to make them a bit more broader analytics data related. So the first one is, what’s the biggest challenge today that you think will be gone in five years time?

[00:40:46] Alun: I think it will be finding the nuggets. We talked about it before. So how, you know, having an analyst to dig down and find the nuggets, it’s automating that so automate the insights. And actually what I think will happen is that the insights will be then generating hypotheses, which will then feed into a broader platform, a testing platform, a CMS to actually make those changes.

[00:41:08] Alun: So I think almost, it’ll be a self fulfilling loop of experimentation based on the analytics and, and for forms actually is in a good position because actually forms are much more structured than say a website in general, you have elements that you can look at every element on the form is there deliberately.

[00:41:25] Alun: So I think you’ll basically see that. And so not that I’m saying CROs will go away, you’ll still need someone to run the machine and sense check and all that sort of stuff. But I think a lot of that will go and it will, will basically become easier to find the insights and easier to implement a test to prove hypotheses.

[00:41:45] Dara: Okay well, life would be very dull without a problem. So assuming that gets sorted out, what will be the biggest problem in five years time? 

[00:41:52] Alun: Well, yeah, I think basically what I think the biggest problem in five years time will be still, we still won’t have got to the stage where we need to be, as I just discussed, we’ll still be going there. So it will be, how do we manage the people involved? How do we get those processes going? And how do we manage things such as, as data privacy, which I don’t think that’s going away. Well it’ll never go away, and I don’t think there’ll ever be a perfect solution for it. 

[00:42:15] Dara: I agree. Okay, what’s one myth that you would really like to bust?

[00:42:19] Alun: One myth I’d like to bust, I kind of touched on it already from a form perspective that just reducing the number of fields does not necessarily improve your conversion rate, it can make it worse or adding fields can make it better. Not exclusively, obviously, if you’re just putting rubbish in there for rubbish’s sake, you’re not going to improve your conversion rate. But it’s not a given rule that many sort of designers like to say. 

[00:42:42] Dara: If you could wave a magic wand and make everybody know one thing, what would that be? 

[00:42:48] Alun: Well, self serving, basically to understand the need for dedicated field level and element by element analytics on a form and the value that that can bring. As I say, very self serving. I would say that one right, but you know. 

[00:43:02] Dara: Okay, and finally what’s your favourite way to wind down outside of work? 

[00:43:05] Alun: Favourite way to wind down? Probably playing board games either online or face to face. Currently, it’s a game called Ark Nova, which is about building a conservation based zoo, which doesn’t sound very thrilling, but it’s quite crunchy and quite interesting.

[00:43:19] Dara: It does to me. 

[00:43:19] Alun: Yeah. It’s a good game. If you like crunchy games, it’s good. 

[00:43:23] Dara: Alun, that’s it. You’re out of the hot seat. Thank you for answering the rapid fire questions and thanks for the chat in general.

[00:43:29] Alun: Thank you very much, it’s been a pleasure.