Everything’s an experiment

Measurelab as an experiment

The founding hypothesis? Two handsome young(ish) bucks can escape the grind of the digital agency world and make a living providing no-nonsense analytics consulting from the rural idyll of Lewes.

Eight years later, the hypothesis has been validated. Measurelab has flourished, but still remains a work-in-progress. The company has grown and evolved, but so has the realm of digital marketing and measurement. The aspirations and expectations of people have also shifted: not least how, where and when they work.

We’re in the business of analytics consulting. This isn’t Coca-Cola or KFC – our original ‘recipe’ for success has to be continually tweaked to account for changing tastes.

Naturally, everyone at Measurelab has their ideas and opinions on how we might adapt the formula to make things better – and who better to trust than the people doing the work? But continually evolving roles, processes and ways of working can be disorienting and anxiety-inducing.

To help us manage organisational change, we borrowed from the world of analytics and introduced our very own pseudo-scientific system: the Experiment Canvas and the Experiment Board. 

The Experiment Canvas

The canvas is simple enough. The template will be familiar to those who have worked in Conversion Rate Optimisation (or studied GCSE chemistry). 

Anyone in the company with an idea they want to bring forward – work or play, big or small, serious or silly – can complete a canvas. Find the Google Slides template here.

They’ll need to provide some relevant Background on what’s happening now, state their (falsifiable) Hypothesis, and propose the Details of what they actually want to do to test it – ideally including any resource or funding requirements. They have to specify a Start & Finish, so the experiment is time-bound, rather than open-ended. 

The Owner is usually the person completing the canvas, but in some cases, the experiment might be adopted by a more senior ‘sponsor’. Let’s park the Results, Conclusion & Recommendation and Next Steps for now. These sections only have to be filled in once the experiment has run its course – typically one to three months. 

The Experiment Board

The canvases are added to a kanban board, which enables us to periodically review new Ideas as a group, moving them into the Backlog if there are no objections (there rarely are) before voting on which should be prioritised and pulled into the Doing column.

Operating transparently gives everyone a say and an opportunity to bring about positive change. The kanban process helps ensure there’s not too much distraction or diversion at any given time – although if something is obvious, urgent or requires minimal effort, we’ll often just let the owner crack on with it.

Experiment owners are expected to oversee the experiment, then report back to the group on their observations and outcomes. They’ll recommend we Discard the idea if it’s failed to deliver as hoped, to Iterate if it’s worthwhile adapting the approach in pursuit of the goal, or to Scale if it’s been a success i.e. formally incorporate into our way of working.

An experimental process

That’s the theory at least. At the risk of getting ‘meta’, the whole thing is still an experiment itself! We’ve not been reviewing the board so regularly since going fully remote. Ironically, we’re not always as disciplined about measuring and reporting on results as we might be. And it has been suggested the requirement to create a canvas may inhibit spontaneous innovation, rather than encourage it. But we’ve seen enough value in the process to continue iterating for now.

We’ve had experiments designed to help us carve out more focus time, tried “work-swap Wednesdays”, trialled a mentoring scheme, and implemented a tool to facilitate more frequent company health checks. Right now, we’re running three experiments to test how we can better support each other as we work as a distributed team.

How do you encourage and manage experimentation in your company or team? Is it top-down or bottom-up? Do you have a formal process or do you just wing it?