The panel book has been a staple of the online research industry for years. Its purpose has always been to show researchers how representative a research panel is and whether it can meet their sample requirements.
But as the digital world evolved to embrace the multi-screen, multi-tasking age, unfortunately the panel book didn’t.
Many of these cumbersome PDFs are doing little more than clogging up your office print queue and giving you something to rest your coffee cup on.
Why? Because the traditional panel book approach among many providers is to display top-line numbers based on the entire registered user base, not active panelists – the people actually doing the surveys.
Scratch a little deeper and the problem could be worse: this much smaller pool of people a panel is truly working from may be largely made up of serial survey junkies. These tend to be experts at gaming the system and frequent respondents who are potentially prone to inherent bias, an issue highlighted in last April's GRIT report.
Are these the people you really want to base your insights on?
The unspoken problem of research panel books
If you’re looking solely to a panel book to help inform your decision about your next online research panel project, you’re likely looking in the wrong place.
There are some great research companies out there taking positive steps to improve sample representativeness and drive innovation in the industry.
But the issues with panel books highlighted above represent a wider problem – that many panels in their current form will not be around in five years’ time. Indeed many of the norms of online research will be a thing of the past. Panels sending five emails a day to a membership base inviting them to complete a half-hour survey full of grid questions just isn't going to cut it with respondents today.
Can you really trust the data coming from a survey that you would never do yourself?
A respondent experience to attract real people
Online research panels should be investing heavily in the respondent experience to stay relevant and attract the kinds of audiences that provide the authentic insights today’s brands demand. Consumers in 2016 are time-poor and expect to access to their favourite online sites whenever, wherever.
Deloitte’s 2015 Media Consumer Survey backs this up. Smartphones are the most valued device for 58 per cent of Australians, with penetration standing at 81 per cent, while laptop and desktop ownership is on the decline.
Further to this, when it comes to those traditionally hard-to-reach research demographics like males aged 18-24, smartphone use is even higher.
Choosing an online research panel provider: a new approach
Unfortunately your standard research panel book won’t tell you what a provider is doing to attract, engage and retain these real consumers.
Sure, they’ll tell you their numbers. We’re in a quant game, after all – it’s what we’re good at.
But for panel books to be truly useful, we need to apply a bit of qual to our approach. That’s why, before you dive into a panel book’s stats, we suggest there are five key questions you should ask your research provider to determine how representative it really is.
We’ve used these questions as the basis for our new panel book. By addressing them upfront – as well as giving you our panel stats – we want to show you not just the demographics, but give you a deeper insight into our respondents.
And as for a PDF? We’ll give you a link instead. It’ll look better on your mobile (and it won’t clog your print queue).