4 Months of 4Q Forever Changed My Perspective on Analytics

Author: Gab Goldenberg

After spending 4 months using the 4Q survey tool from Avinash and iPerceptions, I have a whole new paradigm on web analytics, and the usefulness of clickstream data. More importantly, I have 4 key lessons to share, as well as my own surprising VOC data for iPerceptions!

1. When you’re buying traffic, and especially traffic that has to convert and is not just bought for branding purposes, be sure to pause your 4Q survey tool! I made this mistake when I was hiring recently, which resulted in lowering my visitor-to-job-application conversion rate. At least one person took the survey to tell me that it was annoying to come for a job application and get that, and another decided that he’d give me his email in the survey and let me get back to him that way.

There’s an exception to this rule though: If you’re buying traffic and your conversion rate really sucks, then 4Q can help explain what the matter is.

When I first published my Internal Link Building post, it wasn’t actually available to download yet. I got a good deal of flack and learned my lesson to only announce things when all my ducks are in a row and ready to be used/downloaded/bought etc. Similarly, if your traffic doesn’t convert, you may as well ask them what the problem is!

2. Answers get repetitive after a while, assuming that your site doesn’t change its purpose. In my case, there’s a number of loyal fans and friends in the industry who’ve built a relationship with my blog and I and appreciate the content. There’s also people who find the survey annoying, but I’ll get to that later.

On the stats side, roughly the same percentage of people each month come by for each given purpose. Those looking to read my blog tally about 35% (when I buy traffic) – 50% (when I don’t buy traffic), a tiny percentage come in looking to buy or contact me (under 5% each), and about 30% visit looking to “research.”

In this respect, the tool goes from great to good to just OK :(! As time goes on, you get marginally fewer and fewer original, useful responses and it takes increasingly more time to find an actionable point that you haven’t yet addressed.

Which leads me to my next point.

3. Regardless of what your site does (blog, ecommerce etc.), some percentage of your visitors are probably looking to contact you. If you only offer a quote request form as the sole means of contact (besides listing an email address or postal mail address), then some of your visitors will leave discontent. I myself rarely contact people whose sites don’t offer a contact form. So the lesson here is twofold: (i) Make sure you have a contact page and (ii) Put a form on it. It may be obvious to you, but it wasn’t to me.

4. People don’t read instructions, especially when there are colourful elements in the same area as their eyes are focused on. Graphic design leads the eye around a page, form or whatever else you show visitors, and this is true of survey forms as well. The number of people who thought that they were supposed to do the survey right away leads me to conclude that the instructions were ignored. This is especially so since 4Q uses a popunder; visitors had to click on it to bring it up – they had to make at least a small effort to do the survey.

5. Bit of a bonus, but mostly for newbies: visitors ignore your main navigation. Therefore, your homepage would do well to offer redundant links to your most valuable pages (e.g. “site-purpose”- / “conversion”-wise) .

For iPerceptions, here’s some feedback to improve your product.

A. The usefulness of 4Q voice-of-customer surveys is very limited by the inability to provide custom answers to survey questions. Knowing that people came to “buy” is interesting … but what did they want to buy? SEO services? PPC Management? Text link ads? My soul? Even more mysterious is the answer “research”.

If you’re going to provide a survey tool, it’s not that hard to let survey creators customize the answers to their questions. Survey Gizmo, which I use for my contact and quote request forms, allows even free users to do that (besides offering a lot of goodies to premium users; more on that soon).

B. Asking for permission to survey people right at the start annoys lots of people; they don’t get that they can do it after they’re done with the site. Like I said, expect users to ignore instructions.

At first I thought it was a technical issue, but after contacting your support, I was told that the popunder works fine across browsers and also told that perhaps I was using an older version of 4Q. I installed the new one and continued to get the same feedback, which leads me to conclude that people ignore instructions, as I’ve said. People get annoyed at being asked about customer satisfaction when they haven’t yet used the site!

And before you dismiss this as no big deal and just the price to pay to get valuable insights from other users, consider that this also annoys people using 4Q to get voice-of-customer data. I had to wade through these useless responses to get through to the good stuff. And I do say wade, cuz they made up a significant percentage of visitors.

Besides, this also distorts your satisfaction percentage numbers downwards.

Therefore, perhaps the popunder should be dropped in favour of asking the question on a visitor’s exit and then show them a popup or dhtml survey.

I shared the post with Avinash beforehand so that he could respond to my criticism if he so wished, and correct any errors I’ve made. Here’s what he said:

“In the context of your experience the post is fine. For websites where things don’t change at all and not too many people use them , it might be productive to use 4Q initially and then maybe periodically. But obviously that would be the case of other types of websites, or for pretty much any business website where things change all the time and as do visitors. I am confident that your readers are intelligent and will understand this nuance.”

If you enjoyed this post on qualitative analytics, get my RSS feed for more of the same!

Tags:

Sidebar Story

Comments

  1. Gabriel, Excellent post! You've raised great points and made a very cogent case at the end about the current build of 4Q, one which is nearly impossible for me to refute. So, I won't bother attempting to refute it! Instead, I'll concur with you and Avinash and say that the efficacy of a continuous 4Q deployment will vary with the site type. Also, visitors' sampling tolerance will vary according to the audience. You're right: people rarely do read instructions fully, and a landing-page survey might be extremely irritating to many visitors. Unfortunately, these same visitors are also likely to react negatively to others modes of interruption, so popping up an invite on departure is also problematic. The reality is that 4Q was conceived as an open, transparent, totally "naked" initiative. It will not grow, mature, or evolve as an entity without the help of conscientious users like yourself, who compel us to get better, more flexible, and more responsive. Thanks again for sharing. Rest assured that there are receptive ears listening to you here. Best, Michael

    Comment by Michael Whitehouse - September 12, 2008 @ 3:40pm
  2. Great stuff man, nice to see you've learned a lot. I never actually took the survey but surprisingly didn't find the pop-up too annoying. I think that's mostly because I know what to expect on your site rather than some random crappy content on a site I've never visited Cheers, Glen

    Comment by Glen Allsopp - September 22, 2008 @ 3:56pm
  3. @Michael - answered you privately a while back but for everyone else's sake, it's maybe worth repeating ... I was really impressed by your fast response and positive approach to the critique. @Glen - Thanks for the kudos bud. Appreciate also the info, which helps me better understand my visitors :).

    Comment by Gabriel Goldenberg - September 22, 2008 @ 5:08pm

Leave a Reply