Carlos and I were discussing qualitative data last week.  Specifically using qualitative data to determine where you rate amongst your vertical.  My opinion is this: I love and trust data, I do not love and trust polling thus I am quite skeptical of a good portion of qualitative data.  Yes there are many different types of qualitative companies where you can get good info to back up your data.   I just particularly have beef with the companies that use polling answers to give you qualitative feedback.  Not that I don’t trust the polling company (ok maybe there is a little distrust), but I feel that there are several inherent flaws in obtaining a truly good understanding of where you rank.

My first issue is: the pollee often doesn’t really have a real answer so they make something up.  For instance, if there was a question of “What are the top 5 apparel stores that you shop?”  The pollee could answer this many different ways.   She could answer with the last 5 stores that she shopped at, or she could answer with the 5 best stores that she sometimes shops at, or etc.

The second issue with polling is locale.  I don’t trust that many polling companies can really collect enough information for a true representation of a nationwide or even less so a world wide audience.  As well, what if you are predominantly a West Coast Company, or an East Coast Company, but your vertical is mostly the opposite? This can probably be adjusted for, so maybe this isn’t a real problem, but you get my point.

The final issue is demographics. Unless you pay your qualitative company to poll your specific type of customer demographic, you will not truly have a good understanding of what your type of customer is really doing.  Unfortunately there’s also a huge conflict of interest with paying your qualitative company to poll people (because you want them to be completely unbiased, right?).

Regardless, the polling company takes any answer from anywhere from any demographic as hard truth and uses it as feedback about what the top shopped sites are in your vertical (or whatever qualitative data that you are trying to find out, departures, conversion, etc).  This to me is just way too loose of a way to determine where your site ranks amongst the competition.

The definition of qualitative data is this:  ‘Soft’ data that approximates but does not measure the attributes, characteristics, properties, etc., of a thing or phenomenon. Qualitative data describes whereas quantitative data defines. (According to www.businessdictionary.com)  Still, I don’t think that qualitative data should just be about how some random person feels at a specific point in time.

I know that there is some information being collected from Social Media platforms such as Facebook and Twitter.   In fact, I see more and more paid advertising from companies that I either talk about, or am a fan of, or etc showing up on my Facebook page.  So clearly these social media forums are collecting this type of data and utilizing it.  Maybe Social Media will be the new way to get qualitative data about your customers preferences.

Please don’t misunderstand me.  I love qualitative data and think that it is valuable information when collected and distributed properly.  The qualitative data that I am talking about is more of the indirect, site wide data and not so much the opinion/feedback data that you should also be collecting and analyzing for your site.  Then again, there is a lingering question that I often of “Do we cater to one person’s feedback or do we stick with our gut and keep our site as is?”  So even this type of direct qualitative data can be questioned.

What are your thoughts on qualitative data?