The Holistic Web

I’ve been thinking a bit lately about how you work out whether a website is a great site or if it’s underperforming. It’s one of those life the universe and everything questions that is normally left to "experts" or subjective opinions. It got me thinking that if you asked a dozen different web professionals you’d get a dozen different answers, typically skewed towards that person’s specialty. "Innovative design", "the site has to be usable", "great content" are all answers you are likely to hear (probably from designers, usability and content people respectively). They are all components of a great site, but I don’t think any of them by themselves make a great site.

So what conclusion did I reach? A very simple answer with interesting repercussions. A great site is a site which does what it was made to do. Google is a great site because it’s designed to help you find stuff on the web, and it
does. Amazon is (in my opinion) great because it helps you find & buy books easily.

The big question is "Is my site a great site?" To start, you need to define what the site is meant to do. In a lot of cases this isn’t clear, on both big and small sites. Once you’ve worked out what it’s meant to do (and that’s a whole other can of worms), you need to work out if it’s performing. That gets interesting - and tricky. There’s literally a book that could be written on this subject, so please forgive me if it’s a bit shallow in places.

Below I’ve made a list of what I think are broad goals which should encompass the vast majority of sites.

Categories of Goals

A list of goals with a brief definition, some key metrics to work out if you are achieving those goals, and some typical sites falling into that category.

Direct Action: Where your aim is to get a customer to do a specific thing - buy something, sign up to a newsletter, complete a questionnaire, any specific action.  Focus here would be conversion statistics (including abandonment) - how many of your visitors take action, as well as looking at where a customer originated from - search engine, online campaign, offline campaign, etc. Applies to a range of sites, especially commerce.

Indirect Sales: Where people are researching a product, but buying through another channel. This can be the most difficult to measure as there will usually be a large number of elements in play in addition to the website (such as broad reach advertising, etc) which makes it hard to attribute sales as resulting from a
particular marketing vehicle as the sales model is, by definition, disconnected. Typical sites are demand generation type marketing sites (including most fast moving consumer goods sites). I spent 3 months last year tackling this problems on a consulting gig. It’s a complex, interesting, and with enough work and process it can actually be cracked.

Brand Awareness: Measuring brand awareness is a well established field and I’m not aware of internet related activities directly changing this. There’s indirect changes though such as the ability to track people who have viewed banner ads & tracking their subsequent recollection, making it a little more scientific in some cases (with the right technology, which exists) but as far as I’m aware most brand people still focus on offline qualitative, I’m not sure this is the best way but I’m not going into that murky area.

Customer Satisfaction:  This typically uses a traditional mix of quantitative (quant) and qualitative (qual) measures, sometimes online, sometimes offline, usually a combination of both. I personally think there’s a lot of room for growth particularly in the qual area by using technorati and similar services to do "proactive qual" - monitor what people are saying about you, but not telling you. This is relevant to most (but not all) sites.

Customer Service: Using the traditional post-service survey reigns supreme here. Don’t confuse this with customer satisfaction, there is a fine line. Customer service is more or less a subset of customer satisfaction - not all customers deal with customer service (help, support, etc, be it phone or online) but all customers have a
level of satisfaction.

Advertising/affiliate sales revenue: These businesses are often focused primarily on volume of people, and so the traditional page views & unique user metrics apply.

Community development: Usual community/activity related metrics such as membership numbers, number of posts, number of active members, etc can be used. It’s possible to go further and look into number of active threads, number of unanswered threads (to work out if there’s lots of people asking questions but no one answering).

Process automation: things such as online billing, data gathering, profiling, etc. Popular with insurance, banks, utilities & governments. They are more interested in reducing costs by moving numbers of people online, less concerned about the actual value of those transactions than an e-commerce site would be, but the cost per transaction. Online banking would fall into this category.

Summary of goals

Below is a table with the types of goals and the types of measures. Hold your mouse over the measures for more information on them. I’ve put in High, Medium or Low for priority level. Ones with a * have a comment - mouse over.

Type of Goal	Direct Revenue	Indirect revenue	Traffic	Online Quantitative measures	Offline Quantitative measures	Online Qualitative Measures	Offline Qualitative Measures	External referrals (eg search engines)	Cost savings
Indirect Sales		H	L	L		L	L	M	M*
Direct Sales	H		H*	M		H*	L	H
Brand awareness			L		H*		H*
Customer satisfaction				H	L	M	L
Customer service			M	H	L				H
Advertising/affiliate sales revenue	H		M					H
Community			H*
Process automation			M	M					M

That’s a long post, I don’t expect you to digest it in 2 minutes! Please, leave your comments & thoughts.

I should do another post on my thoughts on online qualitative measurement. Short version: I think in general it’s under rated, under used, and people are unimaginative in how they use it and interpret it.

(Thanks to Alex for his help with this post)

As is no doubt appearing on many blogs, I have some invites for Windows Live Messenger Beta (AKA MSN Messenger 8.0). If you want one, please let me know. I’ve been running it for a few weeks. Fundamentally it’s nice, although I think there’s quite a few UI issues to work through. Anyway, 10 invites up for grabs.

[Ok, here’s a little rant]
I have a relatively new sony digital camera. Nothing special (don’t remember the model off hand). Point and click stuff, I’m not photographer, it’s just for taking snapshots of the kids, etc.

You know what drives me nuts? And it’s not just my Sony camera, it seems to be most digital cameras. The interface. I look at the screen at any point in time and there’s about 5 icons on the screen which are no doubt telling me something terribly important, but damned if I know what they are.

Who invents these things? “Oh, let’s put a squiggle with a circle around it to indicate that it’s set in low light mode” or whatever.

But what’s worse, and this is an interesting though to contemplate for user interface people, is that it’s not just a matter of “if they don’t care they’ll ignore it”. That’s true to an extent. But I know that with my camera, I tend to stress out. I think “oh, I don’t think I’ve seen that icon before, what does that mean? Have I set something wrong? Is it in “outdoor” setting but I’m indoors at the moment? Are my colours going to come out all funny?” Maybe I’m a worry wart, but I bet I’m not the only one.

I don’t know the solution, but I bet there is one. Compare a Nokia phone (simplicity to navigate) with a Motorola phone (confusing to the non-expert) and that says to me there is a solution.
[end rant]

We’ll now continue with our regularly scheduled program.