The Holistic Web

John Perry Barlow of the EFF & the Grateful Dead (what an odd mix!) reportedly once said:

The Web sees censorship as damage, and routes around it.

I love it.

Ok, so I’ve been using Google Analytics for a week or two now. First of all, it is the full version of Urchin. I never actually used Urchin, although read a fair bit about it. From what I’ve heard from reliable sources, it’s not only the full version but been improved somewhat.

My first impressions: it’s good. It’s very good. And for the price, it’s absolutely amazing.

Things I like about it:

• It’s free.
• It’s geographic targetting features are nice (although I question their accuracy)
• The filters are very powerful. Took me 30 seconds to exclude my IP address from reports so my own usage of the site (which probably makes 5 - 10% of all traffic) doesn’t skew the results.
• Goal tracking with funnel analysis. Who doesn’t love this feature. Having said that, in practice I’m seeing problems with it under reporting data (I *know* we got the sale, but google doesn’t seem to), but I haven’t investigated thoroughly - maybe the problem is mine. It might be getting confused by the session ID in our URLs (yes, yes, I know, I hate session IDs as well,it’s on the to do list)
• The focus on originating traffic. Distinguishing between referrals, organise keywords and PPC keywords. I love it!
• I could keep going, it has great content analysis features, webmaster stats, ad testing (not played with this but looks awesome) etc

Bad things:

• Maybe not a “problem” as such, but for people used to a simplistic log analysis software such as AWStats, they’ll probably find it confusing
• The lag. Google has been overwhelmed and so the stats are very late to show up. This will no doubt improve with time.
• The filter tables which allow extensive search and replace rules (eg. product 11439 is actually a Small Blue Widget, making the reports more readable) can only be uploaded by google staff, no opportunity for self upload, or even better, a data feed (XML file or whatever). Makes this feature useless for me with rapidly changing data, but I *really* want to use it.
• Without that feature in the point above, it copes badly with query string driven sites

My conclusion: if you have a smallish site, it’s worth putting on, but don’t expect to use or understand all the features.

If you have a medium site, this is awesome. Invest some serious time into getting into it, maybe even get in someone to help (such as me!) and really squeeze the most of it. You could get a really deep, actionable plan on how to improve your site with this set up & analysed properly.

If you have a large site, I’m not sure this would deal with the complexity most large sites have - you’d have so many filters it would become unmanageable. But then you probably have a budget to do something better than this anyway!

So I’m having a debate with a client of mine at the moment. It’s one of those perennial debates on the web. There’s two fundamental schools of thought as I see it.

One, is that you do your best to keep people within your site. This is the Yahoo!/MSN/AOL approach to business. As much as possible, point people within your own walls, and keep them there as long as possible.

Second is that you send people away as quickly as possible. Provide them with what they want and get rid of them. This seems a little counterintuitive at first. However, the argument in favour of this is that if you become known as a valuable resource, you may not get a good length of visit, but you’ll get a great frequency of visit. Google is the text book case here. Like most people, I don’t spend long on any one visit to Google, but go there a dozen times a day.

I’m going for the second. It’s a long term investment in becoming a place to visit. They are pushing the first, let’s get the traffic now. It’s a bit of a stalemate at the moment.

What are your thoughts? Anyone know of any research, case studies or anything else on this debate?

You may remember a little while ago Google bought an analytics company called Urchin. I had a bit of a look at Urchin and was very impressed with the statistics. It went way beyond analysing traffic patterns, etc, to look at goals and profitable traffic patterns. Eg. 100 people clicked on this link on the home page and 50 clicked on that link, but out of those 50 twice as many actually ended up buying something. That sort of super-useful stuff that really helps optimise.

Anyway, Google have just released a free version of Google Analytics. I’m not sure if this is a cut down version of Urchin or the full blown thing, but either way, I’m extremely jaded with most analytics packages out there, and so have just signed up for it on my wife’s e-commerce site (which amongst other things, makes a wonderful test bed for looking at such things).

I’ll let you know how I get on. Initial impressions are good though.

Oh, and for those saying “don’t let Google know too much”, well, frankly, I don’t care how much Google knows. If they can offer me value, then they can have my data.

Ok, so it’s a tease title I sure don’t understand women. But my wife does, frighteningly well. Fortunately she’s just launched an Online Discount Cosmetics outlet ecommerce store. Yeah, we are a totally wired household. Sorry to my international readers, at this point it’s Australia only (although if you are in NZ we can probably work out something).

So, blatant plug: guys, tell your girls to shop. The stuff is brand name, and it really is very cheap. Girls, get shopping! Tell your friends!

And a special “for my blog only offer” my wife has done, guys, if you want to impress your girls with a sensitive present (Christmas soon!), but have no idea what to buy her, send a photo of her to my wife, nicola AT smartpoppy.com.au, and she’ll advise on colours and all that girly stuff and will recommend some stuff that will look good on her - and we will even be offering gift wrapping soon! Remember this stuff is roughly half the price you’ll get in the shops, but you don’t need to tell her that

Blatant plug over, we will now resume to our normal rambling.

(you are more than welcome to google bomb http://www.smartpoppy.com.au with the term “discount cosmetics”, hint hint)

So, Ray Ozzie sent a memo about becoming a services company.

I know Microsoft has transformed themselves before, but this time, I dunno. The quote that jumped out at me:

Only a few years ago I’d have pointed to the Weblog and the Wiki as significant emerging trends; by now they’re mainstream and have moved into the enterprise

The problem is that Microsoft has a lot of traditional people. I guarantee that at least half, probably more like 70 or 80% of Microsoft employees couldn’t tell you what a Wiki is, don’t know what an RSS aggregator is, and probably are only vaguely aware of blogs. Embracing these doesn’t require a new marketing campaign, it requires a whole new way of marketing, a whole new way of thinking about business.

This will be interesting to watch.

Edit: oh, and if you are wondering what “a whole new way of marketing” is about, check out this previous post for a good primer.

There’s an interesting article by Scott Miller at Verster (via Stephan Spencer) about homepage split testing.

He makes some really good points, a few things I hadn’t considered about split testing. There’s one point I’m not sure about though:

But the time will come when you decide to make changes to your text, images, or layout permanent

I guess we get into semantics here about what “split testing” is, but I say, why make it permanent? Why not permanently have split testing on your homepage? Sure, split testing two fundamentally different designs permanently is foolish, but I would argue that testing a few links, a few small sections of your page permanently is a good thing. I am certainly surprised from time to time by what does and doesn’t get traffic, what people click on, and so by doing permanent split testing and especially storing that knowledge well, you can build an impressive body of knowledge over time which allows you to reach a level of optimsation on your site that print publishers can only dream about.

You’ve probably heard the stories of the mechanical turk, a “mechanical” chess machine in the 1700’s that beat many quality players at chess. The secret was of course that there was a chess master hidden inside.

Amazon have come up with an ingenious mechanical turk for the 21st century. Say you have a problem which is hard for computers to solve but easy for humans. An example currently live on their site is a collection of photos of a shop. You need to choose which photo best represents the shop front (including the “none of the above” option). Easy for a human, effectively impossible for a computer. For doing that, you get paid a few cents. Amazon take a 10% cut of that.

But the interesting bit is that anyone can tap into this and take advantage of it. For example, I run a site where people post reviews. Reviews are moderated before going live. I don’t want to editorialise the reviews, I just want to eliminate rubbish, abusive, or very poor english reviews. At the moment the volume is low enough that I just do it. But what if I was getting a few 100 a day? Rather than spending an hour or two a day, I could pass them to the mechanical turk site (via a web service), pay people a few cents to check them out, and then respond that way.

I love this - such an ingenious solution to solve an otherwise difficult problem very cheaply.

The site is http://www.mturk.com, but it’s getting hammered at time of writing and running slow. Nice to see the good people at Amazon innovating.

Update: just saw slashdot beat me to it…

I generally don’t talk specifics about work, especially about previous employers. However, this is an issue which I think many people are curious about and which I have a fairly unique perspective on. A bit of background: I spent about 3 years managing http://www.microsoft.com.au, which involved a lot of interaction with the Microsoft.com team in Redmond, and also spent 2 years working for MSN, half of that as lead developer for 24 of the MSN sites globally. So, I have a good perspective of Microsoft and the web. There is no bad blood between Microsoft and me, if the right opportunity presented itself, I would consider returning, and presumably that is reciprocated as I have received numerous offers since leaving.

Let’s get the rumours out of the way: there is nothing malicious, evil or anti-competitive going on. I have never heard anyone talking about, “sabotaging” non-Microsoft browsers. The rumours about MSN deliberately sabotaging Opera aren’t true, I know that team. Opera was such a tiny percentage of visitors that it simply didn’t make the test matrix and the problem was a bug.

That said, the vast majority of Microsoft sites (including MSN) are poorly coded and have poor standards support. Why is this, especially as Microsoft has in the last week released the latest version of Visual Studio which has “…clean HTML editing, including XHTML 1.1 support”?

I think there’s a few key reasons:

• Education: the argument for supporting web standards isn’t immediately apparent, and moving to web standards isn’t a trivial job. Why do it when there’s so much else to be done? More about this later.
• Background: many of the developers come from an applications development background. Applications development is a different paradigm from web development, and many of the developers don’t have a web mentality that embraces the choice of different browsers. For them, developing on IE is like developing on Windows - it’s a platform choice. For a web developer, the web is the platform, and a browser is a different way to access that platform.
• Education: learning standards based web development is significantly different from “traditional” web coding and has a steep learning curve. Without the skills, both the willingness and ability to execute is lowered.
• Priority: there’s no perceived pressing need. When you have vice presidents, or higher, jumping on your back demanding you get project X out the door yesterday, standards development is secondary.
• Legacy: there is so much back end code which needs to be changed, and so many users of that code all using it in different ways, the task of migrating is frankly daunting. It’s made harder as there is an enormous number of people working on the site (globally, including agencies, it would be well over 1,000) and the site is essentially decentralised - making universal change on the site is like herding cats. While that is a good excuse for a slow migration, it’s not an excuse for no migration.

So, for those reading from Microsoft.com (hi Olivier), why should you bother? The reasons are both technical and political. I’ll start with the technical reasons:

• Improved maintainability: updating sites which are CSS based is a joy. It’s easy to do, quick and cheap.
• Browser support: with a standards based site, the vast majority of browsers will work. With Microsofts focus on developers and IT Pros, who also happen to be the audience most widely embracing alternative browsers (and I’ve seen the stats - firefox usage on Microsoft.com is significant), ignoring these people, or giving them a second rate experience, harms you far more than them.
• Reduced bandwidth: standards based sites tend to be smaller and load & render faster.
• Improved accessibility: frankly, I don’t know how Microsoft.com has escaped the accessibility wagon for so long. I’m not saying the whole site ignores it, but it’s certainly not given the priority that I believe a leading company should give it. Accessibility is easier in standards based sites. With accessibility starting to become a legal issue, change is inevitable.

Some of the political reasons are:

• Catching up: I’d like to say leadership, but it’s too late to be a leader. The least you can do is play catch up on the standards trend.
• Dogfood: Microsoft has a great history of eating its own dogfood. Yet with Visual Studio 2005 and it great standards support out now (and not before time!), and IE7 on the way which allegedly will have good standards support, where is the dogfood? Where are the ASP.NET 2.0 XHTML 1.1 sites?
• Being seen as a player: Microsoft is starting to embrace standards, and is certainly trying to project this image. Why not do it on your own website?

Get someone to plan it. Evangelise it. To help work it into the product plans for each product under development. To advise people on pitfalls and benefits. To develop strategies for global adoption. To make it happen. I lead a team just before I left to make the Microsoft Australia homepage an example of best practice (yes, I know about the 2 validation errors, it is a platform limitation I’m afraid), and to show what can be done (and no, this isn’t a blatant pitch for work). It was a steep learning curve, but one of the things I’m most proud of in my time there. Give yourself the opportunity to take the technical & moral high ground.

I’d love to hear from people, especially standards specialists (which I am not) and Microsoft.com people (which I am also not) about what you think about this. Am I off course? Missing a point? Inaccurate? Too preachy? Comments are enabled - let’s hear what you have to say.