Conventional Wisdom Frequently Isn’t

“Sticky is good!”

“Bounces are bad!”

“People won’t register!”

“Google is enough!”

I’ve heard each of these pronouncements as if they were carved on stone tablets referring to how we should approach our web sites, their policies, their assessment, and their design.   Over time, I’ve found that many of them don’t really fit reality:  in my own web usage, my goals and the bar I set for a “successful visit” to a site are NOT mapped by the time I spend on a site, whether I have to register for it, or whether the task at hand can be completed in one page view.  So where did these “rules” come from, and why do we still stick to them?

I think that most of the dissonance comes from the underlying differences between the commercial and non-commercial web.

All roads lead to the checkout!

With commercial web sites, you WANT people to spend time window shopping (and filling your shopping cart), you WANT multiple page hits because you want to expose your users to as much of your product line as possible, even if what you’re showing them isn’t exactly what they sought when they came to your site (your supermarket does the same thing – I’m a sucker for end-cap specials…).  You WANT registration in order to boost customer loyalty (and to collect as much user metadata as possible for marketing!).   Much of the CW silently assumes that the operational model of the web site is similar to the model of walking into a brick-and-mortar store:  keep you there, expose you to additional potential sales, encourage you to re-visit, etc.     All roads lead to the checkout line:

Non-commercial sites are "cloudier".

In the non-commercial setting this is fundamentally hazier.   While you’d LIKE people to spend lots of time hitting all of the pages on your site, and you might really like to have them register (to collect as much user metadata for fundraising!) those conditions only represent a fraction of your total traffic that your USERS

see as worth their time.   For example, if you have weather forecast site, you might let people browse all over the planet to compare the weather all over the globe with historical information, media, etc.   But – I’m willing to bet – that MOST of your users come there to get a very QUICK check on whether they need to remember to take an umbrella when they leave the house and they VERY much like the quick access to that information.

This behavior of making shortcuts to places inside sites (even if only one layer “down”) also goes against one of our other “unspoken rules” – namely that the site HOME PAGE is the big front door that everyone goes through – so much so that we spend a LOT of time designing it separately from all those (less important!) “inner pages” that all generally get the same look-and-feel.  But – when you’re looking for something on the WWW through your favorite search engine, how often is the page you decide might have the information you seek on your digital quest an INTERNAL page versus the site’s “home” page?     Our “portal” UI design comes from wrapping a big front porch on a site that better resembles a sponge or Swiss cheese metaphorically.

But fundamentally, a successful site visit doesn’t require hitting any particular page or process.  It might not involve your home page.   A search engine result might lead a visitor to EXACTLY the right page buried deep in the site, they visit it, obtain the information they were seeking, and leave.

I’ve seen some brave site designers try to re-work their sites “inside out” with the idea that many visits start on the INSIDE of the site and are attempting to alter the UI such that you have a better sense of “how to get around” from the inside that doesn’t rely on a “top-down” hierarchy of pages.   It’s an interesting idea!

So, looking at metrics such as “bounce rate” and “time on site” – what percentage of the visits used to calculate both of those stats are actually “successes in disguise” even if their actual values defy “conventional wisdom”?     What specific parts of our sites attract one-hit or short-visit successes?   Can that content be rearranged in some way that better showcases our success (and drives usage, loyalty, etc.)?    If there’s particular content buried in the site that can be valuable to visitors, you can then promote that information (e.g., search engine ranking) and generate more traffic.   Depending on the circumstances, it might also motivate you to reconsider site navigation (or at least give you ideas for the next redesign of the site).

Revelation #2

I am a “typical user”.

OK – I might have more experience than some, and understand why and how things are done the way they are.   But my reasons for going to a site (say Amazon, or Google, or Facebook) are pretty much the same as everyone else’s.

Revelation #1

If there were ONE “magic metrics analyst superpower” I’d like to possess, it’d be to be able to look through the net and watch visitors to my sites.   No – there’s no nefarious purpose in that ☺ but it would provide me with the most up-to-the-moment honest “ground truth” assessment I could ever want.    Is the visitor interested (“Yay – this is what I was looking for!”)?  Confused (“This site MIGHT have what I’m looking for, if I could only FIND IT!”)?  Upset (“Why is this taking so long to load?  I don’t WANT to look at a 60-second Flash animation!”)?   Overloaded (“There WAY too much here all at once!”)?   Relieved (“This is the 34th site I’ve gone to – and I FINALLY found the right place!”)?

If we COULD read the thought balloons of visitors to our sites, we’d know:

• whether the visit is a “success” and to what degree;
• how easy it REALLY is to find things on the site;
• how well the navigation and user interface works;
• an opinion of our site design and “look”.

In short – we’d have great assessment information.

Sure, you can get some of this information in other ways:

• surveys
• focus groups
• reviews

and so on, but as we’re all aware, those come from only a very small sample of the user base (generally the people who like to fill our surveys and do focus groups), and most of the “quicky” ways to solicit feedback (e.g., “star” ratings, etc.) don’t give much detail as much as a very shallow reaction.

So lately, when I surf the Web for my own purposes, I find myself monitoring my own thought balloons (and sometimes muttering their contents)…   (Even when writing this post, I had to exclaim “OK where is the ‘Preview’ button?” – a genuine review of WordPress’s UI…).

This led me to an experiment where I went back to sites I’ve worked on – especially those that launched some time ago – to discover what “thought balloons” came up as I revisited them.   (“Why is the search box there?”   “When did all that stuff end up below the fold?”) — hmmm.   I wonder what a completely new visitor is thinking when they look at my sites, though I fear that most of their thought balloons would be frighteningly candid about the site’s shortcomings!

If you re-visit your own sites what are your thought balloons? What do you think that a first-time visitor is thinking?

Try it!

Assessment – Why? (and What?, How, to Whom, etc.)

Commercial web sites have the fortune (pun intended) of having an overall goal: i.e., profit.

So, many of the fundamental questions generally relate to that easy-to-understand goal:

• Why: Profit!
• What (do we use for assessment):  Measures of Profit!
• Who: anyone with cash to turn into Profit!
• How: Look for signatures of Profit!  (generally, “Did the user come to the checkout?”)

For non-commercial sites, it’s harder to perform this assessment because there isn’t that obvious gauntlet for us to drive visitors down (there’s a reason that the “Gift Shop” is located near the exit).

• Why becomes the question: is the end result worth the effort to create, produce, and maintain it?
• What can be used to accomplish this assessment?
• Who are those informed/helped (i.e., the user population), and Which metrics and methodologies best identify them?
• How is the site used?   Which behaviors are strategically important?

That’s far less focused.  When I try to diagram the process, I get something like this:

The process of assessing educational web sites

There’s the obvious process of getting the site “out” in the first place based upon that brilliant idea that started the whole endeavor.  However, as soon as that first launch happens, several other things kick in.

• You want to get reports on traffic to get sense of “how things are going”;
• If you rely on funding to keep the site going/growing, you need to establish markers for showing success;
• You want to market the site to drive traffic;
• If you’re lucky you still have time and energy to kick back and think about what’s going on – or at least attempt to understand it;
• You have to prepare for the next release, make changes, fix bugs, deal with issues, and so on.

All these relationships are complicated!

First everything is evolving at once!   Even “Web 1.0” style site builders quickly realized that after launching a site, you couldn’t usually just “walk away” from it patting yourself on the back for a job well done!   There was always post-launch support (typically not budgeted, I might add).

While Web 2.0 sites incorporate content evolution from contributions by the users themselves (e.g., reviews, discussions, media files, etc.), other issues in terms of adaptability to the amount of content, searching through the content, archiving content, and in many situations refereeing content became just as much of a resource commitment (also typically under-budgeted, I might add).

Then, those assessment questions of why, who, what, etc. all seem to also evolve over time!   As new challenges and opportunities appear, as you learn more about your role in the communities you serve, you find that you’re re-assessing your medium- and long-term goals.

That leads you to wonder: Where am I headed?  When do I need to make strategic changes?

As usual, questions tend to lead to more questions.

Marketing people will tell you that it comes down to two fundamental things:

1. Who are your visitors/users?
2. What are they doing (and why are they doing it)?

… and they’re right – these two questions apply to all sites, commercial and non-commercial alike.  Answering both questions requires information that isn’t readily available from server logs.   Getting accurate personal data about users is difficult – users are reticent about providing information they feel is too personal unless there’s an obvious application of that information that benefits them directly (giving your address to a site so that the widget you’ve just ordered will actually arrive is an obvious example).   At the same time, trying to model behavior from correlating and comparing thousands (if not millions) of site visits — especially on non-commercial sites where entries, exits, and all of the site meanderings often seem completely random leaves little hope that staring at the ever-mounting log files will ever be manageable, let alone fruitful.

Analytics services (e.g., Google, Omniture, etc.) have services that let you look at specific paths, but don’t do much for comparing them, and when they do they generally rely on searches where you identify specific URL patterns which limits the analysis to only a subset of potentially relevant visits.   They’re also good at giving you some information about the user (e.g., location) that’s generally only used for making maps like stamp collecting (“Oh look – we got our first visit from Liechtenstein!”)…    Trying to make sense of what’s there has been a challenge for me for several years.    I THINK I’m starting to make headway on both questions from concentrating on the following goals:

1. What can I find out about non-registered users?   In the absence of personal information, are there other data that can provide at least a “fuzzy” picture of who they are?
2. Is there a way to identify common behaviors from path analysis even if the individual paths aren’t the same and without requiring hitting specific URLs?

Question: are these goals also appropriate/meaningful for your site?    Are there others we should be thinking about?

Where to Begin? Metrics 101

Let’s go over some of the basic “units of measurement” that are related to web metrics.

1. “Hit”

I’m going to use the slightly-fluffy definition that a “hit” is any file on your site that gets requested by a browser.   It does not mean a web page – in fact it’s all the pieces that go into a web page: the HTML code, the images and logos and other design elements, the style files, and other program files (e.g., Javascript).    In fact, rendering a web page might actually involved dozens of server “hits”.

This gets slightly muddied in the context of web metrics because invariably someone will ask you questions like “So, how many hits does your site get?” when they actually mean “page views”.

Another way to think of hits is “anything that creates a new row in your server log files can be considered a ‘hit’.”   So, if you’re relying on your log files to look at page views, etc., you’ll have to filter out those lines that are really page elements.

2. Page

Simply, let’s take this to mean “what the user sees in the browser.”    It can be a static page, or dynamically created as part of an online service (e.g. a search results page).

3. Visit

The sequence of pages displayed to a user/visitor from the same site (which doesn’t necessarily mean the same server).   There’s no absolute definition about when a visit ends: for example, if someone is looking at your site, gets up to do something, and returns hours later, is it one visit or two?   One of the conventional practices is to treat a gap between page views of >30 minutes as the boundary between one visit and the next, even if by the same user.

4. Visitor and User

The person (or persons) who are experiencing your site through a browser.   Generally they are identified in the server log by hits within the same visit from the same browser (identified by IP).   Depending on the nature of your site, you might want to make a distinction between “visitor” and “user” if you have registered users.

Now some of you have already poked holes in EACH of these definitions:  What about non-human “users?”   What about IPs that are gateways for many computers? and so on.   One of the fundamental truisms about dealing with web metrics analysis is that the data are extremely “fuzzy”.   Sometimes it’s possible to get a sense of when atypical behavior is present.    Sometimes you have to accept the uncertainty and hope that a useful picture becomes clear over time.

Generally Speaking…

Every web site that’s out there started out as an idea that took the form of “wouldn’t if be cool if…?” and with the overall belief “if we build it, they will come…”

The point is that from the very beginning there was ALWAYS some set of expectations of what visitors were supposed to “do” at the site and another set of expectations as to what they were going to “get” from the site.

For commercial situations, it’s straightforward: there’s a product, a sale, and (step 3) profit!   No matter how huge or how complicated your virtual shop is, there’s that bottom line you can look at and get a sense of whether the expense of creating and maintaining your commercial site is meeting needs (and hopefully, expectations).   So “success” can fundamentally be determined from the answer to the question “Did we make (enough) money?”

In non-commercial situations there are still goals and expectations, but the fundamental criteria for demonstrating success becomes more murky and generally harder to detect.  I’ve done a lot of work with web sites whose primary focus is education.   If I were to try to reduce the goals for most of those sites to a single overly-broad question to illustrate success, it’d be “Do people know more after coming to the site than they did before they visited?” That’s harder to ascertain.

Hello world!

Why am I here?

I’ve been working with web metrics in some form or another for 16 years.   At first it was more or less like stamp collecting, trying to identify the location of my sites’ visitors on the map.

As time went on and things got serious, it became a part of my career.  Over that period I’ve had time to think about the topic, come up with some tools, and have formed opinions on items of interest.   Ideally, I’m hoping to meet other people with similar interests and have some interesting discussions.

Why are YOU here?

I hope it’s because you’ve got some DATA, and you’ve realized that you have to put it in SOME kind of context so that it MEANS something.   Maybe you’re looking at Google Analytics and have discovered that your site had 35,000 visits last month.   What does that mean – really?   Is that good?   How can that figure be used to inform you about future decisions?

What if you’re NOT a big company that hires people to think of these things for you?   What if your sites aren’t commercial in nature (for example, education-based) so many of the typical analysis models aren’t particularly relevant to the environment you’re in?   Hopefully, there’ll be something here to help you along.