Fun with Web Metrics

What's going on?

Assessment – Why? (and What?, How, to Whom, etc.)

Commercial web sites have the fortune (pun intended) of having an overall goal: i.e., profit.

So, many of the fundamental questions generally relate to that easy-to-understand goal:

  • Why: Profit!
  • What (do we use for assessment):  Measures of Profit!
  • Who: anyone with cash to turn into Profit!
  • How: Look for signatures of Profit!  (generally, “Did the user come to the checkout?”)

For non-commercial sites, it’s harder to perform this assessment because there isn’t that obvious gauntlet for us to drive visitors down (there’s a reason that the “Gift Shop” is located near the exit).

  • Why becomes the question: is the end result worth the effort to create, produce, and maintain it?
  • What can be used to accomplish this assessment?
  • Who are those informed/helped (i.e., the user population), and Which metrics and methodologies best identify them?
  • How is the site used?   Which behaviors are strategically important?

That’s far less focused.  When I try to diagram the process, I get something like this:

Assessing Web Sites

The process of assessing educational web sites

There’s the obvious process of getting the site “out” in the first place based upon that brilliant idea that started the whole endeavor.  However, as soon as that first launch happens, several other things kick in.

  • You want to get reports on traffic to get sense of “how things are going”;
  • If you rely on funding to keep the site going/growing, you need to establish markers for showing success;
  • You want to market the site to drive traffic;
  • If you’re lucky you still have time and energy to kick back and think about what’s going on – or at least attempt to understand it;
  • You have to prepare for the next release, make changes, fix bugs, deal with issues, and so on.

All these relationships are complicated!

First everything is evolving at once!   Even “Web 1.0” style site builders quickly realized that after launching a site, you couldn’t usually just “walk away” from it patting yourself on the back for a job well done!   There was always post-launch support (typically not budgeted, I might add).

While Web 2.0 sites incorporate content evolution from contributions by the users themselves (e.g., reviews, discussions, media files, etc.), other issues in terms of adaptability to the amount of content, searching through the content, archiving content, and in many situations refereeing content became just as much of a resource commitment (also typically under-budgeted, I might add).

Then, those assessment questions of why, who, what, etc. all seem to also evolve over time!   As new challenges and opportunities appear, as you learn more about your role in the communities you serve, you find that you’re re-assessing your medium- and long-term goals.

That leads you to wonder: Where am I headed?  When do I need to make strategic changes?

As usual, questions tend to lead to more questions.

Marketing people will tell you that it comes down to two fundamental things:

  1. Who are your visitors/users?
  2. What are they doing (and why are they doing it)?

… and they’re right – these two questions apply to all sites, commercial and non-commercial alike.  Answering both questions requires information that isn’t readily available from server logs.   Getting accurate personal data about users is difficult – users are reticent about providing information they feel is too personal unless there’s an obvious application of that information that benefits them directly (giving your address to a site so that the widget you’ve just ordered will actually arrive is an obvious example).   At the same time, trying to model behavior from correlating and comparing thousands (if not millions) of site visits — especially on non-commercial sites where entries, exits, and all of the site meanderings often seem completely random leaves little hope that staring at the ever-mounting log files will ever be manageable, let alone fruitful.

Analytics services (e.g., Google, Omniture, etc.) have services that let you look at specific paths, but don’t do much for comparing them, and when they do they generally rely on searches where you identify specific URL patterns which limits the analysis to only a subset of potentially relevant visits.   They’re also good at giving you some information about the user (e.g., location) that’s generally only used for making maps like stamp collecting (“Oh look – we got our first visit from Liechtenstein!”)…    Trying to make sense of what’s there has been a challenge for me for several years.    I THINK I’m starting to make headway on both questions from concentrating on the following goals:

  1. What can I find out about non-registered users?   In the absence of personal information, are there other data that can provide at least a “fuzzy” picture of who they are?
  2. Is there a way to identify common behaviors from path analysis even if the individual paths aren’t the same and without requiring hitting specific URLs?

Question: are these goals also appropriate/meaningful for your site?    Are there others we should be thinking about?

Advertisements

September 2, 2010 - Posted by | philosophy | , , , ,

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

%d bloggers like this: