Description Versus Evaluation in Assessing Content Quality

Colleen Jones (@leenjones), one of my colleagues from the Content Strategy Consortium (#csconsortium) recently published, “Toward Content Quality” on UXMatters.com. In it, she presents her cool checklists to use as heuristics when evaluating content quality. She’s invited feedback on the checklists, and so I am writing partly for that purpose, but also to put forth a complementary technique.

Colleen’s checklists cover the most important aspects of evaluation, but they imply (to me, at least) that a whole lot of background is already understood, such as the users’ needs, the overall strategy of the site, and most importantly, the measures against which one might gauge the success of the content under review. I don’t mean any criticism at all; I think she’s just more generous than I am, presuming someone has given these issues at least a little thought.

I’d like to propose a parallel activity to a heuristic evaluation of content, which I’m going to call a heuristic description of the content. Instead of saying whether the content meets specific success factors (e.g., does the content do this effectively?), infer from the content itself what its goal is and describe it as fully as possible.

By describing the content from its own implications, you can arrive at a complete view, for example, of whether the content is doing one thing really well, or whether it’s trying to be many things to many audiences, and not really serving anyone’s interests very well. Setting up this descriptive context first gives one a clearer lens through which to use the checklists Colleen proposes.

This idea is related closely to my previous post on Content Strategy Divination, but it addresses more directly the content types on the website under review.

Step 1: Identify the site’s content types

To look at the content descriptively, you need first to gather samples of each content type. Content types are the specific kinds of content, whether press releases, recipes, video demonstrations, or product reviews. Having consistent content types unifies your site and helps users grasp the full range of the content on your site.

By the way, if a website’s content types are easy to identify, then you already have an indication of the content’s effectiveness.

Now for each content type, look at your samples…

Step 2: Who needs this content?

You can infer the content’s target audience from many factors. What is the style of language? Is it casual or formal? How much education do you need to understand it? Is it technical or general? Use your intuition: Imagine who finds this kind of content helpful. You can be as extensive as you want. You’re building a profile of the target user from how the content communicates.

Step 3: What can you do with this content?

Content has a purpose and a function. What is this content meant to help you to do, given your understanding of its target audience.  Are you meant to learn something practical? Are you meant to accomplish a specific task? In what situation would you reach for this content? What questions would it answer for you?

Step 4: How do you interact with this content?

Less and less content on the web these days is passive, just sitting there on a web page to be admired. How, then, can you connect with this content? Do you study it? Can you change it? Can you comment on it? Can you download it? Does it do something active with you? Does it help you find something? How does this content invite your participation as a user? Can you sign up to receive updates?

Step 5: What is this content’s life-cycle?

A website should always have some process for creating, publishing, updating, and archiving content. As you’re looking at this content, specifically with the previous steps in mind, how would you describe the longevity of this content? Is it durable for ages and ages, or is it an immediate alert that should disappear almost as soon as it appears? Does it seem like there was a lot of careful composition and redaction of this content? Did another user create and post it? Where does this content come from, and where does it go?

Step 6: How are other content types related to this content type?

Now, look at the context of the content samples. What other content appears around the content you’re examining at the moment? What are the relationships of the content on the page: Are they closely or loosely related? What ability does the user have to change the arrangement of content on a particular page? What does the arrangement of the content communicate?

Step 7: What else do you notice?

Really, this is your opportunity to paint as complete a picture of this content type as you find helpful. What else catches your attention about it? Is there some other dimension that would be useful to describe?

Once you’ve spent time soaking in the content, describing it as it is, you’ll be better prepared, I believe, to evaluate it according to the factors that “Toward Content Quality” recommends. If there really is a well-defined content strategy from the website owner’s point of view, you have a powerful comparison among what the site intends, what the site seems to do, and how those two perspectives stack up against the evaluation heuristics.

Thanks to Colleen for being the champion for content quality!

About: rsgracey

@rsgracey has spent his life moving from one area of interest to another, collecting knowledge, skills, and experience (and TOOLS!) for a wide range of creative and professional fields. If you need someone to help you "think through" any problem of information, communication, and the community, don't hesitate to call him in.

One single comment

  1. Ian Waugh says:

    Just found your blog, excellent work. I’m finding my way in CS too, it hasn’t really emerged in the UK yet but I’m hopefully getting myself ready for when it does!