How Does a Product Get a Five-Star Rating?

Last week (Oct 3, 2011) I spoke on a panel on the evaluation of interactive media at the Fred Rogers Center and I referred to this page, from the September 2011 issue of Children's Technology Review.

I issued a strong disclaimer that every theory can find a champion in technology -- in other words, one person's view of quality can (and should) differ from another person's view. As Jesse Schell (of CMU) reminded the group, measuring quality in an interactive product is like trying to assess beauty.

So, because interactive media is so diverse, and no single checklist, rubric (or as David Kleeman says a "rubrics-cube") can capture all the possible attributes of quality, here are some common general attributes of products that our particular instrument attempts to capture. Most of these are similar to teacher/child interactions that might be characterized as “quality” or “educational.”

A five star product may be one that:

• increases feelings of child control. A child feels empowered, in control, or able to make things happen from the experience, rather than depowered, slowed down, or hindered by the experience.
• provides success in the first few seconds. We use a filter we call the MUC (Miniumum User Competency) to help us gauge the prerequisite skills that are required in order to succeed with an experience.
• doesn’t trap children. Children can always get out of what they get into.
• looks good; sounds good. Today’s hardware makes it possible to deliver “retina display” graphics, and surround sound audio.
• innovative. We see dozens of games of concentration. We like to reward products that are the first to tap the potential of a new technology.
• well leveled. As the child moves through the experience, they don’t run into “hey mom” bottlenecks.
• state-of-the-art. Makes use of the latest tools and techniques to reach the widest number of children.
• is affordable. Compared to competitive products, the product delivers.

What attributes might earn lower 1, 2, or 3 star ratings?

• talks too much.
• sluggish, laggy, or less than crisp.
• talks down to children, feels “sugary.”
• not well developmentally matched.
• “homemade” feeling art, narration and/or music.
• buggy, crashes, buffer problems.
• typos, bad grammar and/or sloppy craftsmanship.
• doesn’t use the potential of the hardware and software.
• competitive products cost less and do more. The free market can be a cruel place, but a 4 year old doesn’t care

Learn more about the rating method

How not to make a children's eBook: Notes from the 2013 Jurors of the BolognaRagazzi Digital Prize.