3 Tips to Get More from E-Learning Course Reviews

By Ethan Edwards, Chief Instructional Strategist / @ethanedwards

ethan

Recently I was speaking in New Orleans and had a little bit of time to kill before heading to the airport for my flight home and was searching for a place of interest to occupy my time. I spotted the New Orleans Museum of Art online and a few clicks took me to the TripAdvisor reviews.

I was curious to get some idea of the collection before heading there. Here’s a sample of the kind of comments that the reviewers thought were relevant for me to assess the value of a visit to that museum:

I must be prepared for a loud floor buffing machine.
It isn’t as good as "the art museum in New York" (presumably referring to the Metropolitan Museum of Art, one of the finest art museums in the world).
I must start at the cafe, which has a reasonably priced selection of light meals.
I should maybe consider a jazz Sunday brunch instead.
 I may find that some things might not be to my taste.
If I drive and park my car, I should be prepared for a "smash and grab" resulting in the loss of articles worth $800 from the back seat.
I can expect air conditioning.
City buses may have an odor.
It's a great place for photos. 
If I don't want to spend time in a museum, I might want to go somewhere else.
Security guards may look at me.
I will be able to choose between salads and entrees in the café.
If I arrive after the posted hours of operation, I should expect not to be able to get in (this can be very upsetting).
If I make a selection in the gift shop, the clerks will be really busy talking about an upcoming trip to Spain, so I might want to just buy the item on Amazon.com
And oh yeah, there are some "nice Monets.”

I find the comments sort of humorous, actually, but I’m struck by the fact that virtually all of the reviews had little to do with art, and everything to do with trivial elements that really told me nothing about the quality of the museum.

But in pondering the reviews, the situation seemed all too familiar. How often have you sought reviews of your e-learning modules only to be inundated with comments like these?

The punctuation at the end of the bullet points wasn’t consistent.
I couldn’t log on.
I don’t like the color of the buttons.
Has legal department approved this wording?
I like ILT better.
The display of the company logo is not consistent with corporate web standards.
I didn’t see it all because I couldn’t just page through it. Can you give me a printout of the content?
My network seems slow.
It didn’t seem as compelling as what I see my kids playing.

Just like the comments about the museum, these comments really have nothing to do with the instructional experience we are trying to create.

(But I need to be clear, I am in no way suggesting these things are necessarily irrelevant. These comments often can contribute to the overall impression of quality that your work represents. For the most part, they have very little to do with actual instruction. I’ll go out on a limb and propose that no ineffective e-learning lesson was suddenly made effective simply by making the punctuation consistent.)

Unfortunately, this is all a result of asking for input too late in the design and development process. Reviewers are not being uncooperative or unhelpful; it’s just that human nature urges us to comment on that which we think we can have some impact on. When a lesson is all but finalized, it seems pointless to suggest changes to the underlying strategy that probably can’t be changed. So, oftentimes comments end up veering toward the irrelevant. It almost makes one want to just skip the review process altogether. However, there’s absolutely no way to improve our e-learning designs without seeking input from others who were not directly involved in their creation.

So what’s the answer? Here are three tips to tackle the review process of e-learning courses:

1. Seek Reviews Early

elearning_course_reviews

It is a complex situation, but one easy and miraculously effective step in creating better designs is to simply seek reviews earlier in the process. This is really the central idea behind the Successive Approximation Model (SAM) for design that we use at Allen Interactions.

2. Create Rough Prototypes

Create very rough prototypes of interactivity. Don’t worry about the wording; don’t worry about the color choices; don’t worry about the size of a button, don’t worry about the quality of the artwork. At least, don’t worry at this point in the e-learning development process. Those are all problems very easy to resolve once you know what you are striving for in the underlying interactivity. Reviewers will be much more able to make constructive comments when it is clear too much work hasn’t yet been invested in the details.

3. Focus Reviewers' Attention on Interactivity

Put the e-learning design ideas in front of your reviewers before it is masked by details of formatting. You need to make sure the core interactivity makes sense. Say goodbye to conversations mired in incidental but largely inconsequential aspects of the design. Instead, you will find ideas that create the foundation for successful learning. Your conversations need to be about forging more meaningful context, exploring what facets will challenge and motivate the learner, how to craft meaningful gestures to simulate and reinforce actual on-the-job behaviors, and developing opportunities for intuitive intrinsic feedback.

And in case you are curious, as it turns out the New Orleans Museum of Art is a really lovely institution. Indeed, there were some “nice Monets.”