Tony Karrer's eLearning Blog on e-Learning Trends eLearning 2.0 Personal Learning Informal Learning eLearning Design Authoring Tools Rapid e-Learning Tools Blended e-Learning e-Learning Tools Learning Management Systems (LMS) e-Learning ROI and Metrics

Monday, November 24, 2008

Evaluating the Performance of Knowledge Workers

I've recently been talking to people about evaluating performance of Concept Workers. I want to thank Dave who has provided thoughts in several posts such as Getting to Exemplary.


Evaluating concept worker performance is an interesting challenge:
  • No right answer - Most often there is no single right answer. Which authoring tool, LMS, etc. we should use in a particular situation - you can't possibly get it exactly right. You are trying always to arrive at a reasonably correct answer given all the other factors (amount of time you can spend finding an answer, etc.)
  • Evaluator knowledge limit - In most cases, the person doing the performance evaluation knows less about the subject that the performer. So, they can't directly judge the answer, but may be able to sense when answers are possibly not correct.
The interesting bottom line has been that the way you evaluate a concept workers performance is by looking at signals such as:
  • Process - They went through a reasonable process to arrive at their conclusions.
  • Reasonable - Their conclusions are reasonable in your opinion (if you can formulate one).
  • Compare - If you took what they did and compared it to what you would expect from other similar performers, would they have arrived at the same result.
To me this has heightened my sensitivity to the need for concept workers to reach out to people for knowledge work tasks in order to ensure they will receive a good evaluation. It reinforces the fact that Leveraging Networks is Key Skill. I'm coming to believe this is the most important Knowledge Worker Skill Gap. This makes this things like: Network Feedback, Finding expertise, Using Social Media to Find Answers to Questions, Learning through Conversation very important topics.

What I've been saying in recent presentations is that going to Google and searching for information as your primary mechanism leaves you open to criticism. Instead, having a conversation with a peer can give you feedback on:
  • Was my process appropriate?
  • Is my answer reasonable?
  • How does my answer compare?
And if you want to be mercenary about it, the real bottom line is that if someone challenges your answers, you have already done their evaluation for them. You can say, "Look, I talked to a couple of people who have done this before. They said I've gone through the right steps. I've looked at the right stuff. My answer seems pretty reasonable. If they would have done it, they would have come up with the same thing."

There's a beauty in this!

But it does require better ability to reach into networks for help.


Anonymous said...

Tony, thanks for the compliment.

I might add one thing: exemplars aren't always aware of how they arrive at the decisions they do. Ruth Colvin Clark talks a lot about this in her book, Building Expertise.

Expert practitioners deploy a great deal of tacit knowledge without realizing it. Things are second nature to them -- usually, not because of some innate capacity, but rather because of range and depth of experience.

Tom Gilbert's phrase for this was "watching Martina Navratilova's feet." He meant two things: the exemplar isn't conscious of everything she does, and she can't easily observe herself.

Damon Regan said...

Hi Tony,

When does the important skill turn into a distraction? It seems to me that fear of such an interpretation is the biggest barrier to its use.

Also, did you see Andrew McAfee's most recent post on rating knowledge workers?

Is there a better way to share a link in a blog?

Tony Karrer said...

Dave - that's a great point.

Damon - to leave a link you can use an anchor tag.

I have a lot of respect for Andrew McAfee, but I'm coming at this from a completely different perspective and thus I really am focusing on something quite different. He is looking at measuring things from a fairly broad perspective. I'm looking at evaluating singular performance.

Did that individual concept worker come up with a good result?

How does the manager or someone else decide?

His use of E2.0 tools is not necessarily related unless they resulted in the person having the right kind of conversation or vetting the answer through people. If the E2.0 tools accomplish that, then AM and I are on the same page.

Blogger In Middle-earth said...

Kia ora Tony

I'm still getting my head round this idea of a 'concept worker'. I've read your earlier posts and looked up your citations, etc. Perhaps you might clarify a few things for me.

A concept worker, per se, is presumably one who is or can be highly creative - right? I guess this person would have to be creative to be able to make connections that, otherwise, could leave a skilled subject expert (who is not) still thinking about what's going on.

For instance, would Leonardo da Vinci be considered to be a fine concept worker? How about someone like Thomas Edison? Or are these examples too explicit in their approach.

The thing that gets me about a 'useful' concept worker, is that they need to have extremely good communication skills. They also have to be good teachers. For in order to convey their wares, they have first to communicate and second to do so in a way that people can understand - which may also require a bit of teaching.

Concept workers have to work (directly) with people for their wares to be effective - right? Otherwise they become theoreticians who are too difficult to be understood to be of any practical use.

Einstein was often verging on this, despite his brilliance. There have been other examples in history of those who had it all but couldn't communicate it well enough to be effective - Cavendish was one.

And I'm not talking about icons here - I'm trying to identify the essence of what an effective 'concept worker' needs to have in the way of the compass of skills (not to mention knowledge).

Is it conceivable, for instance, that an effective concept worker could be a (genuine) jack-of-all-trades, but master of none? Is this perhaps a pre-requisite?

As far as I can make out, tacit knowledge seems to be one of the key attributes, yet it would be among the most difficult to ascertain. Da Vinci undoubtedly possessed vast tacit knowledge, aside from his explicit knowledge.

Or am I being too naive here :-)

Arieliondotcom said...

Your reasoning is fallacious (unless you're channeling Dilbert). If a rater doesn't understand the knowledge worker's job s/he can't determine the processes used, needed, or reasonable expectations.