Tony Karrer's eLearning Blog on e-Learning Trends eLearning 2.0 Personal Learning Informal Learning eLearning Design Authoring Tools Rapid e-Learning Tools Blended e-Learning e-Learning Tools Learning Management Systems (LMS) e-Learning ROI and Metrics

Wednesday, May 20, 2009


I just got through reading Jay's post and article (with Clark Quinn) around Become a chief meta-learning officer – one of the hot list items from two weeks ago.  It's a great article, definitely worth a read.  It discusses the needed transition in focus of a CLO

The scope of the job of the CLO is mushrooming. CLOs will neither prosper nor even survive if they fail to take responsibility for the overall learning process within their organizations.

Your charter as Chief Meta-Learning Officer is to optimize learning throughout the organization, not just in the pockets that once belonged to HR. This takes a broader perspective than what you deal with day-to-day. You’ve got to rise above the noise to see the underlying patterns, and then optimize them.

The reality is that this equally applies to Learning Organizations and Learning Professionals.  The broader perspective he is talking about is to look beyond formal learning to informal learning, social learning, collaborative learning, and personal learning. 

What was particularly interesting about his article was the use of the term meta-learning.

Wikipedia defines meta-learning (in education) as:

The idea of metalearning was originally used by John Biggs (1985) to describe the state of ‘being aware of and taking control of one’s own learning’. you can define metalearning as an awareness and understanding of the phenomenon of learning itself as opposed to subject knowledge. Implicit in this definition is the learner’s perception of the learning context, which includes knowing what the expectations of the discipline are and, more narrowly, the demands of a given learning task. Within this context, metalearning depends on the learner’s conceptions of learning, epistemological beliefs, learning processes and academic skills, summarized here as a learning approach. A student who has a high level of metalearning awareness is able to assess the effectiveness of her/his learning approach and regulate it according to the demands of the learning task.

Jay's article is really using metalearning in a different way – take responsibility for learning across the organization.  Look at all the different ways learning can occur.  Close the training department.  Etc.

Still, when I read the title, I couldn't help but think that Chief MetaLearning Officer was particularly apt, especially when you take "metalearning" according to its definition above.  Metalearning is really about:

being aware and taking control of one's own learning

It is a critical element to success moving forward.  And it's exactly what I've been talking about over the past few years.  The only way to handle long tail learning is to focus on providing the tool set and personal learning and working skills (work literacy) that are central to concept work.  Where work and learning are not separate, metalearning is really the focus of performance improvement. 

In Learning, Extended Brain and Topic Hubs, the focus is really on a new process for learning.  Being aware and in control of the process is metalearning.  Nancy Devine in a comment suggested "schema building" which is similar to pattern identification.  But all of this is really about metalearning.

Way back in 2006, I wrote Improving Personal Learning - A Continuing Challenge for Learning Professionals, that also cited a CLO article on Implementing Learning How-to-Learn Strategies (which doesn't seem to exist anymore).  Three years later, we recognize the increasing importance, and the need for greater metalearning development opportunities and the role of learning professionals and learning organizations in this.

Is metalearning a good term to encapsulate what we are talking about?


Blogger In Middle-earth said...

Kia ora Tony

I have been guilty of using the ugly term metacognition in posts. My hunch is that it is akin to, if not the same as the term metalearning used in your post. I was introduced to the term metacognition from Ben Williamson of Futurelab. I'd be interested in your take on this.

Catchya later

Unknown said...

Ken - Great second citation.

It's strange - I've certainly use the terms metamemory and metacognition many times during presentations and posts, but until I read through that source, I didn't quite have the right language.

Time to write up a post to think through this and capture it.

Bill said...

Before you start defining metalearning, you need definitions for:
1) formal learning
2) informal learning
3) social learning
4) collaborative learning
5) personal learning
that are more than just marketing buzzwords.

What is the difference between these five concepts? What are the strengths and weaknesses with each? How does one know when they are practicing one form or another?

Before you start shutting down training departments, hiring Chief Learning Officers, and coining an umbrella term for different learning methods, you need to establish what you are actually talking about and why it is preferred over other methods. And you need to back this up with some empirical data.

Blogger In Middle-earth said...

Kia ora Bill.

I have the hunch that what you broach is a bit chicken&eggish. If Tony's metalearning is anything to do with metacognition, and I think it is, then I'd say that we don't need to define what all these other terms to do with learning are. I'd tend to agree with you that they are buzz-words, nevertheless.

It is an academic exercise to classify modes of learning (as in your list) when metacognition is really to do with ‘being aware of and taking control of one’s own learning’.

If I can use a metaphor here, classifying the elements of grammar in a language, such as syntactic analysis, is simply not necessary for an individual to learn the language, despite the place of parsing in grammatical academia.

Every child who learns to speak acquires the language, in some cases almost perfectly, without any knowledge of these classifications.

Catchya later

Tony Karrer said...

Great question Bill. It's going to spark a few blog posts over the weekend!

Donald Clark said...

In Robert Marzano's "A Theory-Based Meta-Analysis of Research on Instruction," he describes the metacognitive system as meta components responsible for organizing, monitoring, evaluating, and regulating the functioning of thought. It is basically the "executive control" of all thought processes. It designs strategies for accomplishing a presenting task -- "How do I go about solving this complicated task?"

Where I have always associated the term "metalearning" with knowing how to learn and learning how to learn. Thus the term, "Metalearning Officer" strikes me as completely wrong -- "I will control all your learning processes." Next it will be the "Metacognitive Officer," aka, Big Brother :-)

Sreya Dutta said...

Hi Tony, great post. I really like the sound of the term meta-learning better than meta-cognition, but there is a valid reason to why this comparison came about. I too can't differentiate between the two immediately except for the fact that for a newbie, meta-learning is easier to cognize. But to my understanding, cognisance holds are greater meaning in terms of being fully aware of yourself and your surroundings. Meta-cognition would be being fully aware of your state of awareness itself. So i would like differentiate learning and cognition that way. Where learning is focused at an objective or information that one is looking for. Then meta-learning is being aware of your own learning patterns. Again, learning officer makes sense but I wud probably agree with Don on meta-learning officer.

Am I making sense? I hope so.


Sreya Dutta said...

on 2nd thought, I'm thinking that meta-learning officers would do what instructional designers would do if they were to look at just organizational learning.


Tony Karrer said...

One part I'm struggling with on this:

Do metacognitive strategies include the skills, process, tools, and knowledge that we apply to our work and learning?

Is Work Literacy really all about metacognition?

By the way, a meta-learning officer would be the person responsible for helping people throughout the organization with self-directed work and learning. Isn't that a big part of what we want CLOs to do?

Bill said...

Hello Tony:

"Is Work Literacy really all about metacognition?" I believe it is and I base my answer on Leonard and Swap's concept of "Deep Smarts." They define deep smarts as the accumulation of knowledge (tacit and explicit) and a deep understanding of a how particular business or profession operates. If you envision a body of knowledge as a bell-shaped curve, novices start at the middle. As they confront rarer experiences, their "curve of knowledge" expands outward.

According to research, developing deep smarts takes around 10 years but the authors suggest various methods to speed up that process. There is mentoring, directed readings, simulations, and so on. That part of the book actually reminds me of Ruth Colvin Clark's work on building expertise.

So, I think you are onto something with metacognition and I encourage you to consider Leonard and Swap's deep smaarts as the definition you are looking for.

Deep Smarts: How to Cultivate and Transfer Enduring Business Wisdom
by Dorothy Leonard and Walter C. Swap (

Tony Karrer said...

Bill - Thanks for your great comments. I'm really enjoying the dialog (and the learning).

I found a few great resources online including a great presentation deck online by Dorothy Leonard on Deep Smarts.

You comment about this sounding a bit like the studies of expertise struck me as well.

And lots of good stuff for me to think about relative to how build expertise.

What was striking to me is that I'm not sure that I'm really thinking all that much about "Deep Smarts" true expertise as much as I am about how you can extend your value as a knowledge worker by better leveraging the tools available to you. This capability extension is both Shallow and Deep. In a lot of cases for me personally, it's about rapidly getting up to speed on a topic and building knowledge and capacity in that area.

I'm sure that this all occurs within the context of Deep Expertise about my consulting, work, etc. But within the context of the new area, I'm really trying to move quickly from Shallow to some midpoint. And likely it's partly how I can tap into Deep Smarts when I need it.

Vic Uzumeri said...


I am going to risk a strong responses by suggesting that the management of metalearning demands a level of operational knowledge and experience that elearning professionals of my acquaintance have too often lacked.

The process of learning always occurs in context. For most organizations, that context revolves around the organization's core technical processes and business activities.

I have great skepticism about any focus on 'learning about learning' or 'learning how to learn' that isn't anchored to a deep understanding of a specific body of knowledge and operational behaviors that must be learned.

The introspection implied by metalearning has to start with practical questions. What is known? What is knowable? Who might know it? Who needs to know it? What are their day-to-day operational priorities and distractions? What are the biggest threats from lack of knowledge? ...

The answers to these questions influence the learners' motivation, their level of preparation, the difficulty of the learning challenge and the value that the knowledge is likely to hold for the funding organization.

In short, when you really understand the nature of the knowledge in your organization, that's the time to worry about how stakeholders can be helped to learn how to assimilate it.

Not before.

FWIW, I came to my interest in eLearning somewhat later in my career, after I had already worked in engineering, project management, finance, sales and operations - first in industry and then in academia.

This undoubtedly colors my view. Possibly I rely on that experience so much that I am unable to see how others could be effective without it. That may be a contextual weakness in my ability to engage in effective metalearning.

Regardless, I can't imagine being able to visualize any number of critical learning issues or processes without having a deep understanding of the operational issues and processes that they are required to address.

Tony Karrer said...

Vic - interesting comments. A few things ...

I'm not sure that I buy that a person who is good at metalearning coaching necessarily needs lots of expertise in the subject matter of the learner. Is it that different than having a background in performance analysis, ID or even management?

The learner will need to be able to define how it applies to them, their needs, etc.

I agree that learning is normally within a given context - however, a lot of the metacognitive strategies you learn in college are taught to you so that you can later apply them in a new context.

In fact, you gave us a list of questions that would seem to be fairly universal. Answering them is context specific - and I would even claim person specific.

I think the most important issue though might be around your statement - "In short, when you really understand the nature of the knowledge in your organization, that's the time to worry about how stakeholders can be helped to learn how to assimilate it. Not before. "

I'm not 100% sure I understand. what you mean by "understand the nature of the knowledge" - But is that knowable? KM fails when it tries to know the knowledge of the organization. I assume you mean more around what could be effective knowledge work strategies in the organization.

I personally think that an organization should have people who are trained in metacognitive strategies and performance analysis ready to move through the organization to upskill the workforce and help them (coach) build new work literacy.

Yes, the more this team has an understanding of the business and roles of the individuals they are coaching, the better job they likely can do. But, there's quite a bit you can do with little understanding - and I would caution that even a deep understanding may cause you to push for solutions that are not aligned with the worker/learner's personal learning styles.

I would really like to hear thoughts on this.

Vic Uzumeri said...


I knew that I'd have trouble explaining myself. So let me offer an example.

Engineering graduates in Canada go through an 'Iron Ring' ceremony. We get a metal ring to wear on the small finger of the right hand. The ring is now stainless steel, but it used to be iron from the debris of the Quebec bridge. An engineering design error caused the bridge to collapse in 1907, killing 75 workers. The ring is the permanent, tangible reminder of an engineer's professional obligation and debt of humility.

Any engineer nearing retirement, who has earned the respect and trust of their peers, has a personal iron ring implanted in their psyche. It doesn't matter where they trained or practiced, or their discipline. Mechanical, civil, electrical, chemical, aerospace, or transportation makes no difference. Every experienced, respected, old school engineer has made enough mistakes over their career to absorb the equivalent lesson. They will spend most of their time thinking about what could go wrong. The ones that only plan for success seldom have long careers or gain peer respect - and they are probably not worth learning from.

Why is knowing this important? Because, when I want to probe experienced engineers for their knowledge, I know they've spent years worrying about possible failure mechanisms. If I frame questions around their challenges and failures (real or avoided), it almost always elicits a rich and vivid set of memories. Moreover, these memories of threats and remedies are often the information that the organization most wants to retain.

Looking at the field from outside, most people would fail to see this pattern. They might think it is all tacit knowledge. They might even think that focusing on the negative would be intimidating. But I am convinced that this is a generalizable metastructure underpinnng much of the tacit knowledge in engineering - and it derives from the fundamental technical demands of the field.

If we look hard enough, we can probably find similar metastructures in other knowledge fields. I can think of some analogous patterns for sales professionals, accountants and software developers. Perhaps the doctor's dictum of "first do no harm" is a similar starting point.

So, the questions that I listed in my previous comment were fine as far as they went. However, they can't get at these deeper structures - and that means they will miss a ton of valuable knowledge. What can be done? Off the top of my head, I would posit three strategies:

1. If the KM or ID practitioner happens to be an experienced domain expert, they can apply introspection to try to find these metastructures in their technical field and then use them as they work with others.

2. The KM practitioner can find a domain expert and try to groom them to serve as a KM practitioner. With the right expert (one who is naturally curious and introspective) and the new generation of 'rapid authoring' tools, this might just work. However, it is hard to predict who will take to it and who won't.

3. The KM practitioner can seek out a sage, experienced expert that they can partner with. The KM practitioner is the metastructure hunter and the expert is the hunting guide. I would look for an admired and respected expert who has recently retired. I would look for someone who is thoughtful and has diverse interests. Every company has at least one or two of those. Fortunately, you won't need very much of their time. It will keep them intellectually engaged, but it shouldn't crimp their bass fishing or the time with their grand-kids.

Obviously, I think the third option is probably the most practical approach for the majority of KM practitioners. But that's just my $0.002.

A final thought. Wouldn't it be useful if the KM community were to try to elicit and catalog a sampling of these patterns from each major knowledge domain? I don't think that we need to find or understand every structure. We just need to recognize enough structures that we can use them to winkle out the information that we are seeking.

Amit said...

Jay and Clark have put up a great piece there. Though I am not 100% with the use of CMLO.
Just put up a response post on our blog. Access it here:

Tony Karrer said...

Vic - It's very interesting that you are getting at Deep Smarts as was discussed already. I agree that there's value in that and that someone outside of the profession or role or whatever really won't get to the point of Deep Smarts. But ...

I still am not convinced that you need to be a domain expert in order to help provide value. Would you say that engineers are necessarily good at using new tools and processes for storing and sharing knowledge? Do you believe that these tools and processes are not useful to them? I don't think you would argue that. Rather, it would seem your argument is that you need to know how this fits with the underlying goal of risk avoidance. Maybe these tools need to be used in a different way in the context of an engineer or maybe they do not apply. To me the sin is not asking where and how they apply. And that's what I believe is happening today.

Of your strategies #3 closely matches what I'm describing.

But you've lost me with proposing that strategy and your statements above that seem to suggest that it's not a good strategy because the coach wouldn't have deep expertise. Or maybe that's not what you are saying?

Tony Karrer said...

Amit - great post. I agree with you that "we need to take out the CLO KRA sheet again and probably include or move up items like Social Learning, Informal Learning, etc."

Even two years ago, there were a lot of CLOs who didn't recognize this as their responsibility. And likely a few that still don't.

Your comment that "bring into focus the huge amount of changes happening (and predicted) in the workplace and in the workplace learning domain and the need to act quickly to match the speed of changes themselves."

Extremely well said - and that's really the point of Jay's article I would guess. They need to recognize the additional focus to provide coaching and support for these new forms of work and learning.

Donald Clark said...

Tony, you asked, "Do metacognitive strategies include the skills, process, tools, and knowledge that we apply to our work and learning?" I would say yes in that metalearning is a subset of metacognitive skills. In addition, metacognition includes work related skills such as self-reflection and goal setting for work.

Vic Uzumeri said...


Absolutely, a generalist coach can apply meta-learning concepts to do a far better job than a generalist coach who doesn't.

But it doesn't follow that better qualifies as complete or best - or even as good enough. For that, you have to go down another level and intuit the mental models and learning patterns that the domain practitioners actually use. These are typically tacit knowledge - unspoken and derived from years of experience. Yet, in my experience they are similar for most experts in a field because the experts' formative professional experiences were similar.

If I were to use an imperfect analogy, I would equate the concepts and techniques of meta-learning to panning for gold. It beats farming and you might well get rich - but you have to know that there is a lot more gold somewhere up in the hills.

I also don't see a conflict between my contextual scenario and #3. My point is that many sophisticated domains have their own native patterns of meta-cognition that are driven by the exigencies of the field. To understand these patterns, you need a good guide. The guide is not to help you find nuggets of knowledge, but to help you understand how the experts really think about the knowledge in their field. When you know that, you'll have a shot at finding the mother lode.

Finally, you distilled my engineering examples to a general concept of 'risk avoidance'. That is barely a faint echo of the way a good engineer thinks about his work. If I had couple of hours (an equal number of beers would help), I think I could convince you that there is a hell of a lot more to it than that. As independent evidence, there are large, mature bodies of knowledge in engineering that are solely devoted to understanding, quantifying and remediating those risks. Most engineers know all about them. Most non-engineers don't know that they exist.

I think I could give similar scenarios for a star software developer or a high-performing B2B salesperson.

Tony Karrer said...

@Donald - thanks for chiming in on that. That's pretty much what I'm understanding as well, but wasn't sure.

@Vic - I agree with you that there are deep mental models and thinking that experts do that are very hard for anyone to get to and understand. BTW, I used the general term "risk avoidance" as merely a name for what you were describing. I don't claim to understand it - but I don't think you gave me a term to use either.

I can't tell anymore where we disagree. Here's what I'm hearing.

The metacognitive coach will be working with an expert to understand the work, search for structures, discuss application of new tools and processes, basically be a guide. This corresponds to your strategy #3, but I'm also including the work of looking at more than "metastructures" but also to take the expert outside of how they do their work today to consider where new tools, processes, strategies might apply.

Are we on the same page on this?

I realize that gets back to who the metacognitive coach will be and your contention that it might not come from the ID/performance analysis field.

BTW - I'll be curious about the term metacognitive coach.

Vic Uzumeri said...


I think we are close to 100% agreement. The differences, if any, reside in emphasis and mechanical details.

I would take your description of the metacognitive coach (MC) (yeah, interesting term) and tweak it as follows:

1. The MC understands meta-learning and meta-cognition. The MC also knows the tools and their efficient application.
2. The Domain Guide (DG) is steeped in the practice and ethos of the field. Usually, not always, the DG hasn't spent much time thinking about how they actually think about the problem (the tacit knowledge problem).

As I would see it, the MC would approach the work in two steps:

1. MC would recruit a willing and capable DG and raise the DG's awareness of meta-level issues. The MC would probe the DG to try to coax out some stable, usable cognition patterns that are generic to most domain practitioners. If the DG is the right person, they will come aboard pretty quickly.

2. The MC takes the elicited patterns and uses them to direct a) the acquisition strategy for the deeper domain knowledge and/or b) the repackaging of that knowledge in forms that help practitioners relate and assimilate it more easily.

In the latter pursuit, the MC would take the lead in figuring out how to align the revealed domain metamodel with the capabilities of the available tools.

A quick example.

If you accept, as I propose, that most good engineers tend to approach design problems by working back from all the possible threats and consequences, it might make sense to organize eLearning to exploit that.

Instead of organizing courseware as a structured review of a topic, it might be cool to begin with a set of juicy horror stories and use a wiki-like approach to link back to the engineering principles that could avoid them. A different wiki view might then assemble those engineering principles into a loose topical organization more suitable for browsing.

If I were betting, I would expect that an engineer would go first to the juicy stories. When they were convinced that there was something to be concerned about, they would eventually shift to reading the organized topics in logical order, checking back to the juicy stories every so often to retain motivation.

I doubt that an MC, by themselves, would figure that out. However, an MC who partners with a thoughtful and curious DG might come up with something that is even better and absolutely riveting.

Tony Karrer said...

Vic - great stuff! I like your definitions. However, I would also say that in addition to doing this with a Domain Guide (DG) you also need to work with individuals who will likely have their own personal learning and work styles. So, even if you are successful working with the DG to define new work and learning patterns for people in the domain, it still may require differences in working with particular individuals.

The other thing that I should point out is that while you jumped to an eLearning model, I'm thinking about solutions that do not involve content creation by a learning organization as well. In other words, if you can show the DG how they can better find, organize and share information, do you even need to create the resource you describe? (Probably yes, but in many cases the answer is no - or - we can't afford to spend the time to build that resource).

I hope a few other folks will chime in on where we are landing on this.

Vic Uzumeri said...


"if you can show the DG how they can better find, organize and share information, do you even need to create the resource you describe?"

Funny you should ask :-)

I've been working for about a decade on finding better ways to use video to accomplish those goals. It was slow going in the early years, but recently progress has begun to accelerate. Several avenues seem especially promising:

1. Technically, we have reached the point where we can now exchange 'video memos' instead of text and graphic memos. I'm not talking TV quality video. I'm talking about spontaneous memos that are simply Flipcam/Camtasia video files.

My company has developed a 'private YouTube' application that combines better-than-YouTube video handling with fine-grained permissions and access controls.

Key concept 1: Practitioners hate to write, but they usually love to show and tell with colleague. If we could reduce the effort and camera-shyness, new web video technologies make sharing much faster and easier than most people believe.Key concept 2: Some types of knowledge have an intrinsic dynamic. I call it 'mind music'. Many tasks have a 'rythm' that is virtually impossible to explain with symbolic language.2. Mixed media documents, including videos embedded in PDFs.

3. A critical insight about the role of video in cross-cultural communication. In effect, visual explanations are effective in every language. This, I would submit is a huge untapped potential for global firms.

4. Some experiments in Flash video delivery. We have a Flash player control that edits a playlist of clips into a seamless video - after you have downloaded the clips to your browser. This means that we can edit video-based eLearning materials much later in the media development cycle - literally up to the point of page load.

My overall goal is to get to the point where it is faster and easier to author eLearning in video than it is to do it with a software authoring tool or text.

Getting closer.

Tony Karrer said...

Vic - that's an interesting sounding application and I agree there's something quite good about visual capture. There's also a challenge with accessibility of small chunks of information from the middle. To me, it's a balance.

Tony Karrer said...

I've been reading about Metacognition, Deliberate Practice and Expertise quite a bit as a result of this discussion. And most of what I'm finding is aimed primarily at creating Deep Smarts or Experts.

There's something very important in that, but I personally find myself needing to be an expert in Shallow Smarts. Quickly getting up to speed on a new topic. Knowing how to dive in more deeply when needed. My gut tells me that most of us need to be really good at this.

Is there something here?

Vic Uzumeri said...

Re: video

After a bit more than a decade of experimentation, I have formed some strong opinions about video's role:

1. Video is at least as valuable a knowledge capture tool as it is a knowledge delivery medium. There is absolutely nothing wrong with recording an expert on video, then hunting through the video to help you write a text manual or text and image eLearning course. You can grab stills to illustrate the points. Just because you record it doesn't mean that you have to inflict it on some innocent viewer.

2. The shorter the video the better. Chunk, chunk, chunk - then use other media to organize the chunks.

3. Including synchronized subtitles makes a rapid seek far more effective.

4. You still can't really search video content directly, but you can search a companion transcript, text descriptions and meta-tags. We have been working on merging video with a Google-like search metaphor. Enter a keyword and see all the places in hours of video where the term appeared. Jump directly there and play. Hop around looking for the 'hits' that answer your question.

5. Video is superior to text for explaining some types of knowledge. For example, download this 2MB pdf and click on the play buttons on the images to play the embedded videos. I dare you to try to explain that in a text document :-)

6. There are tons of things that should never be put in a video and no one should ever be forced to watch - a talking head reading a dry script is my pet peeve.