Tony Karrer's eLearning Blog on e-Learning Trends eLearning 2.0 Personal Learning Informal Learning eLearning Design Authoring Tools Rapid e-Learning Tools Blended e-Learning e-Learning Tools Learning Management Systems (LMS) e-Learning ROI and Metrics

Tuesday, October 16, 2007

ROI and Metrics in eLearning

I'm working on an article around the use of ROI and metrics in eLearning. I did a quick search for resources and here's what I found. I thought it would be worth posting this list here.
  1. Kirkpatrick’s Level 3: Improving the Evaluation of E-Learning

    Level 3 evaluations measure whether the there was an actual transfer of learning to the actual work setting. This level of evaluation will increase the visibility of learning and development efforts, because successful implementation of Level 3 evaluation

  2. Internet Time Blog: ROI is toast. Use EVA instead.

    ROI is toast. Use EVA instead.

  3. Support - ROI Calculator

    Training & Support Return on Investment Calculator

  4. Investing in Learning: Consider Value, Not Just ROI

    ROI, or return on investment, is king in today’s business world. Touted routinely by managers and project leaders, ROI is used as a selling point in print advertisements and is featured regularly in news and business discussions because, for every purch

  5. Build the Business Case for Training and Measuring ROI - LTI Magazine

    A training program with objectives linked to business results and backed with a solid business case is less vulnerable to spending cuts. To ensure adequate funding and organization-wide commitment to your training program, you must be equally committed to

  6. Online Course Development: What Does It Cost?

    Does it cost less to design and develop online teaching and learning today than it did a few years ago? Are the categories of cost different today from the past and from what the costs might be in the future? The costs of developing online programs are si

  7. Determining Training Return-on-Investment (ROI) - Strategies to Succeed in Training: School for Champions

    Measurement of training ROI starts with defining the reasons and goals for the training, determining how much the training costs, and verifying the amount of return. Questions you may have include: * What is the reason for the training? * What i

  8. Assessing the ROI of training

    If people really are your greatest asset, isn't it time to look at your training programmes as investments in your organisation's human capital and not just as an expense? In this article, Clive Shepherd argues the case for return on investment (ROI) as a

  9. ROI from Workflow-Based E-Learning

    Business process management (BPM), workflow management, business activity monitoring (BAM), and workforce optimization software also are designed to reduce lag time. BPM and workflow technologies have been able to achieve significant gains in productivity

  10. Metrics

    Recently, functional managers have begun using eLearning to meet business objectives. Managers look beyond employees to customers, suppliers, and distribution channels -- everyone benefits from seeding eLearning throughout the value chain. This is where w

  11. Return on Investment in Training - Learnativity.com

    The problem is that nobody is quite sure what the appropriate metrics are for measuring ROI for learning. Is it student throughput or time to mastery? Is it dropout rates or full-time equivalents returned to the workforce? One thing is certain: many smart

  12. The Truth About e-Learning ROI

    There were four key questions that were going to be addressed: 1. What do we mean by ROI? 2. How important is it? 3. How do you calculate ROI? 4. What’s wrong with how it’s generally calculated?

  13. Proving 900 Percent ROI

    The ROI for e-learning internally at Cisco is 900 percent per course.

  14. Measuring the ROI of E-Learning

    E-learning can have an enormous financial impact on a business, but proving a return on investment can be tricky unless you know what to look for. There are hard and soft cost savings associated with e-learning. Hard numbers are the external factors, such

  15. E-learning ROI: How to Build Your Business Case - LTI Magazine

    Despite the challenges associated with measuring ROI, there are simple steps that companies can take to create a meaningful ROI in a faster and more cost-efficient way. Below, we'll explore how companies can conduct "pilot" e-learning projects by tapping

  16. Measure the Metrics - How to link e-learning to business strategy - LTI Magazine

    At this point, someone higher up in the organization will start asking about the return on investment (ROI) for the program and whether they're getting their money's worth. The way to demonstrate that is by doing Kirkpatrick Level 3 (measuring the extent

  17. ROI of E-Learning: Closing In

    When it comes to e-learning, computing ROI suddenly becomes a complicated procedure requiring thoughtful chinstroking, serious seminar time, and earnest input from consultants and vendors.

  18. A Fresh Look at ROI

    Different levels of management make different sorts of decisions, so it's appropriate that they use different measures of ROI.

  19. ROI for E-Learning

    People fail to recognize many non-cost related benefits of e-learning solutions, such as reach, consistent messaging, and flexibility. Despite those factors (and many others like them), proving a business case for e-learning still means being able to demonstrate value.

  20. eLearning Technology: Formalizing and Investing in Informal Learning
  21. eLearning Technology: State of Assessment by E-Learning Developers
Additional Articles Added April 28, 2008

· Cost Comparison: Instructor-Led Vs. E-Learning

· ROI and Metrics in eLearning : eLearning Technology

· e-Learning Centre by Learning Light - Costs, benefits and ROI of e-learning

· Tips to ensure that your investment in e-learning is effective.

· Return on Investment in Training - Learnativity.com

· ROI for E-Learning

· Online Course Development: What Does It Cost?

· ROI and Analytics

· Internet Time Blog: ROI is toast. Use EVA instead.

· Assessing the ROI of training

· eLearning Solutions for custom corporate training - Knowledge Anywhere

· Measure the Metrics - How to link e-learning to business strategy - LTI Magazine

· Calculating ROI

· A Fresh Look at ROI

· ROI of E-Learning: Closing In

· E-learning ROI: How to Build Your Business Case - LTI Magazine

· Proving 900 Percent ROI

· The Truth About e-Learning ROI

· Metrics

· ROI from Workflow-Based E-Learning

· Determining Training Return-on-Investment (ROI) - Strategies to Succeed in Training: School for Champions

· Build the Business Case for Training and Measuring ROI - LTI Magazine

· Investing in Learning: Consider Value, Not Just ROI

· Kirkpatrick's Level 3: Improving the Evaluation of E-Learning

· Measuring the ROI of E-Learning

· Increasing Prosperity through value networks

· elearnspace. everything elearning: Evaluating: ROI

· Learnometer: Dashboard measures of investment performance in learning...

· Learning Circuits - Analyze This!

· OUseful Info: Course Analytics - Prequel

· Course Analytics, Part 1 - Visitor Behaviour

· Course Analytics, Part 2 - When Are Our Students Online?

· Visual Analytics Inc.

· Learning from Key Learning Indicators

· Training Analytics: The Next Big Wave in Learning Management Technology

· Donald Kirkpatrick - Wikipedia, the free encyclopedia

· E-Learning Analytics,

ASTD Josh Bersin What to measure: efficiency, effectiveness, compliance

6 comments:

Benjamin Hamilton said...

Tony,

Great links. I certainly appreciate them. I also look forward to reading your article based on your research. Can you give us a heads up on where it will be so we can be looking for it?

Ben

Lee Kraus said...

Tony, Thanks for putting the links in del.icio.us. I started doing it, but got sidetracked. Now I have them all. Lee

Anonymous said...

Tony,

I thought you would like to know that next week The eLearning Guild will be publishing a major report on Measuring e-Learning Success (it will probably come in around 350 pages or so).

I've been working closely with Will Thalheimer, Sharon Shrock, Bill Coscarelli, Brent Schlenker, Jim Ong, Roy Pollock, Allison Rossett, and Maggie Martinez. We're looking both at the micro level (e.g., does the assessment we just built really measure learning?) and the macro level (do we have quantitative measures that show that our learning interventions benefit the company?)

I'll keep you posted.

Steve

jay said...

Tony, I have plowed this ground for a long time and have concluded that it's more productive to talk about business results than to quibble about whether ROI is the appropriate means to measure them.

I have just posted a scan of Chapter 3 of Informal Learning that does precisely that. It's at http://tinyurl.com/34cczx

The ongoing blather about whether informal learning is 80% or 60% or 40% is not a big deal to me. Obviously, the ratio depends on the context. The fact is that informal learning is a lot more important than we've thought, and learning professionals that overlook it do their employers a disservice.

One cadre of nitpickers is trying to throw out the baby with the bath water, quoting me as saying I made up the cost allocation figures. I would describe it as informed judgment rather than make-believe, but again the point is not 3-place accuracy, it's that most organizations spend precious little budget or mind-share on informal learning compared to formal, to the detriment of the bottom line.

I don't have much patience for the people who say they haven't read my book but they know it's wrong. Geez!

As for the argument that some guy I never heard of made up the 80/20 proportion out of whole cloth and therefore it's wrong..., the critic should get a life. I'll post some sources at informl.com in a few days.

There's work to be done in measuring the impact of informal learning, but this semantic claptrap and faux-statistics get in the way of carrying it out.

How much should we invest in informal learning? Where? I'd like to be able to say more than "more." It's currently tough to take it much further than that.

test said...

Hi Tony,

By now you probably received my email on the Micro-Learning Impacts. http://vftstation.vftdev.com/~vftlps/tp/learning_impacts_1007a.pdf

Any thoughts on this?

Ray

Cheryl McNeil said...

You have provided some great resources here for ROI and Metrics in eLearning.
The ROI on elearning can be expressed in many different ways and each measurement will be distinct to each company, but at the end of the day eLearning saves money over traditional learning.