LMS Satisfaction Features and Barriers : eLearning Technology

Update, Nov. 2007 - I've created a series of posts around Learning Management System (LMS) Selection:

• LMS Selection Presentation
  
• LMS Team Size and Time
• Learning Management System RFP
• LMS Selection Presentation Reformulated
  
• LMS Selection Team and Stakeholders
• LMS Selection Process

I thought I had previously wrote about this, but I realized when I was posting Learning Management Systems (LMS) Gotchas that I'd not previously talked about the results from participating in the eLearningGuild's Learning Management System research report. You can get the abstract for the report and sign-up for a webinar on Sept. 26th on that same page. I was one of several authors who collaborated on the survey questions and wrote different sections of the report.

The survey data itself I find to be pretty interesting and useful for some important aspects of selection. The survey asks demographic and company questions of all the participants so that you know things like the industry, size of company, number of learners, their role, etc. Then it asks questions such as satisfaction level, satisfaction with particular features, are you planning to replace, primary barriers, importance of features, etc.

A few of the more interesting graphs that you can't really see well except by going and seeing a larger version at LMS Satisfaction and LMS Barriers and Features.

Overall Satisfaction in Large Corporations (Count >= 3)



Overall Satisfaction in Small and Medium Corporations (Count >= 3)



Overall Satisfaction in Education and Goverment (Count >= 3)



And to show some of the power of the slicing, I've got a graph of overall satisfaction for Manufacturing and Biotech/Pharma for companies with more than 1,000 employees and 1,000 learners with a count of 1 or more.



Some notes on these numbers...

• These are being reported out of the larger eLearningGuild population and I would claim that they are likely more accurate than survey research that goes through the vendors themselves. However, when you get low counts, the numbers are highly suspect.
• Several of the LMS vendors appear more than once, such as Oracle. Most of the time (as is the case with Oracle) these are different products. Sometimes it's how they were referred to by the survey respondents and it hasn't been cleaned up in the data.

Some comments on the satisfaction numbers themselves...

• The dissatisfaction numbers are high in general. And this is not being reported only in this research. I mentioned in previously in LMS Dissatisfaction on the Rise. An LMS is a big, expensive tool that takes quite a bit of work and is generally harder than you think it's going to be when you start out. It's why I often try to convince people to not Lead with an LMS.
• To me, it's interesting to see news like CornerStone OnDemand Raises $32M - Sept. 17 2007 - when their satisfaction score is being reported so low. That seems like a disconnect. I know the folks from CornerStone and their product does some really interesting things. They are moving towards a broader suite that focuses on other aspects of Talent Management. But given the number of respondents who've rated them low on satisfaction, it would give me pause during a selection process (so that makes me worried about sales), it also would have worried me as a potential investor. Maybe it's good I didn't post this until today.
  
• Moodle scores very high in satisfaction, but we need to qualify that result a bit. I personally feel that Moodle is good at the limited stuff it does. It's free which improves satisfaction. But it's not really an enterprise LMS and has some very serious deficiencies when it comes to many of the needs of corporate training departements. However, I would be concerned if I was a starter LMS vendor because Moodle is going to cause you grief in the bottom of the market. If nothing else, it causes the perception that there's a free competitor.
• SkillPort scores very high as well on satisfaction. However, like Moodle, it only addresses particular needs and so most often it's a starter LMS. I do highly recommend using a starter LMS if you are new and you don't necessarily know what you need later on. Note: this represents a lot of the market whether they admit it or not. Of course, if you are consciously choosing something as a starter LMS, make sure that everyone knows that its a starter and that you plan to move in a few years.
  
• Oracle's (Learning Management and PeopleSoft Enterprise Learning Management but less so it's iLearning) and SAP Enterprise Learning score better than I would have expected. For the past several years during selection I've found that they trail Saba and SumTotal on needed capabilities and they were hard to use for particular tasks that were important to the clients involved. Based on the satisfaction numbers, it appears that they are catching up and it will be interesting to see where they are during the next evaluation. As these systems catch up, this will make the market really hard for Saba and SumTotal. Over the past few years, they've been able to fend of Oracle, PeopleSoft and SAP through superior products. If the playing field is more level, then it becomes harder to argue why you don't want an integrated solution. So large companies with Oracle or SAP implementations are going to start going more towards using those packages.
• The numbers displayed above don't show this, but you can slice the numbers according to the level in the organization and by role. No surprise, but the people who have to work with the LMS day-to-day give a much lower satisfaction number than the managers and directors who get the results and the reports. Performing tasks (use cases) in an LMS are harder than they seem like they should be and it results in lower satisfaction.
  
• I wouldn't read too much into any of these numbers. They can definitely be useful to see what LMS products are being used in companies similar to yours. They can be used to see how LMS vendors stack up against particular features. But you still need to look at differentiating use cases to choose a vendor that will work for you.
  

Other data in the report points to particular barriers and the importance of particular features during Learning Management System (LMS) Selection. To see better versions of these graphs you can visit: LMS Barriers and Features.

Barriers






Importance of Features






Some notes on Barriers and Feature Importance:

• Good news for me - problem with 3rd party consultant scores low as a barrier. :)
• A lot of the other factors are pretty significant barriers, e.g., cost, customization, flexibility.
• On the features importance, looking at the aggregate numbers is not very interesting. However, looking at numbers for your particular type of industry, size, etc. can yield more interesting results.

Some other posts in my blog around Learning Management Systems:

• Learning Management Systems (LMS) Gotchas
• Tracking Without an LMS
• LMS Dissatisfaction on the Rise
• Do You WANT an LMS? Does a Learner WANT an LMS?
• Moving from One to Many - LMS Products are Two Generations Behind
• Leading with an LMS - Harmful to Your Health (or Skipping Stages ...
• Tools for On-Demand Information - An LMS?
• Point Solutions vs. Suites and Composition

LMS Systems:

LearnFlex’TM Operitel Corporation 1 10.00
Extention LMS Acadia HCS 1 9.60
NetDimensions Enterprise .. NetDimensions 1 9.30
OutStart Evolution LMS OutStart 2 8.40
Moodle Moodle 6 8.33
Oracle iLearning Oracle 4 8.25
TopClass e-Learning Suite WBT Systems 2 8.25
SkilISoft SkillPort Skillsoft 12 8.03
CourseMill Learning Manag.. Trivantis 1 8.00
Compliance LMS Pro-ductivity Systems 1 7.50
Oracle Learning Manageme.. Oracle 14 7.48
ResultsOnDemand SumTotal Systems Inc. 3 7.33
LearnCenter Learn.com 5 7.18
WBT Manager Integrity eLearning 1 6.90
Articulate Online Articulate 1 6.60
Saba Learning Suite Saba 9 •::: 6.37
TotalLMS SumTotal Systems Inc. 27 I 6.29
GeoMaestro LMS (includes .. GeoLearning 6 6.27
Overall Rating Plateau Learning Managem.. Plateau Systems, LTD 22 6.24
Blackboard Academic Suite Blackboard, Inc. 8 6.19
PeopleSoft Enterprise Lear.. Oracle 2 6.15
Saba Enterprise Saba 23 5.97
SAP Learning Solution SAP 10 5.97
Virtual Learning System Plateau Systems, LTD 1 5.90
Cornerstone OnDemand En.. Cornerstone OnDemand 4 5.80
Training PartnerTM Learning.. GeoMetrix Data Systems Inc. 4 I 5.68
IBM Lotus Learning Manag.. IBM 1 5.30
KnowledgePlanet Learning KnowledgePlanet 3 4.50
ViewCentral ViewCentral 3 4.43
KnowledgeNet Platform LMS Thomson NETg 2 4.10
GeoConnect GeoLearning 1 3.90
Enterprise Knowledge Man.. Generation 21 Learning Syste.. 1 3.00
IntraLearn 5.0 IntraLearn Software Corporati.. 2 2.70
Pinnacle Learning Manage.. Learnframe, Inc. 1 2.50
TM SIGAL® Technomedia Training Inc 1 2.30
Enterprise Knowledge Asse.. Generation 21 Learning Syste.. 1 2.10
Meridian KSI Knowledge Ce.. Meridian KSI 1 1.60

LMS Barriers:

The Cost
IT Support
Customization
Integration with other
systems (content. HR, ERP,..
Legacy system integration
Mind set to move learning
online
Problems with vendor
Support from management
Flexibility for future
requirements
Support from stakeholders
Clear business goals
Security
Support from learners
Tool/vendor selection
Administration
Problems with third-party consultant
Compliance

LMS Features:

Tracking, reporting, and
measurement
Content delivery
Assessment and testing
Asynchronous e-Learning
Blended learning
Standards (SCORM and AICC)
Training history
The ability to support different
models and sequences of blended..
Ability to create an index so that
people can find a particular topic
Security
User and group management
Registration
The integration with single sign-on so logging in is not required
Competency and skills
Synchronous e-Learning
Regulatory Compliance
Collaborative learning
Certification
Catalog
Instructor-led training management
The ability to support specific and complex business process models