Tony Karrer's eLearning Blog on e-Learning Trends eLearning 2.0 Personal Learning Informal Learning eLearning Design Authoring Tools Rapid e-Learning Tools Blended e-Learning e-Learning Tools Learning Management Systems (LMS) e-Learning ROI and Metrics

Thursday, October 18, 2007

LMS Selection Process

I’m preparing for my part of an upcoming session at DevLearn on LMS selection as part of The Learning Management Systems Symposium. I'm going to prepare for this in a slightly different way. I'm going to create notes in my blog as a series of posts as I think about what I'll present and then I'll turn them into slides. Hopefully, this will:

  1. Allow me to get input from readers and that means you - yes YOU - I see you reading this and thinking - he doesn't mean me - no I really mean YOU. Help me make this presentation better. Please. Especially what am I missing, what will be useful for an audience of corporate learning folks who are contemplating LMS selection, what won't be useful (and I should delete). I truly need to get input or this could be a boring session - and I don't like boring sessions - see my reservations about the presentation below.

  2. Create an interesting resource for participants and readers - yes that's you as well.
So, first, what am I talking about:

How to Conduct a Successful

Learning Management Systems (LMS) Selection Process



LMS Selection Background:

I'm coming from a background of personally working and coaching on a bunch of LMS selection, installation and configuration projects and being heavily involved in the design and development of several custom LMS implementations. My goal in the presentation and in these posts is to provide a basic process and try to capture specific areas that have been learned the hard way through these varied experiences to help define what makes a selection process successful. In some ways, I'm trying to help people achieve better LMS Satisfaction.

My selection experience primarily is in corporate environments, but I’ve also been involved in non-profits, foundations and to a lesser extent education and government. My custom implementation experience is generally with specialized content providers. Most of the time, I’m working with larger organizations who have 2,000+ learners and several different constiuents involved. However, most of this applies fairly well across other kinds of organizations.

Reservations:

I'm somewhat concerned that there's no good way to capture this stuff in a presentation that will be meaningful and useful. Most people only go through a few selection processes in their career. And not every issue is going to come up on every selection process. Thus, can I really identify stuff that will be useful?

Also, I don't want this to be a presentation form of eLearningGuild LMS Selection Tips. This has a lot of lessons learned, but it's daunting and it more or less prompts you about things you need to look out for, but not necessarily what to do. So, if they have 500 tips or something like that and I'm giving 20 tips, how is my presentation going to be useful?

Hence my reservations.

The LMS Selection Process



I'm not sure how interesting it will be to walk through the LMS Selection Process. In my experience, most of the definitions of the process are fairly similar. If you go out and search you will quickly find resources like:

The processes described there are similar to the process that I generally have used. And the process differs slightly based on the context. My rough draft of my LMS Selection Process is:

  1. Form a core selection team and define stakeholders
  2. Define business and learning strategy
  3. Agree to process with key stakeholders
  4. Capture requirements and differentiating use cases
    • Questions
    • Interviews / expectation setting
    • Drill down on non-standard items
  5. Conduct initial research, select initial vendors, make contact
    • sometimes done via RFI
    • eLearningGuild, Brandon Hall or Bersin might help
  6. Prepare and send RFP (Request for Proposal Sample)
  7. Select finalists
  8. Demos
  9. Pilot or hands-on tests
  10. Negotiate
  11. Final selection
  12. Installation and configuration kick-off
  13. Define models, etc.
  14. Configuration, customization
  15. Testing
  16. Pilot
  17. Communication
I'm not sure how interesting it would be to step through each line item of the process. When I've sat in the audience as someone walks through their process steps - yawn.

The LMS Selection Process Issues



So, instead, I plan to focus on key differences in the process based on specific situations and places where I've seen things go wrong before. This will be somewhat similar to my LMS Selection Gotcha list. But I also plan to include specific items such as:
  • Some political issues especially playing favorites.
  • Common communication problems.
  • How much LMS do you really need? And the starter LMS.
  • Getting agreement on strategy and process.
  • Content authoring selection vs. LMS selection.
  • Defining differentiating use cases.
  • How to work with different constituents to capture differentiating use cases and how to use that as an opportunity to set expectations.
  • Difference between an RFI and an RFP. When do an RFI? Ways to down select without an RFI?
  • Making sure you understand a critical requirement as compared to a requirement that's somewhere on the important - nice-to-have scale.
  • What requirements to put in an RFP.
  • How to balance making an RFP response useful to you and not too hard for a vendor.
  • How much should you lead a vendor in the RFP requirments.
  • Examples of good and bad requirements.
  • A few critical questions that sometimes get missed.
  • Contract negotiations.
  • LMS fit is 50% models.
  • Customization
Does this seem like a good list? Am I missing issues? Which of these are not worth discussing?

This list seems like it will take me more than an hour to go through as it stands, so I'll probably shorten, but thought I should list out what I'm thinking right now.

I'll also make a list of resources as I find them. I've got quite a few on the process and various issues. I have a few example templates. I don't have good stuff on the fuzzy front-end, defining strategy, etc. Anything you think would be a good resource, feel free to send me or tag in del.icio.us with lmsselection.

15 comments:

bill7tx said...

For what it's worth, Tony, remember The eLearning Guild's free eBooks on LMS Selection, Implementation, and Management when you're putting together your resource list. Hundreds of actual users contributed their tips for those books.

Bill The Editor

Adam Zamora said...

Hi Tony,

I found your blog recently and am happy to offer my insights on the LMS selection process from the perspective of the vendor (I've been in the LMS biz for almost 10 years now)

Here are my two nuggets of wisdom that I wish more searches would take into account:

1. Do you have an IT dept? Get them involved early. The decision, or at least the preference between externally and internally hosted should be made before you start talking seriously to vendors.

2. Think about RFP's and what you put on them. Are all the items listed on your RFP items that you really need or have you just copied them from another template? Does "Capable of scaling to an n-tier architecture" really mean anything to anybody on your search team?

There's my $0.02 - I'll keep reading.

-adam

Tony Karrer said...

Bill - I believe that's what I linked to in my post. Was there another one ?

Tony Karrer said...

Adam - I completely agree with both of your points. I'll have some more on similar items coming up. Hope you'll continue to weigh in.

TRACY HAMILTON said...

Hi Tony: You'll touch on these for sure, but something we really missed in starting our process was the hands on trial of the product, not just seeing them demo the tools. Another was talking with or visiting other clients of a similiar business nature that could show us how they were using it. Again, not just a demo from the vendor.

Something that a group of users of our tool just started last week was a monthly forum for users to come together and showcase what they are doing, tips, tricks, bugs, upgrades from vendor, a place to tell the vendor which fixes we need first. Might be good to ask as you are shopping if the company has such a forum.

Janet Clarey said...

Hi Tony-
I just took a quick review of this great list and have this to offer...

You may want to think about adding something about the people resources associated with an LMS. Usually one person is needed to administer the LMS (adding data, running reports, etc.). As well, hosted systems might require .5 people in IT support and a non-hosted systems might need 1 F/T IT person. I think sometimes managers see an LMS as a way for a learning department to increase capacity. (gee, you've got this big system to manage everything so why would you need more people...you could even have less...). Not true in my experience.

Budgetary considerations should include the cost of customizations post-implementation (SOW) because there is always some tweeking needed.

Another thing might be the development of learning governance. Who will be responsible for various aspects of the LMS?

Hope this helps.
Janet

Joe Deegan said...

Hi Tony,
I am going through the LMS selection process for the first time and I have found writing a quality RFP is the most critical part of the process. The RFP forces you to think critically about what you plan on using the LMS for and makes it easy for you to evaluate if you are going to get a return on your investment. I equate preparing an RFP to writing instructional objectives. You can't build a building without a blue print.

jjkd said...

Tony:

I see you've got customization on your list, I'd just suggest that you include a brief discussion of 'customization because you can' versus 'customization because you must' in terms of how excessive or gratuitous customization can extend the project time line, risk and cost.

The impact on maintenance and upgrade cost/effort post-implementation can be significant as well. Some vendors seem to treat this as a healthy annuity program for their professional services organization.

Re: Tracey's comment on user groups, that's a good one. I'd add that making sure the user group and customer feedback process is 'real' should be part of the evaluation --- require that the vendor allow you access to such resources up front, and check them out.
--
Joe Kyle
--jjkd--

mike said...

Having just completed an LMS selection project; another beneficial item to touch on the composition of the selection team. We had a team of 15 which to me is way too many - and of those only about 3 or 4 were actively engaged and/or understood enough about learning and the implications of LMS decisions.

As previously mentioned, involving your IT people is very important so that you do not run afoul of any infrastructure or security standards, etc.

Your IT department may be a key factor in the decision to host the LMS internally or externally. First, they may have a preference and you might be surprised by what their preference is. Plus you need to thing of whether or not you'll get better support internally or externally. Even though internal support may be "free" sometimes you get what you pay for! ;-) Plus in our environment there are so many bureaucratic hoops to jump through when dealing with IT we'd get MUCH better response and support by a vendor even though we will probably have to pay a little more for it. I think we'll more than make up for this in things such as down time, etc that it's worth it.

And I agree with an earlier post that this decision should be made early in the process. This by itself can significantly effect your ranking of potential vendors.

Lastly, when we narrowed our list down to the final 2 or 3 it was a HUGE benefit to have them set up demo systems that we could actually use a real system and do many of the common LMS activities we normally due. This is great at showing things like usability issues that will NEVER appear otherwise.

PeggyMN said...

Hi Tony, I was involved in the acquisition of our organisation's LCMS. What we had done right was we used the potential system to build our existing elearning courses to identify the pros and cons of the system. The process also help to concretise how well the system was able to satisfy our new and future requirements. In addition, a test plan was drafted as a starting point for users who will be using the system. This '360' evaluation provided concrete results and questions for the vendor.

Giuliana Guazzaroni said...

I like the way you are preparing for the symposium, your blog, your posts, the comments everything will be used for slides...a very collaborative way of creating interesting knowledge!

Karl Kapp said...

Tony,

Great idea for a conference presentation! Love the idea of multiple inputs and viewing it as a work in progress before the presentation.

So, here is my two cents worth...

I think pointing to resources is an important issue in the selection process and don't underestimate the importance of walking through the process. I've spoken about the e-learning selection process before and understanding the steps was always of concern to the audience members (A great deal of uncertainty about the steps...a lot of money and career equity at stake.)

A few years ago, I did a series of articles for eLearn Magazine on the e-learning vendor selection process covering vendor selection, writing the RFP and other elements of the process. Here are links to those articles that you may want to share with your audience

Selecting an E-learning Solution, Part 1: Who Should be on Your E-Learning Selection Committee?

Selecting an E-learning Solution, Part 2: Avoiding Common RFP Mistakes

Selecting an E-learning Solution, Part 3: Ten Rules for a Smooth, Efficient E-Learning RFP Process

Selecting an E-learning Solution, Part 4: Inviting the Vendor To Present

And finally, here are 24 pages of LMS Selection Templates. that range from scheduling dates, to listing requirements to defining terms to vendor comparison charts that may help others in the process.

Once again, I really like the idea you have created here for your presentation...collaborative research and meeting audience need through crowdsourcing. Love it!

Giuliana Guazzaroni said...

In my experience, in the choice of a LMS we have to consider various factors. The first one is the way we are going to use it. Then, we don't have to forget the specific needs of our future student-customer of the platform. In effects, the ingredient no.1 is the customer/student and his/her needs. Another issue: can the platform (that I have chosen) be enriched of new functionalities? In progress, new requirements, not previously considered, could emerge...

Malinka said...

Hi everybody! I read Tony's presentation preparation and all your comments and would like to remind you to mention: (1) commersial LMS vs. open source solutions and (2) the importance of eLearning standards.
Malinka

Terri C-T said...

Yes, include something about open source LMS -- benefits, risks, costs, etc. I'm really considering it -- the pricing looks great! But I'm wondering if, with open source, you don't "get what you paid for." (Thinking in terms of confidentiality issues -- I work in healthcare, and vulnerability to hacking, viruses, etc.)