The basic issue covered in the post was how to test SCORM courses prior to loading on a given LMS (e.g., Docent, SumTotal, Saba, Learn.com, etc.).
We talked about the SCORM test suite, SCORM test wrappers, and various test tools such as Trident, Reload player (http://www.reload.ac.uk/scormplayer.html), SCORM Test Track, remote debugging using tools like WebEx, HTTP traffic tracking with tools like Firebug, and some other things. Quite a great bit of discussion and useful information. If you are interested in testing your scorm courses, probably worth a read.
But recently someone (anonymous) left comments challenging a few of my base assumptions and I'd be curious to get thoughts. They asked:
"Why is testing courseware different? Why assume that the client or someone else is supposed to provide systems for testing for courseware vendors but not say accounting systems?"All of these challenge the base assumption that I made which is - the LMS vendor really is not looking to work with people who are creating small amounts of courses that need to be tested on their LMS. That's my general sense having working with a variety of different LMS vendors, but maybe that's not true?
"Perhaps you can rent or lease the software? Most major LMS vendors have their own test labs, support, and professional services."
"All of them sell their LMSs."
"There are third-party companies which you can hire to test courseware against LMSs."
"Have you ever tried contacting SumTotal about what options they have available for third-party courseware testing? What did they say? How many other LMS vendors have you spoken to?"
Also, I wonder how much it would take to get the LMS setup for scorm testing? Cost? Hours?
Any comments from the LMS vendors ?
Any experience out there in working with LMS vendors to do scorm tests?
Or is it pretty much that you are using scorm testing tools and then getting on the actual installation to test and debug issues?
11 comments:
Can I recommend something pretty simple to people. We at Ossidian (www.ossidian.com) have developed lots of SCORM modules and tested them successfully using the simple strategy I describe below.
Download a few of the OpenSource LMS products out there - such as Dokeos, Moodle and aTutor. And there are other good ones too.
These all require nothing more than a LAMP or WAMP environment (either local or on your ISP). All free.
Now, load your SCORMS and test all you want !
Also, you can download the free RELOAD products (for assembling manifests; there is a Player too). These will provide about 99% of the testing you need.
We found this strategy gave us absolute perfection! An our courses run on the top 10-15 commercial lms without any problems.
Finally, we develop all our products in Flash. No authorware or cute tools. We never get problems with tracking or test scores.
I mostly agree with the test scenarios laid out above, but you still need to test your content with the actual LMSs you plan to serve your content on.
One example from my work is with the SAP Learning Solution (LSO): If the SCO does not immediately (within 3 seconds) after launch touches the lesson_status variable, the LSO will assume that this is a legacy learning object wrapped to comply with a minimum of SCORM. The LSO will in these cases set the lesson_status to "completed" no matter what the SCO reports. Here the LMS vendor has listened to the requests from one or more of its larger clients and catered to their situation within an "acceptable interpretation" of the SCORM specification. This will happen.
Also many LMSs have different interpretations on how re-taking a SCO or Content Aggregation (CA) should be treated. Some LMSs only handle positive delta values for tests, others deal with absolutes. If there is a qualification imparted by the course, you might end up loosing the achieved qualification when you - on your own initiative - repeat the course a few weeks later.
The worst case I have experienced was an LMS vendor, where their SCORM support turned out to be a moral support - they really appreciated the work done with the SCORM specification and fully supported such initiatives!
Just my .02
Just posted my thoughts on the original post, but I'll bring them over here since it looks like this is where the discussion will continue (somebody tell me if that's uncouth, I'm not up on my blogging etiquette):
>Why is courseware different?
For several reasons:
1) You talk about buying a room full of computers to test against different OS's, browsers and databases. How many different products are we talking about here? In any of those categories, you can easily get to well over 90% of market share by testing 3 product families (for instance IE, Firefox and Safari). The LMS market is much much more fragmented. The top 10 LMS vendors don't even come close to 90% of the market share, never mind the fact that many clients of the larger, most popular LMS's will customize each implementation.
2) Go back to Tony's original post and look at the situation he's talking about "We are developing a SCORM or AICC compliant course either custom or using an authoring tool." It sounds like he's in a situation where he is making a specific course for a specific client targeting a specific environment. This situation is much more analogous to a small software consultancy developing a custom application for a client. In the case Tony describes that client has a non-standard environment but won't give the developer access to test on it, yet still expects seamless interoperability.
3) "most [LMS vendors] do not enjoy wasting time on courseware vendors who have not done their due diligence." Agreed, nobody wants to waste their time with somebody who hasn't done their homework. The rub comes when there is a problem even after everybody has done their homework. We deal with our client's integration problems all time. Unfortunately the prevalent attitude (of both the LMS and content developers) is to point the finger (and sometimes raise a finger) rather than constructively and collaboratively trying to solve the problem.
No worries with cross posting Mike.
Tony, thank you for this discussion, it is so useful for educators who work with some LMS/LCMS and wish to integrate a course developed in one system in the others. In my practice I use RELOAD and SCORM Test Suit - they are perfect tools for scorm conformant test. The problem is in different LMS/LCMS - corporate and open source, because some of them are scorm compliant, but not scorm conformant. This means that some of them cover different implementations of scorm specifications. In this way the scorm packages created in different LMS/LCMS are not compatible.....but SCORM Test Tools are the right tools for identifying which scorm content package is in standardized format.
If after scorm test there are some mistakes about scorm conformant and if educators possess some programming skills, then they can manually open and add some code in the scorm packages according to the mistake report. After this operation the package is ready to be used by the conformant LMS/LCMS system.
Part of the problem as I would see it , having crossed the floor from client to vendor, is that the perception of the SCORM standard for some end-users is that it is clear and distinct - almost in the way that say a .doc filetype is. This is due in no small part to the perception painted by LMS and courseware vendors alike.
Of course, the reality is radically different, but it is only after experience, and some degree of technical understanding, that this becomes clear. In this respect Mike is bang on when he says that the process is like software consultancy - we are building bespoke apps to fit a special model.
Dan - I've wondered why we can't tighten up the semantic side of SCORM a bit. Especially around simple SCORM aspects like saving scores and impact on completion status. The syntax is fine, it's dealing with the differences in these simple areas that seems to cause the problems.
SCORM seems to be growing new capabilities and could use a bit of clean-up in these areas. Or at least have a requirement around LMS vendors specifying their expectation around their SCORM implementation. Most often there's a bit of trial and error.
You do find that large companies (who run a given LMS) will create their own guide to the particulars of SCORM on their system. Maybe the LMS Vendor can steal a couple of these and help everyone with the information.
I've got to come to defense of SCORM here a bit in response to Dan's comment:
"perception of the SCORM standard... is clear and distinct - almost in the way that say a .doc filetype is...the reality is radically different"
Have you ever tried to open a .doc file in Star Office, Open Office and Google Docs? It's far from perfectly clear and distinct, there are all kinds of little differences. Same with HTML in the half dozen major browsers. SCORM is trying to make courseware compatible across hundreds (perhaps thousands) of LMS's. Overall, I'd say it's doing quite a good job. (See previous posts on the other thread and on our corporate blog.)
I do agree with those who argue with the need for even more tightening in certain areas. If you have specific thoughts on these areas, don't be afraid to make them known, ADL/LETSI always wants to hear suggestions to improve SCORM.
Thank you for your tips, eamon, I definitly will try it especially there are some open sources can just work on local.
Thanks!!
Chang-
hello im a brand new user of scorm, i use ispring master suite and work in moodle. I have a problem though, scorm runs properly, but grades are not reported... what shjould i do, or what did i do wrong...? any help plz email me to pombiux@gmail.com
thanks :D
Post a Comment