It's very common for organizations to require completion of some number of hours of learning as part of a certification, continuing education or compliance. There are many, many examples out there.
The International Association for Continuing Education and Training defines CEUs purely in terms of "contact hours"
Continuing Education Units
One Continuing Education Unit (CEU) is defined as ten contact hours of participation in an organized continuing education experience under responsible sponsorship, capable direction, and qualified instruction.
The recent California AB 1825 requires two hours of "effective interactive training" on sexual harassment for all supervisors. They include the following definition (which is at least a bit better than most CEU definitions):
“Effective interactive training” includes any of the following:
(A) “Classroom” training is in-person, trainer-instruction, whose content is created by a trainer and provided to a supervisor by a trainer, in a setting removed from the supervisor’s daily duties.
(B) “E-learning” training is individualized, interactive, computer-based training created by a trainer and an instructional designer. An e-learning training shall provide a link or directions on how to contact a trainer who shall be available to answer questions and to provide guidance and assistance about the training within a reasonable period of time after the supervisor asks the question, but no more than two business days after the question is asked.
(C) “Webinar” training is an internet-based seminar whose content is created and taught by a trainer and transmitted over the internet or intranet in real time. An employer utilizing a webinar for its supervisors must document and demonstrate that each supervisor who was not physically present in the same room as the trainer nonetheless attended the entire training and actively participated with the training’s interactive content, discussion questions, hypothetical scenarios, quizzes or tests, and activities. The webinar must provide the supervisors an opportunity to ask questions, to have them answered and otherwise to seek guidance and assistance.
(D) Other “effective interactive training” and education includes the use of audio, video or computer technology in conjunction with classroom, webinar and/or e-learning training.
(E) For any of the above training methods, the instruction shall include questions that assess learning, skill-building activities that assess the supervisor’s application and understanding of content learned, and numerous hypothetical scenarios about harassment, each with one or more discussion questions so that supervisors remain engaged in the training.
They also include:
“Two hours” of training is two hours of classroom training or two hours of webinar training or, in the case of an e-learning training, a program that takes the supervisor no less than two hours to complete.
When it comes to online learning / eLearning, commonly this requirement for some number of hours translates to some amount of time spent in the online course.
Someone just asked me questions about this and it's a problem that I've faced many times myself. I'm hoping that you will chime in with your thoughts. And I think we would all love to see pointers to influential resources that we could use in the future to help argue our case.
Problem 1 – Measure time equivalence?
I'm sure you've been through the design discussion where you figure out how you will show that people spent at least ten hours in the online course so they get their online CEU credits. You've probably also sat through discussions about what to do if someone manages to finish in less than ten hours. And you may have experienced this effect when you went through online traffic school. Many of them force you to be online for some amount of time.
Most often the simple answer is to make sure that you have enough audio or video to ensure seat time is met. Oh, and you disable the next button. In other words, you ensure that the design of online learning enforces CEU credits based on seat time. But what if you don't want to do that? What if you have something that is text-based? Or exploratory?
Are there common measures for time equivalence not based on time online or seat time?
Problem 2 – Reduced Time for Online?
I know that many of you reading this want to jump to the part where we convince the organization that they are wrong to base their standards on number of hours. However, most of us are well aware that in most cases it will not be possible to get a large complex organization to change it's standards, so we have to work within the standards.
At the same time, we know that:
- We can do things like speeding up the audio without loss of comprehension.
- Variability of learner pace means that many learners can learn the same amount in less time.
- We can often teach the same concepts faster in online Learning / eLearning.
So …
How can we convince the organization that our variable length eLearning is worth X CEU Credits even when seat time may not come out to the set amount of time?
Problem 3 – Collaborative, Informal, Social Learning?
Now the hard part. We know that forcing someone through 10 hours of courseware is probably not the best idea, especially if we want them to learn a lot and have a good experience. We can certainly bring in a social element through a webinar (and these have the nice property that there are time lengths). But what about using other kinds of social learning.
Can we effectively measure the time equivalence of collaborative, informal, social learning?
How can we convince an organization that participation in collaborative, informal, social learning experiences equates to some amount of time equivalence?
Problem 4 – Influencing the Organization?
Again, in most cases, we won't be able to actually change how the organization measures these things. And when you look at from their perspective, they don't want to write up the specific content that must be covered, because it's too broad and will change. Thus, they are really just saying – you need to go through X amount of learning.
Are there other good ways to define X amount of learning?
What have you seen that are good ways to handle online CEU credits?
What influential examples exist that might help us influence the organization?
15 comments:
HUGE issue! You are spot on to press and to urge addressing the issues you raise...especially informal learning.
By engaging in such a conversation with a client, we are giving credence to the false notion that time on task (or in front of a screen) is directly proportional to performance or learning.
This question reminds me of Gloria's Gery's suggestion to weigh students as a measure of training effectiveness.
It would be good to measure a learner's engagement in an online discussion..in addition to time spent taking elearning. E.g. Number of posts a learner authored in an online forum or number of expert questions answered (LinkedIn measures this).
Harold,
I agree. The issue for many of us is that the CEU issue is not going away anytime soon. Many of our clients and ourselves need CEUs (or something similar) for recertification or, I understand, to keep jobs.
If we have to live with the system for now, how can we deal with it?
John, as an independent consultant, I guess it's part of my job to change the mindset, because I'm paid to give the best advice possible to my clients, even if they don't want to hear it. That's the difference between a consultant and a contractor.
Changing the system from within is difficult and really depends on the culture of the organization. Showing alternatives that work, such as results oriented work environments (ROWE), might help.
This is a definite problem, and I've been in front of such examples in the past. In my experiences with these sorts of elearning deliveries, each slide had to be a certain percentage complete before it would let you advance.
In my Masters program with an online university, factors such as attendance and immersion in the material are measured by required discussion participation. I think this is a good - though itself imperfect - model for handling the variability with which learners get through the material. In these required discussions there are usually those that answer earlier in the week than others, and those that answer right at the deadline. The quality of responses is part of the grade, and you must pose challenging questions to your peers as a means of not only exploring and understanding the content yourself, but as a means of helping others refine their work and practice. A learner will not get graded well if they simply answer "good idea!" or "good job!"
From week to week there is usually more than one discussion topic, each about a different element of the week's readings and practice, and many have a series of questions posed by the professor that must be answered in a compositional fashion (i.e. not simply providing direct answers to the question, but writing a piece that integrates and relates all answers together). Each discussion topic also requires responding to at least 1-2 co-learners, if not more.
I think this model is a good one to handle the variability in learning styles among individuals. The work in executing is involved in getting the moderator or instructor to monitor a discussion frequently and to make it part of their process.
As far as convincing powerful stakeholders against having timed training, I think it is possible to sell them on the idea that it is illogical to make people spend more time on content if they don't need to, especially when said individual can get back to the task of making money for the company, rather than sitting in a classroom or in front of a WBT for longer than is needed. Highlight the competing standards and let the decision maker choose: Have workers learn and enter the field as efficiently as possible, or make them stay so someone else's hours quota looks good in a report to executive staff.
Harold, the issue is not my consulting clients. The issue is the certifying organizations (e.g. PMI, ASTD). The people at my client organizations need to comply with those programs. I wish I could convince those organizations, but they are not my consulting clients. I am working directly with one of them now, though, to try to change their method of assessing continuing education. I have already convinced my clients that the concept is wrong, but until they convince the organizations that issue certifications, they need to accumulate CEUs or similar.
Part of the problem is likely the US P-20 education system which is focused on time and not on performance. When working with our State's education system, I've tried hard to change that.
Paul, I think it is a good model, too. It is, in fact, very close to what I've suggested to clients.
Tony, I'll take up your 2nd question first, as a way to address the first problem. You ask "How can we convince the organization that our variable length eLearning is worth X CEU Credits even when seat time may not come out to the set amount of time?" By *also* defining an objective outcome (ala Mager) of the seat time, and then testing for that. It's a stealth way of highlighting the problem with time in seat!
But Harold's right, they're measuring the wrong thing, and making equivalences just doesn't work. Social activities can makes the learning stick better (if you do it right :), but it's not an equivalency!
And it's the #$%^& stupid law, and the #$%&# association that have it wrong, and that's a hard row to hoe.
My best suggestion is to figure out what the average person could do in two interactive hours, and create an assessment for that, and then see when they can pass it whether or not it's 2 hours.
Actually, I think you are addressing the wrong issue. Why must "seat time" be equated to "learning". If there is a poor teacher who may or may not get through all the content in a face to face training, is this the same learning as a great teacher who was effective in teaching the content?
The groups I am familiar with will start with the same curriculum, content, and measures of learning, but then change the methods of instruction based on the mode of instruction. They then get these approved by the accrediting organizations FOR THE CLIENT. I agree with John, it may not be the best, but it works for your clients and the accrediting organizations until we can change their mindset.
I also noticed that no one brought up the assumption that elearning was less time consuming than face to face training.
What about this- Document ISD task/skills analysis, objectives, training plan, assessment plan and course evaluation/improvement plan.
Then award credits based on the registered plan, data, and defensible assessment results?
That type of option is available for pilot training under FAA AQP. Credit for the program and learner ability to master, NOT just for butt-hours of training exhausted.
FAA AQP background- http://www.faa.gov/training_testing/training/aqp/
There's another potential argument against CEUs as a dominant unit of measure: they encourage spending the least possible amount of time with content rather than integrating it into one's professional practice and everyday work.
Clark's suggestion that we calculate how long it takes for the average person to develop proficiency, then assess that proficiency regardless of their engagement, is a step in the right direction. But developing a test like that - one that can't be "gamed" - for every new subject that must be taught would be an epic undertaking.
I think Paul makes a great case on the efficiency side, that people should only spend as long learning something as it takes to learn it. This is a simple-sounding lesson that the industry as whole should take to heart: it takes a certain length of time to teach something, and another (often similar though not identical) length of time to learn it. When you fall on the short side of either, the learning suffers; when you fall on the far side, the process is inefficient.
CEUs complicate this process, especially when you're using interactive or immersive tools for teaching, like simulations. Sure, learners will spend some hours in front of a computer playing. But if they learn something about breakeven analysis, and their mind returns to the principles of the sim when they're applying that to their work, can we claim that as part of the "continuing education?"
The best educational experiences blur the lines between learning principles and applying them. Paradoxically, these experiences may be punished by the CEU system because they encourage the mindful application of principles when the learner is working, rather than during inefficient seat-time from which learning doesn't get carried over.
This has been a very interesting discussion to see unfold. I've tried to stay out of it a bit to see where things went.
I'm wondering if part of the answer is that unless the customer is willing to change their policy around length of time, then there's no argument to be had?
From my experience in working on these kinds of things, rarely can you get anyone to agree to a different standard. It's just too hard and we are not involved in that part of it.
That said, the suggestion of a few competing approaches is good. Have people seen time-based standards that can be met with something online that is not time-based? Or that can be met through social learning?
How are professional CEUs determined where sometimes reading an article and the writing about it or answering some questions is acceptable?
I'm sure there is more that we can do to help with the original problem - which we know is quite real.
Good point Tony regarding "willing to change their policy."
Rumor has it that things like AQP have met resistance from some collective bargaining organizations. Some bargaining units have prenegotiated fixed hours/days/weeks for training.
Even if the "CEU" is met in less time, the bargaining unit may demand pay for the allotted time.
This defeats a common perceived benefit for businesses; more efficient training to increase productive time, as well as productivity.
Great post (and comments). My organization is struggling with this issue regularly with courses that need approval from NASBA. NASBA's guidelines haven't kept pace and really only address completely synchronous training (Webinars, VTC, etc.) and completely self-paced (where hours in course are the predominant measure).
I agree with Clark that the focus should really be on meeting the stated learning objectives. In your example of the California initiative, there must have been some idea of what would and should be covered in those two hours. So why not make that the standard--"All supervisors must understand a, b, and c"--whether that takes two hours, 10 minutes, or 6 days. I think it's helpful to get upfront consensus on what the objectives are as well as an accepted agreement that most people (I know, squishy) will need x hours to learn/understand/master those concepts. That measure then becomes your CEU.
Otherwise you get the nightmare that I saw with one of Virginia's approved online driver's ed programs (the online equivalent of the 8-hour class you take when you rack up too many points on your license). Because the law requires that the course be eight hours in length, each screen of the online course has a timer. You can't progress to the next screen until the timer says you can. The course, then, takes *exactly* eight hours. It's predominantly text-based, so if you are a quick reader, you may very well have 2-5 minutes between the time you finish reading the screen and when you can actually advance. Most learners switch to other tasks waiting for the timer to elapse. How's that for missing the point of effective training?
I'm surprised the state of Virginia just wouldn't have a video to watch that lasted 8 hours. At the end of each hour, they have an assessment based on the video (I am assuming that there was an assessment with the text based one). Not only is the text based one discriminatory towards those that may have reading, language or literacy problems, but I would think it would be boring as all get out!
Perhaps one way of getting around the "time" issue is to use video or podcasts that give a measurable time with more non traditional assessment (i.e. comments and discussions).
Post a Comment