Wednesday, January 31, 2007
Tony Bingham, President of ASTD, is doing his introduction, and his content is: Web 2.0, personal learning, easy distribution of software (software as a service), video is here now, push to pull, teaching Gen Y (what he calls the netGeneration). What a great setup for my presentation. Thanks Tony. And, it’s somewhat surprising to see a relatively slow moving organization, ASTD, driving this discussion.
One thing that struck me was his point of relevant vs. perfect. He was part of the team that worked on Britannica Online. He openly questions whether the goal of perfection (think as compared to Wikipedia) should be the primary. To me this relates closely to the question of speed vs. quality.
Thorton May is now up. He has verbal ADD. He only gets half way through a sentence and stammers a few words and then starts the next. A couple of major points: The number one skill set of the future is the ability to learn. Everyone needs to go back to school (interesting that he assumes that going back to school will help).
Friday, January 26, 2007
First - I tend to agree with Donald that most Web 2.0 activities are more independent actions of individuals than true collaborative work (team work) on a work product.
But - I think Donald's missed the point that you can have independent activity that builds on itself to form a type of collaboration. This is true in most project teams. You don't have everyone work on the same part of the document at the same time. Rather, you divide the work, and bring it together.
Donald has missed some things. For example, he asserts - "Even Wikipedia is written by separate individuals, not collaborative groups." Okay, sure, there are individuals contributing to an article, but if several people edit the same article (such as the definition of eLearning 2.0), is that somehow not collaborative? I don't get it Donald?
Another quote from the post:
I’ve never really bought the idea that I’ll learn from a group of learners who know as little about the subject as me. I don’t force my twin son to learn
French from his equally poor French-speaking brother, and am now convinced that much of what passes for groupwork in primary and secondary school classrooms is just chaos.
Donald - do you think you'll learn French without speaking it with other people? Have you been in an MBA course that uses case studies? Yes, you do need intelligent individuals to discuss the case with. But, wow, I think you've really missed something.
I also wonder if Donald's bias is shared out there.
I'll be very curious to see if Nancy White or Stephen Downes respond to this. They'll likely be even more animated in their response.
The HBR List - Breakthrough Ideas for 2007
FYI - I read somewhere that the article would only be available for a limited time for free.
Tuesday, January 23, 2007
Individuals have blogged about conferences for some time. It seems now that more conference events are sponsoring these blogs and trying to aggregate those blogging the conference. It is a way to start the event on a virtual basis prior to the start and then continue the conversations afterwards and provide an archive for what happen and what people thought about it.It made me think back to an earlier post Be an Insanely Great Professional Conference Attendee. The biggest idea in that post is to come prepare with interesting questions to ask other attendees that would spark conversations. And, Bill's post made me realize that likely there were people talking about the conference in blogs prior to the conference. So, quick search found the following:
Monday, January 22, 2007
Friday, January 19, 2007
- They got rid of the different versions - especially getting rid of RoboInfo vs. RoboHelp makes life easier to understand as a customer.
- They made it easier to integrate Captivate movies. This is something we've done before when creating Reference Hybrids.
- They integrated a screen capture utility.
- They've integrated a source control system for shared development.
We used to use RoboHelp and RoboInfo quite a bit to build reference systems. However, over the past year or so for projects that were delivered purely as web pages, we found ourselves using Wikis instead. I'm not sure if the new features will change what we use here or not.
First - a lot of the complaints stem from the word "Quality" (and even more so, the definition of a "Quality Learning Experience" in the original post). Some of the responses say that Quality is "changing a person" or "achieve a performance goal" ... I hate to say it, but these folks are theoretically correct, but they are ignoring the reality of most situations. I'd suggest that Quality gets defined by the client (see What Clients Really Want). The reality is that while we are aiming to achieve the theoretical definition of quality they give, the actual definition of quality is negotiated with the client at the start of a project and their definitions fail to take that into account. However, while I think that they've provided a short-sighted definition of quality, I actually don't disagree with the fact that the industry accepted definition of what makes something a Quality eLearning piece (e.g., wins awards) is really off-base. Just because it has lots of interactivity, looks sexy, etc. doesn't mean it's really meeting client needs. Could something that was produced much more quickly at a lower cost have achieved the same thing or even more for the client? Maybe? But it wouldn't win the award. We may not like this, but that's the reality.
Second - there seems to be disagreement that speed vs. quality is a trade-off. I don't believe we've overcome the old adage of "fast, cheap, good - choose two." Yes, the cost of development for some kinds of eLearning Solutions has been reduced considerably over the past few years. Yes, I believe we've matured as an industry where we are looking at focusing our higher-cost solutions at the most important parts of the intervention and using lower-cost, support type solutions in other places. And yes, there are types of solutions that have a dramatically different cost-effect curve. But, I'd like to meet the person who feels that they aren't forced to make hard decisions about where to spend their time and money on solutions - and that doesn't feel they could create a solution that would have greater impact on performance (their definition of quality) if they had more time and money. After all, if we hired individual professional coaches who were experts at the job and stuck them 24x7 with the person to guide them, wouldn't that achieve a better result than what you did the last time?
Thursday, January 18, 2007
Wednesday, January 17, 2007
- In September 2006, IT employment stood at 3,667,100 up 4.2% from 2005 (an all time high)
- 79 percent of IT workers work in IT-reliant companies (health care and financial services, industries enabled by IT but not focused on IT)
- From 2000 to 2004, the number of incoming US undergraduates planning to major in Computer Science dropped by 60%.
- Estimated 1.5 million new computer and IT related job opening between 2002 and 2012
Based on my personal experience in hiring people in Los Angeles and based on the complaints of my fellow members of the Los Angeles CTO Forum, this already has real impact on us. We simply can't find people to fill positions. And off-shore is not always appropriate. I've talked about this issue before in: Computer Science Dying in the US?
Tuesday, January 16, 2007
- It would be interesting to me and likely to other readers of this blog to see who is attending who also reads this blog (and presumably others). Please drop a comment if you are going.
- On the 31st or 1st we will be having a dinner with some interesting folks to ramble on about topics that are discussed in this blog. If you are interested in going, include that in your comment.
- My sessions are on “Personal and Collaborative Learning Using Blogs and Social Bookmarking” – one is an introduction, the other two are hands-on. It would be nice to know if any readers of this blog are planning to attend.
Monday, January 15, 2007
It will most likely will not be a destination. It is there when you need it and gone when you don't.I almost agree with Lee on this, but I do think there will be destination or at least a guide that can be brought up to provide suggestions about learning opportunities, resources, additional content that relates to what you are viewing.
It will track or pull data from many different web services. i.e. your feedreader, flickr, or youtube.I agree that it will be somewhat agnostic about the content that will be considered part of learning, but there will be a big issue about tagging pages and events that are considered part of learning. It won't be sufficient to visit any page about a topic in order to get "credit."
It will not be an HR system.Not sure I get this. It can be driven by HR, learning or a business unit.
It will both generate and store a lot of the meta-data about learning.I completely agree with these comments.
It will allow other systems to integrate workflow, learning content, and social interaction on top of it.
It will foster single user adoption and empower that user within any environment.
Brent raises some interesting questions about what it will look like. He refers to it as a Learning Dashboard. But I believe it can be a destination, i.e., a learning dashboard, but it should also be capable as being a guide as you complete other things. In fact, when you look at the discussion around Web 3.0, part of it is showing what relates to what you are looking at. There's an opportunity to do this based on learning paths within limited context.
Dave and I are still trying to figure out exactly what this means, but I just ran across a couple of others that got me thinking about this again: The Philadelphia eLearning SIG Blog and The Seattle Captivate User Group Blog.
Both blogs serve the needs of their community. In discussing the blog with Ben Cragio from Phlesig, he says that this looks to be a great tool for the community:
1 - Cover more topics than we could possibly cover in 6 - 12 annual meetings a year.This makes so much sense, especially given how easy (and free) it is to get blogs together.
2 - Item #1 allows us to respond to the specific interests of our group. As a member of our e-SIG (and this is true in most technology user groups), it's quite likely that a topic you might be interested in, that talks directly to the work you are doing or where you are going, simply won't be addressed over the course of the year. Or it may be addressed too late.
3 - Get more members involved with the e-SIG.
4 - Increase the stickiness of the e-SIG by allowing people to maintain and add
connections in between meetings.
5 - Continue the dialogue in between meetings.
6 - Help steer better presentations to our physical meetings. Comments, voting and stats provide excellent guidance into what topics are in demand.
7 - We're an e-SIG so we should be using e-Learning! This blog is an informal e-Learning tool.
8 - Expose our members to blogging as a method of e-Learning and allow them to
judge for themselves where it fits into their toolkit.
9 - Hopefully gain perspectives from professionals outside of our group that we
wouldn't normally here.
Have I just been missing it? Should I have created the Los Angeles eLearning SIG Blog months ago?
It would seem that the critical ingredient is participation. Ben obviously is really helping the Phlesig take off. But it does seem like we may all be missing an opportunity. Thoughts?
Thursday, January 11, 2007
I personally think that the LMS is quickly being relegated to longer, developmental activities and compliance activities. I've talked about some of these problems before:
- LMS Dissatisfaction on the Rise
- Do You WANT an LMS? Does a Learner WANT an LMS?
- Moving from One to Many - LMS Products are Two Generations Behind
- Leading with an LMS - Harmful to Your Health (or Skipping Stages ...
- Tools for On-Demand Information - An LMS?
When I looked at what Lee and Mark had to say, I find myself nodding my head in agreement that there has to be a different kind of LMS that will emerge. It will be much more like a combination of Communication, Search Support and Web Analytics (for tracking), and less like the current self-contained entities that are today's LMS. It won't get in your way when you want to get to information, but may come up along the side to show you other resources. It will track your access of content of various forms without login, provide suggestions for additional content, suggest learning paths based on what you are doing. It's going to be on-the-side and below your activities, not on-top.
I here a lot of what I'm saying in what Lee and Mark had to say. I'll be curious to see if anyone is actually trying to make this happen.
Of course, this started long before me and has gone through virtually all parts of the blogosphere. It's fascinating to think about the implication of this kind of idea spread. It's also fascinating to realize the speed that ideas can now move.
Tuesday, January 09, 2007
I don't make things difficult. That's the way they get, all by themselves.
I've previously asked How People Interact with Blogs? about basic interaction with blogs and received some interesting responses.
However, as I've been using CoComment more, I'm beginning to believe that most blog readers come to a blog and see the comments that exist at the time they get there. Even if they leave a comment, they may never return to see any others. As a blog author (and commenter) it makes it hard because I don't know if the person will see my response.
So I'm hoping people could tell me:
a. Do you read comments?
b. Do you ever go back to a blog to see comments later?
c. Do you use a mechanism such as CoComment to track conversations?
d. Do you ever leave a comment and not come back to see what was said?
FYI - I'm still trying to help this out via an on-going discussion list via CoComment. Find out more at:
eLearning Technology: eLearning Discussions - An Attempt at Better Discussions in the Blogosphere
What are the trade offs between quality learning programs and rapid e-learning and how do you decide?There's a lot involved in this question and I plan to revisit it several times during the month. I'm hoping to see some contributions that will help thinking.
Let me start with some important words of wisdom that I like to use at meetings:
You rush a miracle man, you get rotten miracles.Props to the first person to leave a comment that identifies the source of this quote.
Only slightly more seriously, consider the following graph that I often use in presentations to illustrate a point:
Expenditure on training has a limit on the performance gains you can achieve. After a certain point, the cost of the Training exceeds the value of the performance gains. And often there's an minimum level that you can do where anything below that level would cause too many problems. This graph is completely over-simplified to make a point. In reality, there are many different ways we could make our expenditure and each one would have a different cost and effect on performance.
So, the real challenge posed by the Big Question is knowing when its really worth it to spend dollars on what we might consider a higher quality solution than providing something simpler that we know won't be as effective at improving performance.
And this isn't a theoretical question - it's something we face all the time. We are pretty sure that people will learn less from a rapid eLearning piece that's basically just a PowerPoint + Audio as compared to an hours worth of fun, interactive courseware. But, if the cost is significantly higher, is it really worth it? In what cases?
And it's not just an ROI question. Often, there's a lot more to What Clients Really Want than business outcomes that can be quantified into dollars.
In my mind, there's clearly no easy answer to this question. I'll be curious to see some of the ways that people attack this over the next few days.
Monday, January 08, 2007
Let me start with something I consider to be almost misleading in our industry. We hear all the time about how we SHOULD be doing Level 3 & 4. Will just said in his post Assessment Mistakes by E-Learning:
Stunning: Even after all the hot air expelled, ink spilled, and electrons excited in the last 10 years regarding how we ought to be measuring business results, nobody is doing it !!!!!!!!!In Approaches to Evaluation of Training: Theory & Practice the authors state.
Evaluation becomes more important when one considers that while American industries, for example, annually spend up to $100 billion on training and development, not more than “10 per cent of these expenditures actually result in transfer to the job” (Baldwin & Ford, 1988, p.63). This can be explained by reports that indicate that not all training programs are consistently evaluated (Carnevale & Shulz, 1990). The American Society for Training and Development (ASTD) found that 45 percent of surveyed organizations only gauged trainees’ reactions to courses (Bassi & van Buren, 1999). Overall, 93% of training courses are evaluated at Level One, 52% of the courses are evaluated at Level Two, 31% of the courses are evaluated at Level Three and 28% of the courses are evaluated at Level Four. These data clearly represent a bias in the area of evaluation for simple and superficial analysis.Maybe the authors didn't mean to say this, but they clearly say that 10% transfer is because of inconsistent evaluation. And clearly we are all biased towards the "simple and superficial." Yikes, I feel so dirty not doing Level 3 & 4.
But wait, we've all had the experience of when we try to do Level 3 & 4, we face considerable push back from internal clients and line managers. It's not that expensive to get the data nor really that much work, but it does take commitment. Obviously, the level of effort isn't worth it to the line managers. And this will be a topic for a future post, i.e., why don't clients care? But for know recognize that when you start with a client, you need to do a quick assessment of what they really wants:
Do they really care about effectiveness: changing behavior and driving business results? Many clients don't really care about this. They have a particular product/project in mind and they want you to get that done. As Karyn said in her post, the client will tell you, "Don't worry, your job is safe." They don't care about business outcomes. You'll often find this out pretty quickly when you start asking questions about "What do you expect people to do differently after this intervention?" or "What numbers are we trying to hit?" A blank stare often indicates that they don't really care. I actually think a surprising number of "training" projects involve clients who are on the don't care end of the spectrum.
Do they care about the looks of the product/project? Many clients come in with expectations about what will be produced and likely around what a "good looking" project will look like. They may be looking for highly interactive, or fun, or engaging, or games, or simulations, or ... If you suggest that a checklist might actually have better results than an hour-long, interactive training course, you may get lots of push-back. Some clients honestly don't care about looks, but most do. In fact, its much more common for a client to care about looks than to care about personality (I mean business results). Good looks are important in that it often implies user engagement, but this post is purely about assessing what the client really wants.
Remember that beauty is in the eye of the beholder. You definitely need to understand what they believe will be a good looking project. It may include what kind of format, graphic design, level of interaction, etc.
Also keep in mind that your assessment of what the client wants may also need to include thinking about other stakeholders (end-users, the client's boss, etc.) and what they want. Clark Aldrich recently suggested that helping your client get a promotion might be the ultimate goal. Thus, the "wants assessment" in that case needed to include assessing what your client's boss really wants. Do they care about business results? Do they want something good looking?
Finally, please, please, please do not show this graphic to your client (or ask the implied questions directly). Every client will tell you that personality is much more important than looks. Of course, they want it to be effective! Of course, it doesn't matter how good looking it is! They know that's the right answer! That's what they will tell you if you ask. Your job is to find out what they really think.
Stay tuned for more ...
Friday, January 05, 2007
- Blogs and Community – launching a new paradigm for online community?
- eLearning Discussions - An Attempt at Better Discussions in the Blogosphere
- Move from Discussion Groups to World of Blogs?
- Top Ten Reasons To Blog and Top Ten Not to Blog
- I found it inside my blog reader!
- Blogs vs. Discussion Groups or Mis-Understanding Blog Reading and ...
And this is a case where it's only one blog. In our world, there are many blogs. So, it's even more complicated of an issue.
Thursday, January 04, 2007
And, yes, I know I'm a week late, but since you've now violated half of your resolutions, you can replace those with a couple of these. :)
So, here are my suggested resolutions:
- I will find a project even if I have to do it on top of my current work where I can make a difference in business outcomes that truly matter to the business, already get measured, and will do my best to really make an impact on these outcome.
- For the rest of the projects, I'll find out up-front where on the spectrum we are from checking-the-box to really impacting performance and will make sure that I act accordingly. In other words, I'll be passionate about impacting performance when that's really what's being asked for. I'll try to minimize the amount of time I spend on check-the-box training.
- The next time I'm asked to produce an eLearning course, I will figure out the least content that I could present up-front to get people started and move the rest of the content I will present as a Reference Hybrid.
- For the little bit of content that I do create as an eLearning course, I will spend at least 30 minutes brainstorming with colleagues on how to make it interesting, relevant and fun. I'll consider games, simulations, compelling storylines. I'll think like a novelist, movie producer, someone who cares about engaging the audience.
- I'll collect a list of resources such as Clark Quinn's - Seven Steps to Better eLearning that I will use to challenge my eLearning course design. (What else do people use?)
- I'll help my boss understand that they aren't Captain Piccard and that the enterprise is not the Starship Enterprise. "Computer make it so" doesn't yet work in this day and age. Even with Rapid eLearning and Ruby on Rails.
- I won't do shovelware unless its required, then I'll figure out how to get other people the tools to shovel it out.
- I will propell my own learning forward by starting my own blog and participating in the Blogosphere.
- I will avoid featuritis when selecting tools especially an LMS.
- I will get smart in eLearning 2.0 tools for personal and group learning and will help at least one group use them during 2007.
These are only suggested resolutions. I'm sure we'd all like to hear your actual resolutions or suggestions for additional resolutions. Either that, or tell me I'm nuts.
And once you create you actual resolutions, I would highly suggest you share them with your boss, your colleagues at work, and, of course, all of us (your peers).
As a bonus Top 9, I reformatted and edited some of Basex's Email Resolutions so that I could circulate them internally.
- I will refrain from combining multiple themes and requests in one single e-mail.
- I will make sure that the subject of my e-mail message clearly reflects both the topic and urgency of the missive.
- I will read my own e-mails before sending them to make sure they are comprehensible to others.
- I will make sure that the first two sentences of the email make it clear what the subject and required actions are relative to the email.
- I will make my emails as short as possible.
- I will not overburden colleagues with unnecessary e-mail.
- I will reply judiciously, avoiding one word responses such as "Thanks" or "Great!"
- I will be more conscious of the fact that "Reply to All" sometimes really does go to all.
- I will not use email when the content of the communication has emotional aspects that won't be effectively communicated via email.
11% said they did NO evaluation
26% said they did Level 1 smile sheets
48% said they measured Level 2 learning
15% said they measured Level 3 on-the-job performance
0% said they measured Level 4 business results (or ROI)
His discussion further points out that doing Level 2 assessment immediately at the end of training (i.e., did a post course test ... likely multiple-choice) was not nearly as accurate as assessing later on.
The comments that follow are interesting, but I'm surprised that we aren't being more honest about this. I personally am a big proponent of using metrics throughout the solution (at analysis to define what it is we are really going after, as part of the intervention to provide direction, to assess the outcomes and make change), but often unless the metrics are already being collected by the organization (e.g., customer satisfaction surveys), then Level 4 will not happen. And, likely Level 3 - even though it can be approximated (cheaply) with a very simple survey won't happen unless those questions are already being asked other ways.
Why is it? Are we bad at our jobs? Do we not care? Hardly. But we must make choices about where we push. When you discuss a simple post performance intervention survey to assess how we did in changing performance with internal or external clients, the response you get is "they won't spend the time to do it" ... and they are generally right. Again, unless its part of the intervention itself (pre-post assessment) or part of the culture of the organization, this isn't a battle that's worth fighting for as compared to other battles.
Tuesday, January 02, 2007
- Wesley Fryer article that says that in 5-years, school district says - "not a single desktop in this 52,000-student school system in metropolitan Dallas will carry the image of a proprietary school software program."
Wow - dump your Microsoft stock now!
- Doug Belshaw post of his "20 top blog posts of 2006"
Let me ramble a bit here and then there's something really good at the end (at least I think so).
A really great comment came in during the holidays relative to the post: eLearning Technology: Move from Discussion Groups to World of Blogs? Craig said something that has troubled me for a while:
Blogs are not that great as a platform for discussion.I partly agree with him in that I've felt for a long time that the Blogosphere is a loose conversation and lacks the single location effect that you find in discussion groups. Craig highlights this with an example:
Case in point: If I have a tech question and I post it on my blog, my chances of getting a response are fairly low. If I post it on a blog where they talk about the tech I am interested in, my chances are higher. However, if I post to a tech forum, I am almost guaranteed to get a good discussion going.And, I would tend to agree. However, if I have an eLearning Question, I think I'm much more likely to get answers to it by posting in my blog right now than posting a question in a forum. Especially a question around something like eLearning 1.0 vs. 2.0 - Help Needed (and I got lots of great help). On the other hand, if I want to find out something more specific around ROI, I would definitely post on the ROI group. They may change the question entirely and answer something else, but I would be more likely to get a response.
Craig then comments:
I also think that the relatively low numbers of individuals in elearning who participate in blogs and discussion groups may be in part to the fact that:
a) They are so busy developing / launching their elearning programs they just don't have the time to participate outside of the job.
b) The number of individuals in the elearning industry embracing the newer technology is still very small.
c) The elearning industry as a whole is very competitive and they haven't warmed up to the idea of transparency like Scoble evangelizes. Sharing best practices may be frowned upon from a competitive / corporate standpoint.
Just sharing my thought. Am I completely off the mark? Would love to get a discussion going ;)
What's interesting about this is that its a case-in-point of what is wrong in the world of blogs. Craig may not have his own blog and if he did, likely he doesn't draw a large crowd (who in eLearning does?). How can Craig ask this question in the world of blogs and get a response? Also, there are all sorts of posts and prior conversation around this that relate, but how could this question and those poeple be included in the conversation?
Instead, what will happen is that because Craig asked his question on an older post (its at least a few days old now - gasp - so old), it will be buried only to be seen by the Blogger who may put in a response, but no one else will come in to discuss it. The only way it will surface is if the blogger (me in this case) highlights the post by creating a new post (oh I just did that, eh). If not, Craig will get a response from me only - so he won't get his questions answered - which proves the point - not a great discussion vehicle.
So, finally, to the good part. I've been a casual CoComment user for quite a while and have used it to track my own conversations, i.e., to see responses to comments that I make on other web sites. However, Dave Lee did an interesting thing with the LCB that uses CoComments to collect comments around the Monthly Big Questions so that we can keep track of it. That inspired me.
Starting today, I'm going to start to track lots of interesting conversations as I see them occur using CoComment (where CoComment works) so that a question like Craig's will at least surface to those who are tracking these conversations. In other words, as I find interesting conversations (IMHO) going on around eLearning topics that I'm interested in, I'll track the conversation. That way you can see a little bit of the conversation in the right column of my blog or you can visit my CoComment page. Better yet you can subscribe to the RSS Feed which should link you back to the blog and so you don't have to visit my CoComment page.
This is partly in response to comments in eLearning Technology: How Do People Interact with Blogs? where several people said that they visit the blog in order to see the comments. That makes sense, but it's not all that efficient. I hope this turns out to be useful.
A couple caveats - CoComment is not the cleaness or prettiest feed out there. It also doesn't really do a very good job of "threading" the discussions - you'll need to jump around to follow the discussion as it rambles through the blogosphere. Further, its not always easy to know where good conversations are happening, although normally people will post about them. So, it is what it is. Finally, this is still a mess. If someone else finds interesting conversations and links them via CoComment - how do you filter the overlap? But, its the best I can figure out for now.
Now a couple of questions (an assignment):
1. Is Craig right? Are blogs a bad vehicle for discussion? For asking questions?
2. Is Craig going to get his questions answered? Do you have any answers for Craig?
3. Any thoughts on better ways to surface the conversations that we all know are going on but that are happening in such as scattered way?