How to evaluate social and informal learning

Dan Pontefract had a great post on TrainingWreck about the inadequacy of the Kirkpatrick model in a world where learning is increasingly collaborative and networked. Dan takes the logic further than I did last week in my post on how to assess learning initiatives. In brief, the Kirkpatrick levels are good for events, not processes. Increasingly, learning is migrating from events (think “workshop”) to processes (think “social learning”). Kirkpatrick is about push, not pull, learning.

I’ve been lobbying hard for injecting learning into an enterprise ecology, something I call a “workscape.” A workscape is not a thing; you can’t see or touch it. It results when work and learning are fused together. You learn the job by doing the job. We already know many ways to make networks more effective. That’s our new responsibility, baking ideal practices for learning into the system. For example, take one aspect of a purposeful enterprise network, profiling.

Profiles are good for learning and for business operations. Profiles enable members of the community to search for answers to questions, to expertise, and to the wisdom of experience. Of course, profiles also help out when you can’t find Sally’s phone number or who we have stationed in the Falkland Islands. You can’t isolate the effect profiles had on learning because the learning is intertwined with the business.

Evaluating the workscape

What to do about evaluation? Shouldn’t we include measures of the effectiveness of the workscape? Why do we need to evaluate at all?

This is not about ROI or other assessments to show the value of training and justify further investment. No, you evaluate a workscape to find out where you can make improvements.

Social networks are inevitable in enterprise. That’s a given. Social networks are like telephones. There’s no question that the corporation is going to provide phone service. Nobody’s going to come around and say times are tough, so we’re going to cut the phone budget by 30%. What you can do is evaluate the system in order to make improvements. There is plenty of data: the duration and distance of every phone call is recorded. Analyzing that data, you might notice that your offices in India are not taking advantage of our internal VOIP capability. Perhaps you’d see that sales force in North America no longer uses land lines; everything is on their smart phones. These are the sorts of measures you can act on.

What kinds of things might we use to evaluate the effectiveness of our social networks?

At the front end, we might do the online equivalent of the smile sheet. However, our questions would focus on ease of use, understanding, and experience using this or that new feature. We’d drag a few people into the lab and watch them use the workscape for learning, noting where the entire network could be improved.

In process, we might look at the analytics surrounding our network. Are some nodes not participating? What if we discover a mere handful of our employees in Switzerland logged onto the network last month? What if that were happening in Zimbabwe? The Swiss situation is probably a management and cultural issue; in Africa, the issue may be bandwidth or availability of terminals and PCs.

Network analysis

Social network analysis provides another level of sophistication in analyzing network activity. Social network analysis lets us look at groups. You might be concerned if your salespeople on the West coast weren’t interacting with their brethren in the East. You might find that 95% of the traffic in that outfit you acquired is internal, that the newcomers are not becoming integrated into the corporate culture. Also, by adding qualitative measures to the analysis, you could pinpoint toxic nodes (people no one wants to deal with) and other individual situations.

Value network analysis could yield economic information that social network analysis fails to cover. Tracking economic flows can highlight opportunities that lead to changes in organizational structure. This one is clearly more on the business side of the workscape, but transitions inevitably increase the need for learning, so we’re peripherally involved. More thoughts along this line in Jon Husband’s and my article titled Not Your Father’s ROI.

At the end of the day all of your analysis and resulting actions better be making a significant contribution to the bottom line. Do double-check that you’re on target, see How do You Assess the Value of Learning.

Link to Original

Leave a Reply