On principle, practice, experimentation, and theory

On twitter today a brief conversation ensued about best practices versus best principles.  I’ve gone off on this before ( I think Dilbert sums it up nicely), and my tweet today captures my belief:

“please, *not* best practices; abstract best principles and recontextualize!”

However, I want to go further.

Several times recently I’ve had people ask for research that justifies a particular position. And at a micro-level, that makes sense.  But there’s little ‘micro’ about the types of problems we solve.  So I hear it at a larger level: “why should we make learning more scenario-based”, or “what is the empirical evidence about social learning in the organization”.  And the problem is, you can’t really answer the question the way they think you should be able to. On principle (heh).

The problem is, most empirical research tends to be done around very small situations: these 3 classrooms were trialed in this state or province.  In many cases, there just hasn’t been the specific studies that are close enough to make a reasonable inference. And it’s hard to coordinate large studies that are really generalizable for pragmatic reasons that include logistics and funding.

What’s done instead, when sufficient cases arise, are meta-studies (as the recent one that said online learning was somewhat better than face to face), that tend to look across research, but you need a sufficient quantity of comparable studies (and someone capable and motivated).  Or, you can point to long programs of studies that are based around theoretical positions (e.g. John Sweller’s Cognitive Load Theory).  And expert practitioners typically have created  or procedures across long experience that can guide you.  In any case, you’re making inferences from a variety of studies and models.   One of my favorite models (Cognitive Apprenticeship) actually came from finding some synergy across several bodies of work.

So what’s a person to do? Sure, if you can find that specific relevant experiment, go for it.  Otherwise:

  • look to what others do, but don’t try to immediately adopt their practices, look to find the underlying principles and adapt those,
  • look to theories folks have proposed, and see how they might guide your approach,
  • bring in someone who’s had experience doing this,
  • or, think through it yourself, conceptualize the relationships, and determine what should be appropriate approaches.

(Note that the latter likely will take longer.) This is a ‘design-based research‘ approach, and to continue you need to trial, evaluate, and refine. Please do bring your reflections back to the conceptual domain.  We need more transparency!

The point I’m trying to make here is that, particularly in the learning sciences (e.g. when you’re working with the human brain), the properties aren’t as predictable as cement or steel; there is a bit of ‘uncertainty principle‘ going on (studying it changes the situation), and your intervention can very much affect how the individual perceives the task and possibilities.  You should expect to do some iteration and tuning.  And your bases for decision will not be individual research studies, by and large, but frameworks, models, and inferences.

Still, it’s systematic, based upon research and theory, and the best we can do.  So what are you waiting for?

Link to original post

Leave a Reply