As my regular readers know, last week I was at an Assessment Institute. What has always been odd to my mind is how much effort is often spent selling Assessment.
I refuse to sell, market, or otherwise carpetbag for Assessment. Note the capital A.
Do not get me wrong; Victoria College does assessment and Assessment, will continue to do both, and I will be proud of my record as a coordinator of the latter and a practitioner of the former.
The problem with selling something, whether a set of professional linguistics, rote spells or recipes sold off by master practitioners (or charlatans), or any other facet of Assessment, is that you cheapen your product.
In my opinion, assessment does not need selling. Do you believe in self-improvement? Do you believe you have the power, the innate ability, to make your part of the world, no matter how large or small, a better place? If you answer yes to any of these questions, then the instant follow up is: “How do you know if you’re making a difference?” And in that case, there’s no need to sell assessment. Because assessment is just a buzzword for the scientific process of research. And I dare say science is humanity’s boldest effort to utterly master the world around us.
If you answered ‘no’ to any of those questions, I do not need to sell you assessment. I need to give you hope. Hope that no matter how dreary the circumstance, no matter how ghastly the storm, you do in fact have extraordinary abilities to make changes. I also have a book I need to give you to read. So, if you did answer no, please come see me.
Back to Assessment. It is a sometimes not-so-well explained, not-quite-perfectly executed set of trending-towards-actionable numbers designed to appease legislatures and accreditation agencies. I think this is where the apologetic sales pitch comes into being. People realise that far too much of Assessment is summative. Thus, the apologists leap into sales mode because they intuitively understand assessment is required.
On the other hand, there are professors who do assessment all the time, but, when they see the trappings of Assessment, like good iconoclasts, they have natural scepticism to the obvious herd mentality, and the, ahem, rubrics, one tends to see in such activities.
I want to close off this week’s post with two important thoughts that I bring back from this Assessment Institute.
The first is that Assessment, that legalese bureaucracy that does help us tell our story yet is quite tedious, is going to change. There are some new words that will be required. There’s going to be a change from formative to summative. This is driven by state and federal legislation. I feel better prepared to make the case for Victoria College now. It may not be glamorous; all the same, this is important work. And Victoria College is worth it. But I am not selling it.
The second is a very cool bit of psychometric validation technology. For the first time, I met a fellow user of Shiny at an education conference (although I’m sure there must be more of us out there). He developed some quite nice code that makes it very easy to measure and rate inter-rater scoring (think the type of scoring that happens when one uses a rubric). Dr David Eubanks gave an amazing talk about these inter-rater facets, and I was quite glad I picked his session. Plus, since he’s an applied mathematician, I may have just found my new hero for assessment.
Do you use rubrics with multiple raters? Are you curious how reliable your rubrics or raters are? If you’d like to talk, drop me a line or come visit me in CAPE 110!