I posted this over a year ago before anyone probably knew this blog even existed. I am reposting it now for two reasons: 1) I have raised this critique since then and found that a lot of my Presbyterian Church (USA) friends agree with it. 2) The issues about assessment that I raise seem to have no answers. I posting this in the hopes that those currently taking these exams don't read it and then get upset off before they finish them up this weekend. That would not sit well and create anxiety that is not helpful. But when they are done, this is a reflection worth having. I passed all exams first try which was great, but I never knew what it actually meant. I am even less sure now that I know something about educational and professional assessment.
I think that the process lacks assessment rigor and needs to change if my assumptions here are at all correct.
My point is not so much about gate-keeping as it is with how we measure observable outcomes and expectations and how those standards are used to train ministers. Do the ordination exams have standard outcomes that are regularly reviewed? Are the five exams tested for their effectiveness to meet those outcomes on a regular basis? Are these results shared with Presbyteries in order to improve preparation programs? Are there standards for preaching that Presbyteries of care and search committees can use to determine best of fit or levels of preparedness? Is it that much based on not getting a 1 0r 2 rating on your exams? Moreover, what do those ratings mean? Are they tied to a standard rubric to measure standard outcomes? Do we know that these are reliable variables across reviewers? Do we test inter-rater reliability to assess the assessment instrument if it at all exists? I suspect that the rigor I am discussing did exist 10 years ago when I sat down for exams, and I suspect that it still does not really exist.
Look at how teacher education and evaluation is performed (among other disciplines). Look at Nursing. The issue is not so much the exam, but what it measures and how we know it. These disciplines use exams and field work, but all are tied to specific assessment variables. This is crucial and lives depend on it so they take these measures seriously. Why does it seem that ministry preparation is so haphazard in terms of assessment? Are we not dealing with the most intimate layers of human living as well?
My thinking is that all candidates should have a portfolio that is assessed and scored with an effective rubric or rubrics. This would balance exams with field work, and even variables that can be measured in an assessment that the Presbytery of care should administer about the candidate based on standard variables. This ought to include a sample of preaching and it can all be done electronically and exported to a CD for search committees to review so the assessment of the candidate is tied to their PIF. I think that the PCUSA should function as an accrediting agency over the Presbyteries to ensure they are meeting support and training standards for the candidates who are under care just like NCATE does with teachers. It also means that robust assessment should be coupled with training sessions and search committees to find what they most need and desire from a pastor. Yes it's a lot of administration and work for people to do. It takes expertise in assessment and good strategic planning, but don't our young ministers and struggling congregations deserve something more explicit and clear?
I simply do not know how assessment of preparation programs is implemented or how changes are implemented as a result. It just seems to me that such rigor does not exist. I have been through the process and looking over my materials now that I have had experience with educational and institutional assessment it just seems to be disconnected and thrown together far more than I see with other professional disciplines that require standards of their candidates. What I produced was a series of disconnected documents that are not tied to any measure of assessment whatsoever. The package does not give a coherent picture of the candidate, and it is not clear what this person's level of proficiency is in any given area because those objective proficiencies are no where mentioned and so, they are no where observed and assessed.
(BTW: A good assessment can also account for variations because you will be dealing with different persons who exercise different gifts. But it can do that while making sure that all candidates are meeting very specific and clear assessment variables. So while we think about standards, let's not block the discussion with the concern over "freedom" at the get-go. I know that it comes up immediately. I have been there many times before and am discussing that with different kinds of institutional assessment even at the moment.)
The point is that we are assuming that people will be on the same page with this process and there is no evidence that I know of, perhaps there are exceptions as with any rule, that there is any consistency with the process. Just as if I flip through dozens of Congregational Information Forms (CIF's), they all start to sound the same, imagine congregations looking at Pastoral Information Forms (PIF's) that all start to sound the same. Sameness and actually producing evidence of clear competencies, it seems, has been detrimentally confused. SO if we the PCUSA want to actually get serious about assessment rigor, sign me up to help. As long as we are OK if we need to scrap the entire process if need be, we can do great things for our young ministers by tying assessment to training.
Related posts:
