Matt Wittstein follows up with Mark Burnham from Season 1, Episode 8, “Demonstrating Knowledge Using Oral Assessments.” Mark shares his experience with using oral assessments in his Biology course, as well as some feedback he received from students and his future hopes with the new assessments.
See extended notes for this episode at https://www.centerforengagedlearning.org/refresh-demonstrating-knowledge-using-oral-assessments/.
In this episode of Limed: Teaching with a Twist, host Matt Wittstein follows up with Mark Burnham from Season 1, Episode 8, “Demonstrating Knowledge Using Oral Assessments.” Matt and Mark discuss Mark’s experience with implementing oral exams in his Biology coursework, the feedback and reactions he gauged from his students, and his future plans with oral assessments and other non-traditional methods of assessment in the classroom.
This episode was hosted by Matt Wittstein, edited by Jeremiah Timberlake, and produced by Matt Wittstein in collaboration with the Center for Engaged Learning.
Limed: Teaching with a Twist
Episode 11 – Refresh: Demonstrating Knowledge Using Oral Assessments
Matt Wittstein (00:10):
You are listening to Limed: Teaching With a Twist, a podcast that plays with pedagogy.
(00:23):
This summer, Limed: Teaching With a Twist is reaching back out to a couple of our guests from season one. This month, we chat with Mark Burnham from episode eight, demonstrating knowledge using oral assessments. Mark was implementing oral assessments in both his introductory and advanced biology courses to help graduating seniors prepare for their oral graduation assessments at Emory and Henry College. Enjoy the show.
Matt Wittstein (00:51):
We have Mark Burnham back with us from Emory and Henry College. Since we last talked to Mark, he's helped proctor his department's oral graduation exams and wrapped up his spring semester. Welcome back, Mark.
Mark Burnham (01:02):
Thanks, Matt. Glad to be here.
Matt Wittstein (01:04):
I understand you haven't really had a chance to implement a lot of what our panel suggested and what their ideas were. So I want to start by asking, what did you learn from proctoring the graduation exams? How was that experience for you?
Mark Burnham (01:19):
Yeah, it was a pretty interesting experience. I think the biggest thing that I got out of it was just the variety of student reactions and level preparation for different students. I wouldn't even say level preparation. I think it was more the variety of student comfort level and reaction when faced with that sort of situation. So there were students that were very nervous and there were some that were much more confident. And I think that the level of material preparation was relatively similar between most of the students. It was more how comfortable they were entering the room in that setting. So that was probably the thing that stuck out to me the most. All of our students passed and all of them did really well. It was just a very wide variety of reactions to the situation, reactions to the scene, and different styles of answering different comfort level between the different students. I think that was the thing that stuck out to me the most about that setting.
Matt Wittstein (02:23):
So if I recall, one of your learning outcomes of doing oral assessment was to help prepare them for this. So having experienced that, how do you think you're doing with what you implemented this spring with your earlier coursework and your more senior coursework?
Mark Burnham (02:41):
Yeah. I think that it went well. I hope that they'll be more prepared for the senior exams by doing these exams throughout classes. I implemented them in different ways in my upper level versus introductory level course, and I think that they had actually very similar levels of comfort with the exams between the two levels. The introductory level students may have been a little more nervous, but they still did really well with them. The upper level students, it was a mixed bag. Some of them really liked it and some of them prefer more traditional sorts of assessments. But regardless, I think that having them sprinkled throughout their college experience would prepare them better for doing it at the senior level, at least for the biology majors. And we'll see how that comes to fruition over the next couple years. I think that at least one, maybe more than one of my upper level students will be doing their senior orals next spring. So we'll see if they feel like they are more comfortable with that setting after having done orals during the class.
Matt Wittstein (03:50):
So with what you did with your classes, did you get any direct feedback from students this semester? I know end of your evaluations are probably out. There's usually some open answer stuff. Did you glean any bits of wisdom from the student perspective?
Mark Burnham (04:06):
Anecdotally, yeah, but not really systematically, no. I just popped up in my student evals to review them again and see if anyone mentioned orals. I thought I remembered some people mentioning orals, and two students mentioned orals. One said it was the most valuable aspect of the course. The other said it was the least valuable aspect of the course. So I don't know if I can glean a lot from that. But as I continued to do that, and I should say the one that said the most valuable was an upper level student, the one that said it was the least valuable was an introductory level student. So as I continue doing them and gather more data points on that, hopefully I get some more information.
(04:46):
For feedback after the orals, just in personal interactions, multiple students seem to give positive feedback. One of them said that they really prefer the oral assessment to traditional written ones. One of them said they would even rather do it by themselves rather than in a group or in a pair. So anecdotally, there was some pieces. No one told me that they hated them, but I'm sure students that were particularly nervous coming in may have not really preferred them. So that's one thing that I would really like to assess more during reflections when I implement them in the future.
Matt Wittstein (05:31):
When we had talked last time, we talked a little bit about creating rubrics, and as you're reflecting on some of the feedback that you've gotten and starting to think about how you're going to prep to do this in future semesters, have you started any of that work? Have you figured out what goes on an oral assessment rubric that's different from a written exam rubric in your field?
Mark Burnham (05:56):
Yeah. Actually, I implemented rubrics for the oral criticism in my upper level course after our follow-up conversation. And it's a very general rubrics so far, and I'd like to refine it more. But essentially, how I implemented them was I gave the students a list of questions, a list of potential questions that I might ask. So I think it was about eight questions, and I told them that they could choose one of the questions and I would choose the second one. So I split those out in the rubric evenly weighted. So how do they did on the content of the one that they chose, how they did on the content for the question that I chose, and then also had rubric categories for clarity of response, and then a category for how they responded to my follow-up questions. So I'd always ask them a follow-up question or two based on their responses, so how they performed in those.
(06:55):
I'll probably leave them broad structure like that again this coming fall just so I get to use them more and refine them more. But that's how I structured the rubric so far in my first shot.
Matt Wittstein (07:11):
It's always good practice to actually stick with something for more than a single semester to see if it actually worked well and to really work out some of the kinks in it. So I like that you're going to continue with that.
(07:22):
I might need some clarification here, but you only did one round of oral assessment with that senior level group, or did they have a couple opportunities to do an oral exam? Did they have a midterm oral and a final oral? My real question here is, did you see improvement in them? But if they only did it once, then you won't necessarily see any improvement in that?
Mark Burnham (07:44):
Sure. They had two oral assessments in the upper level course, and the first one, I had them do it in pairs. So it was two students paired together just since it was the first time they'd done it, and then the second they did the oral individually. As far as improvement wise, I think I saw improvement in the comfort level. So they knew what they were getting into. As far as the content and the clarity, it was actually really good both times, but I definitely saw improvement in the comfort level of students coming to the classroom.
Matt Wittstein (08:23):
The other piece that our panel talked a lot about that I've always been a big fan of is that metacognitive piece, that reflection piece in learning how to be a better learner, and I think that's a valuable addition to almost any learning setting. So did you add any of those pieces to your courses?
Mark Burnham (08:44):
I did not in this spring, but I would like to in the fall. I think that'll be a really important piece both for them as they progress through the semester and are faced by multiple oral exams. If they think about how they performed or how they prepared on the first one, how they felt going into the first one, they may be able to prepare differently or more effectively or feel more comfortable going into the follow-up oral exams. Having them reflect on it will be really important for me as I'm figuring out how to best implement them in the class. So it'll also serve as a evaluation of the oral exams.
Matt Wittstein (09:26):
I think one of the things I've noticed in my own teaching is when I use a little bit more reflection is that piece where the student says it's the least valuable thing they've done, it actually gives them opportunity to ruminate and think about how maybe they are developing different skills, maybe not content knowledge in biology, but different skills that may suit them well in their career path or in other courses or in other learning settings. So I think that'll be probably a worthwhile addition on a lot of fronts.
(09:57):
Mark, that's all I have for you today. Thank you so much. It was great catching up with you, and I hope we can see how you continue to evolve your assessment techniques in the future.
Mark Burnham (10:06):
Great. Thanks, Matt. Thanks for having me on. It's been super useful and very interesting.
Matt Wittstein (10:19):
Limed: Teaching with a Twist was created and developed by Matt Wittstein, Associate Professor of Exercise science at Elon University. Dhvani Toprani is Elon University's assistant director of learning design and support and serves as a producer for the show. Jeremiah Timberlake is a class of 2024 computer science and music in the liberal arts double major at Elon University and Summer 2023 intern for Limed. Music for the show was composed and recorded by Kai Mitchell, a class of 2024 music production and recording arts student at Elon University. Limed: Teaching With a Twist is published by and produced in collaboration with the Center for Engaged Learning at Elon University. For more information including show notes and additional engaged learning resources, visit www.centerforengagedlearning.org. Thank you for listening, and please subscribe, rate, review, and share our show to help us keep it zesty.