Mark Burnham wants to help his students get more practice and feel psychologically more prepared for their oral graduation examinations in Emerson and Henry College’s biology program. Our panelists (Dr. Laura Pipe, Dr. Olivia Choplin, & Maddie Fayne) share their thoughts on oral assessments as a practice, the importance of connecting your assessment with your learning outcomes, and tips to help oral assessments go smoothly.
See our extended show notes at https://www.centerforengagedlearning.org/demonstrating-knowledge-using-oral-assessments/.
Mark Burnham wants to help his students get more practice and feel psychologically more prepared for their oral graduation examinations in Emerson and Henry College’s biology program. He already plans to include oral assessments in some of his courses and our panel shares their thoughts on oral assessments as a practice, the importance of connecting your assessment with your learning outcomes, and tips to help oral assessments go smoothly. Panelists include Dr. Laura Pipe (UNC Greensboro), Dr. Olivia Choplin (Elon University), and Maddie Fayne (Elon University).
This episode of Limed: Teaching with a Twist is hosted by Matt Wittstein and produced by Dhvani Toprani and Matt Wittstein in collaboration with the Elon University Center for Engaged Learning.
Limed: Teaching with a Twist
Episode 8 – Demonstrating Knowledge Using Oral Assessments
Matt Wittstein (00:00:09):
You are listening to Limed: Teaching with a Twist, a podcast that plays with pedagogy.
(00:00:23):
Mark Burnham teaches biology and environmental sciences courses at Emory and Henry College in Southwestern Virginia. A unique feature of their undergraduate biology program is that graduates must successfully pass an oral examination administered by the faculty. Mark hopes to help his students prepare for this experience by including oral assessments as part of his teaching practices.
(00:00:46):
Our panel of Laura Pipe from UNC Greensboro, Olivia Choplin from Elon University, and student Maddie Fayne, also from Elon, ask us to focus in on the why as they provide helpful tips for implementing oral assessments in the classroom.
(00:01:01):
The conversation touches on building trust, frameworks for assessment, and creating different ways for students to demonstrate knowing as an equity-based pedagogy. I'm Matt Wittstein and you're listening to Limed: Teaching with a Twist.
(00:01:24):
Hi, Mark. Welcome to the podcast. Would you take a second to just introduce yourself to our audience?
Mark Burnham (00:01:29):
Hi, Matt. Thanks for having me on. My name is Mark Burnham and I'm an assistant professor in the Departments of Biology and Geography and Earth and Environmental Sciences at Emory and Henry College in Southwestern Virginia.
Matt Wittstein (00:01:41):
So when we chatted a little bit previously, you talked about your department has sort of a unique thing that all of your students have to do to graduate. Can you tell us about that?
Mark Burnham (00:01:52):
Sure. So this is specifically the biology department at Emory and Henry, all of the seniors to graduate have to pass what we call a senior oral exam. It's sort of similar to many graduate programs have like an oral prelim and an oral assessment to graduate with a master's or PhD, but we incorporate it into the undergraduate curriculum.
(00:02:12):
So it's essentially a general, across all fields of biology, oral examination. And within those oral examinations, they're taken at the senior level, and we have two professors that have one student at a time and they're assessed on, again, just general biology knowledge across the fields.
Matt Wittstein (00:02:35):
From my perspective, that would've felt very intimidating as an undergraduate student. Then you're a new professor at Emory and Henry, so what are you seeing in some of the students that you've gotten to know as they're preparing for these exams?
Mark Burnham (00:02:50):
Yes, so this is my first year at Emory and Henry, and so this is also my first go-through on the assessing side of the oral examinations. And I'm just getting to know a lot of these students and one thing that I noticed, and this would be applicable to anyone that's going through this sort of oral examination, it's very nerve-racking and it can be really intimidating, especially at the undergraduate level.
(00:03:16):
And I can't imagine sitting in a room with two professors as an undergraduate, that it would be very intimidating. And I do see that quite a bit. They're very nervous and they also are unsure a lot how to study for these exams since it's based on biology across the curriculum from general biology up through their upper level courses.
(00:03:39):
There's sort of some uncertainty about they haven't had these sorts of examinations before and that gets into the problem that I've brought to you today about incorporating them more into my classes. But there's a lot of nerves around the students as they're preparing for these because there's so much uncertainty.
Matt Wittstein (00:03:55):
How long has Emory and Henry's biology program been doing this? What's the rationale from the department for doing an oral examination compared to a written examination?
Mark Burnham (00:04:06):
You know what? That's a question that I have to ask. I'm not sure how long the department's been doing it, at least for the last 5 or 10 years to my knowledge. And I think that a big focus of our department has been on really preparing them for going into graduate school and also communicating in their biology careers.
(00:04:28):
And a lot of graduate programs have oral assessments. We also have a big emphasis on research projects for undergraduates. So those two things, it's like a preview of what something like graduate school would be like. So I think that's really the goal of it is to start to get them used to that before they enter graduate school because that's again, even higher stakes when they get into a master's or PhD program.
Matt Wittstein (00:04:55):
So what are you planning to do to help with some of those nerves?
Mark Burnham (00:05:00):
So my approach, and I've talked to some of the other professors in the department about this too, my approach is to try to incorporate more oral examinations from lower stakes to medium stakes throughout my classes. So I teach general biology and intro to environmental studies at Emory and Henry and then also some upper level courses in ecology, botany and ecosystem sciences.
(00:05:27):
So what I'm doing is incorporating really low stakes oral assessments into my general biology class and a little bit higher stakes, not as high as their senior orals, but a little higher stakes oral assessments into my junior sophomore level ecology classes.
(00:05:46):
And that's really the meat of this question of how to effectively incorporate them more frequently throughout the curriculum so they're not as, I guess for lack of a better term, freaked out about it when they get to their senior year because they haven't had this sort of assessment yet.
Matt Wittstein (00:06:02):
And I can imagine with all of the chatter recently about artificial intelligence, oral examinations are a great way to combat misconduct in the classroom, academic misconduct in the classroom. So it seems like in some ways y'all are ahead of the curve in being prepared for this.
(00:06:24):
So in your story, where are you starting? Have you already implemented this a little bit in some of your classes? Are you starting from a blank slate? Do you have any previous experience with trying out an oral assessment in a class?
Mark Burnham (00:06:40):
In a class? No. I have, again, as a graduate student, just had oral examinations given by my graduate committee. To prepare for those, my cohort and I did mock orals ourselves where we got together as a group and two or three of us would pretend to be the committee and ask the other member of the group oral style questions. So that was my only experience on that side of the table, so to speak.
(00:07:09):
In my classes, this semester, I've started to implement oral assessments. So for example, I've mentioned that I'm implementing them in a low stakes manner in general biology. So the way I've done that is have group oral quizzes. So they were working groups of two to four students and I ask them a question based on the material that we've covered in general biology. They have a few minutes to use their notes and chat amongst each other to come up with answers. Then I'll ask follow up questions.
(00:07:45):
The way I'm assessing those is completely participation-based. So I've told them I'm not assessing them on their answers specifically, but I am assessing them on the amount that all the group members participate in the answer. And then in my 300 level ecology class, I'm implementing a little higher stakes manner. So they actually will be assessed on their answers, but the first round will be done in pairs and the second round will be individual.
(00:08:15):
So trying to implement it in some ways that are hopefully not as nerve-racking for the students, so they start to get used to that oral communication setting. And also given that I haven't done it before, if something went wrong, I didn't want their grade to suffer because of something that I felt I wasn't adequately doing. So I'm starting to ease into incorporating them throughout my classes.
Matt Wittstein (00:08:43):
I like how you're being thoughtful about having low stakes opportunities for your earlier students and higher stakes opportunities to build those skills. What sort of feedback have you gotten from your students about oral examinations?
Mark Burnham (00:08:57):
Yeah, so that's actually a really good question. And I have one example of a student who was in a class with me last fall and then they did an internship over the winter term, and I also have this student in my general biology class this spring when I've started to introduce the oral assessments.
(00:09:17):
And this student has told me that it really hit home with them because during their internship they were just working on something by themselves in their internship and their supervisor came and said that they were expected to now give a tour to some visiting people in 10 minutes. And so they were obviously very nervous right away and just had to go think on their feet and give a tour of this facility that they were doing an internship in with really no prior preparation.
(00:09:50):
And so the student told me that this sort of oral examination and the style of examination is, or the style of assessment is really relevant to the workplace and they actually got to see that firsthand over their winter internship.
Matt Wittstein (00:10:05):
I'm curious, what questions do you have about oral assessments, implementing it, assessing it? What questions do you have for our panel?
Mark Burnham (00:10:15):
Yes. So the main questions that I'm interested in are how to adequately assess their performance on these oral assessments and how to assign a grade essentially, since it's very subjective. I feel like I'm very subjective when I'm sitting there interacting with the students in this setting. Really more specifically, how do I set clear expectations for the students so they know how to A, prepare, but B, interact with me in that setting in a way that they feel confident about? So those are really the main questions.
Matt Wittstein (00:10:58):
In implementing the oral assessments in your classes, do you have specific learning objectives for the oral assessments as compared to maybe the overall course goals? Do they align well with each other or are you looking for something different in the oral examinations?
Mark Burnham (00:11:18):
So this brings up an nice point that I don't think I've touched on so far. When I'm introducing the oral assessments to my students, one of the points that I make is beyond just grad school or beyond just the narrow scope of their senior orals, being able to orderly communicate biological topics in the workplace setting after they graduate is much more important than being able to sit down and take an exam.
(00:11:45):
In the workplace, they're not sitting down and taking written exams, they'll be communicating and writing. But our assessments in college are very different than how their performance is evaluated in the workplace. A lot of the times, their boss or their supervisor or a client, for example, will ask them a question about biology or ask them a question about their knowledge and they'll just have to answer that question orally. So that's one reason why I'm implementing these. Really a broader reason that I'm implementing these.
(00:12:18):
As far as learning objectives themselves, I haven't specifically written learning objectives in based on these orals yet. I would like to include them more in my classes in the future. But the specific ones that I have in mind are things like being able to clearly communicate biological and ecological concepts, or be able to quickly apply biological concepts to questions and problems in society.
(00:12:45):
So those are the broad ones that I have, but again, adequately and specifically assessing that is a major question. And I think that possibly making those more specific to how the orals are run might be beneficial to the students as well. So this all kind of falls within that sphere of what are their expectations going into these and how am I adequately assessing and assigning feedback and grades to them based on the expectations that I've set out from the get-go.
Matt Wittstein (00:13:18):
Mark, I think we have a really good conversation to have with our panel and I am really excited to see what they have to say to help other people get started with trying to implement something like oral assessments in their class. So thank you so much for taking the time to share with us your question, your problem, and your goals. I look forward to talking to you soon.
Mark Burnham (00:13:38):
Thanks a lot, Matt. And I'm really excited to have the panel's feedback and have a discussion with them about the topic as well.
Matt Wittstein (00:13:56):
Laura, Olivia, Maddie, welcome to the show. I want you to introduce yourselves to our audience, but to get us there, can you tell me what you think the purpose or value of an exam and a class is?
Maddie Fayne (00:14:10):
Hi everyone. My name is Maddie Fayne. I'm a senior at Elon University studying psychology and early childhood education, and I find the most value in exams that prepare me for postgraduate life. In the age of the internet, I think that memorizing facts and multiple choice exams aren't the most effective. And having an exam that really teaches me something and will give me a skill for the real world, I find to be really effective.
Olivia Choplin (00:14:42):
Hey, I'm Olivia Choplin. I teach French at Elon University in the Department of World Languages and Cultures, and I'm also in our Center for Teaching and Learning. And I don't really believe in exams anymore, but I do think that things that look like exams and that give us an assessment at a point in time of what our students are capable of doing with the things that we've been trying to help them learn, it can be useful.
(00:15:12):
But I use them and then I don't necessarily put grades on them, but I just say, "Here's an area where you need to work harder and here's an area where you're doing great. You seem to be on the right track with that learning." But exams in and of themselves I find are a word that makes students a little squirrely and it doesn't give me a productive knowledge of what they can do with French. So I've stepped away from them a little bit.
Laura Pipe (00:15:38):
I'm Laura Pipe and I direct the Teaching and Learning Center at the University of North Carolina Greensboro, and I teach in the Department of Kinesiology. I teach undergraduate and graduate sports sociology courses.
(00:15:51):
And I'm with Olivia. So for me exams, I think more about what does a summative assessment really need to be. And for me it's the student really demonstrating they're wrestling with the content. And so I talk to my students a lot about taking intellectual risk and what that looks like and how do we get intellectually sweaty. So I'm in a kinesiology department, so we talk about practice and why practice is so important and that we don't all start with lifting 250 pounds when we weight lift, we start with lifting the bar. And so it's doing that increments.
(00:16:28):
And so when we get towards the end of the course, that final summative assessment is really about them being able to demonstrate that intellectual wrestling with the content and what that means for them, what it comes out as. And it'll be different for every student. I have not given multiple choice tests in a very, very long time because I feel like it doesn't give them the opportunity to wrestle with content.
Matt Wittstein (00:16:52):
What I love about all of your answers is that, Laura, you talked a little bit about summative assessment, but Olivia, it sounds like you care more about the formative feedback and then when you tie those two together, it leans into what Maddie's saying is the most useful things are where I can demonstrate I know stuff and also get something back out of that experience. So really I love how those sort of melded together.
(00:17:12):
And I'm guessing you all have figured out we're going to talk about exams or formal assessments if you will. And specifically we talked to an assistant professor of biology and environmental sciences at Emory and Henry College in Southwest Virginia named Mark Burnham. And he is starting to implement some oral exams into his intro bio class and into some of his upper level bio classes.
(00:17:37):
And the reason he's doing this is because his department, his biology department, as a requirement to graduate, all of their seniors have to pass an oral examination with a couple of the faculty members in the room. Any question from first year to fourth year, and you can imagine the breadth of biology could be plants, could be animals, could be some environmental sciences in there. So you can imagine that that might be a very stressful thing, and he's starting to think about, "Well, how can I support my students by giving them opportunities to practice some of that oral assessment?"
(00:18:12):
He's really looking for some of the big picture stuff, and I think for our audience, I think that can really lean into how do you get started with doing an oral examination? So Olivia, I'm going to start with you a little bit because I know in world languages the listening and communicating piece are pretty central in learning and understanding a language. But if you had a colleague that had never given an oral exam before, how would you help them get started in terms of what to look for in their assessment?
Olivia Choplin (00:18:43):
I would initially ask them what is the purpose of having this assessment be oral? In world languages and cultures, it's pretty clear that listening and speaking are two of the four skills that we want our students to master, and they're very central to communication in a second language. But in another discipline, you may want to have students practice the skill of being able to articulate their learning either for an expert or a lay audience.
(00:19:12):
Sometimes I get my students if I tell them, I want you to be able to explain what you're learning in my class to your grandma and to tell the story of that in a way that's meaningful, and that might be relevant in the context of science. Scientists are often called upon to ask or to answer questions to audiences that don't have the scientific knowledge.
(00:19:35):
And then in another respect, if I have students that I know are going on to grad school, oral comprehensive exams are part of grad school in many, many disciplines. In fact, they were the most frightening part of my PhD and I knew all of the questions in advance and had written them with my professors and they were still so scary to me because it was an important milestone I think in my experience.
(00:20:01):
But I would have that person start with why are we assessing this thing orally and making sure that the students understand the reason for that oral assessment and starting slow, starting small. I would get them to prep students with a small piece first and then they can move into more complex things as the students get more accustomed to the format.
Laura Pipe (00:20:24):
So I always start with the why. It's that whole reverse engineering piece, like why are we doing this? And so I use oral exams in all of my courses that I teach because one, I like to use differentiated learning techniques. So I like for students to be able to demonstrate multiple ways that they're learning and one of those ways is going to be more comfortable for the student than others.
(00:20:46):
And so that gives them, in some spaces, probably about halfway through the course it's more written, so people who really excel at that can demonstrate their learning that way, folks who are much better at expressing their ideas verbally. I myself am Indigenous. I come from an oralistic culture. For us, being able to talk it out has a lot of strengths and benefits.
(00:21:06):
But also the oral exam I use particularly in sports sociology is because the students who are graduating from our institution are graduating with kinesiology degrees. They're planning on going into sports medicine or to physical education or physical therapy, athletic training. None of them are going to be sports sociologists. I can hope and pray and wish, but that's probably not likely for many of them.
(00:21:31):
And so we spend the whole course really answering one question, which is why is physical activity important? Particularly I try to explain to them, when you graduate, if you can't explain to your community why public recreation space is important or why PE is important in K through 12, your community won't have those things. And you can't explain it in these very clinical ways, you need to be able to talk about it, to be able to pull the clinical with the social and be able to give that conversation.
(00:22:01):
And so for me, that's why the oral exam is important, they're able to have a conversation around the content which is going to be useful for them when they graduate. But for me it also demonstrates another way for them to say, "Here's the content that I've mastered."
(00:22:16):
And so when a faculty member starts the conversation with me about, "Hey, I want to try this new thing," my question is always like, "What are you trying to measure? What are you trying to figure out? What's the end goal? How does this benefit both the student and your ability to do the things that we're required to do, assign a grade?"
(00:22:36):
So yeah, that's where I would start with them is the why. And if you start small and you use the multiple courses, you start early on and you can start with just being able to get students to demonstrate that they've made sense of the content.
(00:22:53):
So I use a lot of Jennifer Moon's work. She did a lot of great work on how we reflect through learning, particularly experiential learning. So there's like five stages and one of those is just making sense. If it's a 100, 200 level class, is that what we're asking, that low level of Bloom's taxonomy? Do you want them just to make sense of the content versus moving into working with meaning and meaning making, which is more of that how do I apply it to a situation?
(00:23:22):
And then there's that transformational learning at the end, how do I apply it to my situation? How do I pull from not just what I'm learning in this course or other classes, but what I'm learning from all of the context around me and how do I apply it to create new knowledge? If a faculty member can tell me where on that spectrum they're really hoping to go, then that really articulates what type of oral exam they might want to use.
Matt Wittstein (00:23:45):
Maddie, I want to ask you from your perspective, Olivia brought up a little bit of oral exams being scary. First, do you have experience having taken oral exams? And second, if you were put in a spot where your professor said, "Hey, your final exam is going to be a one-on-one interview instead of a written test," how would that make you feel? How might you prepare for that experience?
Maddie Fayne (00:24:09):
I do have experience with oral exams. As Madame Choplin was saying, I think that world languages really lend themselves to oral exams, and I've been fortunate enough to take French and Hebrew at the college level. And in both of those world languages, I've had oral exams. And let me tell you from a student perspective, they are terrifying. It is so, so nerve-racking walking into a room to have a one-on-one conversation with someone who is completely fluent in this language and is your professor, and you are just getting on your feet in that language. It's so scary.
(00:24:47):
In fact, my first semester that I took Hebrew, I was a freshman and I vividly remember walking to this oral exam with my professor and just shaking, being so nervous. I was more than prepared. I think a key in oral assessments is trusting your professor. They're not going to ask you something or have a conversation with you on a topic that you aren't prepared to speak on.
(00:25:12):
It was my first semester at Hebrew, so I was very new and we had a conversation about the different writing utensils in the room and how many pens and pencils were on the table, and if I had read a book that was on the bookshelf. And it was very easy, but it was something I was prepared for. And I think the fact that my professor was having a conversation on a topic he knew I would be able to communicate on, gave me that confidence to then go into more challenging assessments later on in my journey of learning. But they are terrifying, but I think that they are very effective, especially with world languages.
Laura Pipe (00:25:47):
When I first started doing oral exams, that was my biggest fear. Learning is scary enough and I think we don't give folks enough credit to talk about how learning really is an intellectual risk. You're opening yourself up to new ideas, you're challenging things, you're doing this also in front of other people. So I structure all of mine in groups, so the students spend the whole class in one consistent group. They've picked these group members, they get really comfortable with them.
(00:26:17):
And so when they do their oral exam, and I've done it a couple of different ways. I've done problem-based ones, which are my preference, but I've also done question-based ones where they get the questions in advance and I pick 3 of the 10, but they're prepared in a group. And so they've gotten really comfortable with their peers. So it's not so much they're having to demonstrate to me, it's that they're able to then have the conversation.
(00:26:40):
And I think that has helped with some of the anxiety around it, but it's a whole different way of test taking, right? When we ask students to write things down on an exam, that can be a very personal one-on-one space and they don't have to watch me grade it. But when they're in that group setting, then they're there with their peers. They're experiencing it with friends. They're experiencing it in a safer space than they would if it was just me. And so that might be helpful for somebody who's implementing it for the first time of easing students into that experience.
Matt Wittstein (00:27:14):
Laura, can you explain the difference between problem-based and question-based?
Laura Pipe (00:27:19):
The oral exam that I've settled on for my 300 level undergraduate sports sociology course is actually an RFP. So they do like four assignments leading up to that exam. So they're in a group of five because five's a good group number. You get three, somebody gets left out. You get four, you get two dyads. Five for some reason, everybody gets to stay included. And they pick somebody's hometown.
(00:27:42):
And so they do three mini assignments ahead of that where they're looking at the history of physical activity in that hometown. They're looking at the culture of physical activity in that town and the structures related to it in that hometown. And so then when they get to the oral exam, the oral exam is actually a mock request for proposals from this made up international recreation association that's willing to give them $25,000 to use recreation to address a social issue in that hometown.
(00:28:11):
And so when we get to the oral exam, what they're presenting on is how they're using C. Wright Mills' sociological imagination and Durkheim and all these great sociology pieces to develop a program that uses recreation, communicates what the social issue is and how physical activity is part of that conversation. And so they're really addressing a specific problem.
(00:28:35):
The question-based one is what I used very early on when I started teaching sports sociology, which was much more about giving them some just baseline questions like explain why sociology and cultural pieces impact sport and physical activity. And they'll get six of those questions. And what I'll say to them is, "I'm going to pick three, so come prepared for all six. You don't know what three questions I'm going to pick."
(00:29:06):
And when they do it in a group, the expectation is that they're all going to take turns answering that question, but they can build off from each other. When you do it individually, it's very much that straight answer to this specific question. But yes, so that's really the difference. A project base is trying to solve a bigger problem using the content of the course.
Maddie Fayne (00:29:27):
From a student perspective, I think if I was going into an oral exam and a professor said, "Here are 6 to 10 questions, I'm going to ask you 3 or 4 of them," I would feel significantly more prepared going in. And especially if that's a building block to a later oral exam where the content could be more open, I think that would be a very good way to prepare students.
Olivia Choplin (00:29:48):
I also use the group format, when I'm doing oral assessments with students in world languages and cultures, at the beginning and I ask the students to have a conversation with each other around a theme. And so when I'm assessing that, and I think that that's a hard thing to do, I tell them, "I'm going to assess your individual grammar, your individual use of vocabulary, your individual responses to the content, but that is also influenced by the group. And then I'm also going to assess your interaction with your peers. Are you listening to and responding to your peers or are you just reciting something that you've memorized in relation to this?"
(00:30:31):
And sometimes I find that it's helpful to give students a context for that. So I have a feigned audience where I tell these advanced intermediate students to explain to a first year student of French, what they have learned about how the folk tales we're reading relate to the cultures and societies that they came from or their histories, et cetera.
(00:30:55):
And so they have to meet up with each other and say, "Well, how do you think this connects?" And they have to create this conversational vibe. It's still an oral assessment. I don't think of it as an exam at all, but I can give them feedback on different aspects of the form of their speaking in French and also aspects of the content of their capacity to connect language to culture in meaningful ways, which are things that I value as my learning objectives. And they're together and they know the context in advance and they can prep, but they're not allowed to have notes.
Laura Pipe (00:31:29):
And letting them know in advance is so important. So prepping students for this is going to be really key when you implement it the first time because most students have never had these kind of experiences before. So they are worried, how are you going to grade me on this? You can't just X out an answer when it's a conversation. And how are you going to grade me in a group?
(00:31:50):
For me, it's been building in reflective activities afterwards where they're really able to share, here's how I felt this went, here's what I actually contributed. But I also give them a rubric in advance so they know particularly what are the sociological themes I'm looking for because you can present a really great idea but not give me the actual sociology. And so we talk about why we have an exam, what's the purpose, what are the things I need to see out of it in order to assess a grade, but what I really want them to also gain from it as well.
(00:32:24):
But the other thing I started to add a couple years ago, and I stole this from Jennifer Stephens who's now at Elon, is I rewarded them around risk. So I'm really upfront with some students where I say, "For some folks, a PowerPoint presentation is super easy. It takes nothing for you to do that. You're really skilled at it, and that's not taking a risk. So if you don't think you need risk points, then do that." But for other students, just standing up in front of a group of people and talking is incredibly risky and you can tell.
(00:32:54):
And so a student who's willing to take the risk and go out of their comfort zone to demonstrate their learning, I try to reward that as well because we only get better if we practice. So giving them that upfront knowledge of here's how I'm looking at this, here's how I'm going to grade this, here's the things I'm looking for, can also really ease them into that experience and help them feel just a little bit more confident going into what they're about to experience.
Matt Wittstein (00:33:20):
Maddie, you had talked a little bit about when you mentioned it being scary or terrifying that one of the things your instructor did was they built trust with you in some way. So I'm curious, how do you all build trust with your students? Maddie, how have faculty been able to gain yours and your classmates' trust so it does feel like a safe setting when you step into that examination room?
Olivia Choplin (00:33:45):
I can say that from a perspective of being a world languages instructor, students are well aware of the fact that they cannot express themselves in their second language the way that they can in their first language and the fear of seeming ridiculous is so real and even perhaps stronger than that fear in their first language.
(00:34:06):
And I try to establish students with that by owning it, by recognizing it, by pointing out to them that I too was a novice. I tell them stories about my own novice errors and just say, "This is what I'm working towards, you not having the same embarrassing moments that I have." Like I told my host brother that Americans put a lot of condoms in their food when I was trying to say preservatives. [foreign language 00:34:30] not [foreign language 00:34:31], just so you know.
(00:34:33):
But I also tell them that you literally can't progress in a language unless you're willing to make mistakes and make an idiot of yourself from time to time because it requires communication. Language progress requires communication, and therefore you have to actually practice that act of communication. And I tell them that mistakes are how you learn. There's nothing wrong with making mistakes. You can't learn without making them.
(00:35:01):
But also I try to help them and say, "If your goal is to be able to communicate such that a French-speaking person can understand you, then my goal is to help you minimize the types of errors that would impede communication." And I think sharing your own experiences as a novice is one way that I like to build trust with them, and also creating an environment where it's safe to make mistakes and students aren't penalized and they know that they won't be penalized for making mistakes.
Maddie Fayne (00:35:30):
I think the supporting of students making mistakes is a really key part of building that trust within a student and a professor, not only for world languages, but also even just in some of my psychology classes. It's scary to raise your hand in front of a group of your peers and share your thoughts on something. And even if you miss the mark a little bit, if a professor is there to guide the conversation and acknowledge that you still contributed something to the conversation, I think that can be really valuable. And I know that I feel the most comfortable in my classes where I feel that support from a professor.
Laura Pipe (00:36:10):
I've been having this conversation a lot with faculty because I think right now we're in year three of a pandemic where students are really struggling and our faculty are really struggling. And helping them understand the difference between a transactional learning relationship and an authentic learning relationship, because transactional relationships in learning are about credentialing. They're not necessarily about learning, but an authentic learning relationship is truly about learning.
(00:36:37):
And going to Zoom and doing all these things that we had to do very quickly in the pandemic, quickly put us into these transactional spaces of, "If you do this, I'll give you this grade back." And it was very much like, "I'm going to feed you this thing, you'll feed it back to me and then that's how learning's going to happen." In order to build trust with students, they need to see us as the human beings that we are and that we've gone through a learning process as well.
(00:37:04):
So I'm really transparent and clear with students around the why I make certain decisions in the courses, why I've structured things the way I do, and I also show a lot of empathy. So I invite them to bring their whole selves into a classroom. So if your car's breaking down or you have to work two jobs or you have all these things, I need to know that as a faculty member, as an instructor, so we can work out a plan for you to still be successful in the class.
(00:37:33):
And so many students tell me they've never had a faculty member have that conversation with them. Nobody's ever told me it's okay for me to share with you the weight that I'm carrying and that we can come up with a plan together. And by showing that empathy and being honest and saying, "I've struggled in these same ways too," or, "I made these same mistakes," but then also talking about learning as a practice.
(00:37:56):
So like Olivia was saying, I'm really clear with my students, homework for me is practice. So as long as you're putting in the energy and effort, you're going to get tons of feedback from me, but I'm not going to grade you on the mistakes. I'm going to grade you on the effort because when you go to basketball practice, you're not necessarily being graded on how many layups you make, you're being graded on the effort you're putting in to practice and get better at doing layups. And so being really open with that gets us to a place of trust and compassion between each other. So they're much more willing to try and to make those little mistakes.
(00:38:33):
And I'm also really honest with them right now, we have a whole generation of students who have been taught that failure is not an option. Well, mistakes and failing and trying, that's part of learning. So we have to reteach them that skill. And so we do spend quite a bit of time at the beginning of the semester in all of my classes making lots of mistakes, creating small formative assignments for them to make those mistakes and get comfortable and realize it's not so scary, really goes a long way to creating that kind of authentic relationship between the two of us.
Matt Wittstein (00:39:08):
Olivia, you had mentioned that assessment is a little bit tricky. You talked a little bit about reciting what you've memorized versus interacting with folks, and I want to lead that into what are the best ways to assess an oral examination? Is there a way to actually go about doing that?
(00:39:27):
And I'm actually going to flip this on its head a little bit, and I'm going to start with Maddie and ask, how would you want to be graded based on an oral exam in say, a psychology course? Maybe let's go away from the world languages example, but in a psychology course where there's both levels of content knowledge, but also application of that knowledge.
Maddie Fayne (00:39:46):
I think asking students if they feel comfortable having the assessment recorded would be a really useful tool. As much as I hate to listen to myself speak, I think having a recording to listen back to and have the recording in addition to notes and feedback would be really helpful because sometimes in these instances you just kind of black out and you don't even remember what you said and it's really useful to have that recording to maybe reflect back on.
(00:40:15):
And then I would say if it's an application question, then that is something that will lend itself better to an oral assessment rather than the student just memorizing something, which they could just as easily write on a piece of paper. There's nothing special for an oral assessment in that sense. So I think that applied questions would be the best.
Matt Wittstein (00:40:37):
One of the curiosities that Mark had was to think about the subjectivity versus the objectivity during an oral assessment. And a written exam, especially a multiple choice, there's usually a right answer and a wrong answer, but in a oral exam that's not always as clear. So what are some best practices in doing that assessment, that grading piece that most of us have to do in higher education?
Laura Pipe (00:41:01):
So I think it goes back to the why, like why are you using this exam? What is the purpose of this? And so I think the examples I gave earlier, if my goal is for students to be able to leave the classroom and be able to make a cohesive argument around the role of a community and the responsibilities of community in the physical health of the people of that community and what that means for a healthy community, then that's what I'm assessing. Or am I assessing their actual sociological theoretical understanding of certain concepts? Then that's what I'm assessing.
(00:41:38):
Either way, I have to be really clear with the students before they even walk into the exam, what is it that I want to know. And so for me, that's a rubric. I give them a rubric in advance. It's got, here's the things I'm going to grade you on, here's what 100% looks like. Here's what not 100%, all the way down. And so being really, really clear with them on that.
(00:41:59):
The reflective activity at the end where the student's telling me what they thought happened, how they felt it went, what did they contribute to the conversation that helps the students who are maybe a little more introverted and struggle in that group space and being able to present, to still be able to demonstrate their knowledge.
(00:42:18):
But I'm also really upfront with students that any type of grading is a subjective process, which is really hard for some STEM folks to wrap their head around. Nothing we do is truly objective. And I'm a qualitative researcher by nature, so that's where that comes from. Even the questions I select as the faculty member when I do a multiple choice test is a decision that I'm making that makes sure it's no longer objective. Giving the students that opportunity to be able to communicate back with me about what they think happened, what they didn't think happened.
(00:42:51):
I also put into my syllabus now a window of time after each assignment gets turned in that they can contest the grade and that we'll have a conversation around it. I do a lot more narrative-based assessment. So not only are they getting that letter grade, but they're getting that narrative assessment of what I've seen them do. Being really upfront with this is why we're doing this as an oral exam, here's what I'm looking to see come out of the other end, that's an essential part to making it more equitable and fair.
Olivia Choplin (00:43:25):
I like what Laura said about preparing the students in advance for the why and the what. And I also think it's really important to share with them what an excellent response would look like and what an insufficient response looks like in different types of context. We're looking for students to apply their knowledge in different ways. I think oral exams can be great for application and they're also a good preparation for life later when you have to think on your feet, for example, in a job interview.
(00:43:53):
So I think helping students understand the types of skills that these type of things are preparing them for in the outside world is really helpful. But then if you are setting up an oral exam to really assess their ability to articulate their understanding of certain concepts that you've covered in a class, then you want to give them some examples of appropriate depth for this type of situation.
(00:44:18):
And I love what Laura said as well about the idea of an exam wrapper. I think the metacognitive piece, particularly for Mark Burnham, if he's prepping these intro students for an advanced exam as seniors, I think having the students reflect back on what they thought they did well in the oral exam and what they thought they could have done better is awesome preparation for later as they become more advanced biologists and they're answering more complex questions.
(00:44:49):
Having them at every level think about their own progress is a really great learning tool and could really lead to seniors with better communication skills than they would've otherwise had.
Laura Pipe (00:45:03):
Key to that is reflection. We have to teach students how to reflect. And a lot of my work around Indigenous pedagogy reflection is essential to how Indigenous communities on Turtle Island learn. For the Tuscarora Haudenosaunee it's about I observe something, I experiment with something, I reflect, and you just keep repeating that cycle over and over again. That's an essential aspect of learning.
(00:45:29):
But we in Western culture have also taught students that oftentimes reflection's that kind of throwaway activity. So give me a one-page reflection and it's always a summary of events. So we also have to spend time teaching students how to reflect and why reflection is an essential tool for learning.
(00:45:47):
And so I mentioned Jennifer Moon's work, that's where her work starts is she's specifically looking at reflection and experiential learning and how those pieces go together. And so I actually give my students that at the beginning of the semester, spend some time with them on here's how you move from just making sense all the way to transformational learning. And I ask them throughout the semester, where are you at on that spectrum with this information? Where are you at with this kind of conversation and what do you need to get to the next level? And they spend a lot of time reflecting in that.
(00:46:23):
What I've gotten back this semester, because I actually started during my mid-course eval asking students about that particular activity, they all keep coming back saying that was essential because they're now using it in other classes because nobody's taught them how to reflect. So that's the other piece of this is you really do have to set up for them too what a reflection looks like in a learning sense because we've not taught them that.
Maddie Fayne (00:46:50):
And on the topic of wrapping up an oral assessment prior to the reflection, I think something really useful could be asking a student, is there something you want to tell me about that you feel prepared to speak on? Explain a concept to me that you know about. Inevitably there's something that you feel really knowledgeable on that you aren't assessed, and that's your opportunity to share, look, I know this content. It might be something you're extra passionate about and interested in and that's why you have a little bit more knowledge.
(00:47:15):
But I think that would be a really useful opportunity for not only the student to share knowledge they have, but also it's a little bit of a confidence booster, like here's this thing I am prepared to speak on and I want to share with you.
Olivia Choplin (00:47:27):
I love that, Maddie.
Laura Pipe (00:47:29):
And Maddie, I would encourage folks to not wait to do that just on an exam. This semester is my first time back in the face-to-face classroom in the last couple of years. COVID put us all online for a long time. And in my face-to-face classes, I now do a quarter assessment of that where I ask the students, tell me one thing that you've learned so far that's been meaningful. Tell me one thing that you were surprised that you thought was going to be a waste of time, but you gained something from it. And then tell me what's one thing that you wish we had spent more time on that you really felt like you learned a lot about and that you could teach somebody else.
(00:48:03):
And it's been life changing for me to see it because I think all of us who teach can tell you about halfway through the course you're like, "Why did I pick that reading? Why did I use that activity? Okay." Because when you build the course, you're building with half the conversation missing. And so when you get in it, you're like, "Oh, maybe that wasn't my best one."
(00:48:22):
And those have been the things that students come back and they're like, "No one's ever taught me how to do an annotated bibliography. Nobody's explained APA to me, and now I have a better understanding of the whole construct of why I'm doing this." Or things like, "Well, I really loved this article on the bicycle as a liberation tool for women, and I'm really glad you added that in here and here's what I went and researched on my own." And I'm like, "Okay."
(00:48:46):
So for me, it's been helpful because students who I thought were really struggling with concepts, I'm seeing that they're not, that I'm asking the wrong questions in my assessments, so I've been able to adjust those. But then also students who I thought were really getting it, I'm also seeing them say, "Well, here's where I've made connections in other classes that I'm having too." And so it's been a helpful tool for me, but I think it's also been really a confidence builder for the students that their voice, their ideas, their knowledge are all valued in this space as part of the learning.
Maddie Fayne (00:49:19):
Yeah, absolutely. And I think that if I was a student and my professor gave me that midterm check-in, I would really appreciate it because it is a way to share your own personality and really how you're feeling about the course with that professor.
(00:49:32):
And then tying it back to the oral assessment, when you're going into a room, whether it's in a group setting or more one on one, there feels like a bit more of a connection. It's sometimes hard depending on the class size to have that connection with every professor.
(00:49:45):
So if you're able to share the things you're struggling with or the things you feel really good about, or any independent research you did on class content, it builds that relationship, which can then make that oral assessment a bit more comfortable.
Matt Wittstein (00:49:58):
I think we've touched on it a couple times, but I'd like to maybe bring it to the center of the conversation, the idea of oral assessments as being a different form of equity in the classroom. It seems like something that we should talk about a little bit more. Laura, I know you've thought about this a lot, so I'm going to let you lead off. But how do you find the relationship between a written exam and our colonialized way of teaching versus an oral exam, which might also create different types of discomfort for different people? How do you find that balance?
Laura Pipe (00:50:34):
For me, it's having differentiated learning assessments throughout the course. And I'm really upfront with students, I purposely choose different types of assessments throughout the entire course because I want you to demonstrate your learning and have the opportunity to do that in a way that you excel at. And it might be this or it might be something else, but this gives you that opportunity, because it goes back to what Maddie said, this confidence building piece. It is so easy for students to feel like one mistake and then be completely defeated in the conversation of learning. And so giving them those kind of opportunities is really essential to me.
(00:51:12):
Rewarding risk-taking is really essential in that. Some students just to be able to talk about and talk through an idea is really powerful and comes to them really naturally much better than giving you a rote response on an assessment. It allows them to go down different pathways and pull in the things, kind of like Maddie's saying, that were important to them, that were personal to them.
(00:51:39):
But other students are like, "Wait, that's not what I prepared for. I've been conditioned and taught my whole life this is how I get an A. This is what I'm going to do and what you're asking me to do now is to have a conversation." And that becomes really hard for them. And so things like adding that reflection at the end, spending some really clear time upfront about why we're doing this, what this looks like.
(00:52:01):
But also being mindful in the grading, showing them that rubric in advance that I'm still grading you on that content in the way that you're used to, it's just a different presentation of it, starts to do that, kind of a balance piece. But it's being very intentional of I'm going to offer multiple ways of demonstrating knowing throughout this course and some of them are going to be your jam, some of them are not and that's okay.
(00:52:27):
And I also am really intentional that that final assessment, that summative assessment is not a make or break. None of my assignments are that high risk. You can fail an assignment and still get an A, A minus in my class as long as you're doing other work and participating and showing up and staying engaged.
(00:52:49):
Like I said before, learning in itself is intellectually risky. I should not be contributing to that in a structural way. I should not be contributing to the anxiety that's already there and underlining. So I try to give students those opportunities to realize we're building to something. And just because you've had this final exam and maybe you bombed it, it doesn't diminish all this other building you've already demonstrated in the course.
Olivia Choplin (00:53:16):
And another thing, Laura had brought up earlier was it gives different students who have different gifts, different ways to shine, in addition to not having any one assessment be a make or break thing, which is just good pedagogy. But some students process things better written, some students process things better orally. So having multiple different ways of demonstrating learning is really important.
(00:53:41):
And interestingly, in a world languages context, we run into this a lot where I have students who, depending on their high school preparation, they may have really stellar grammar, but their pronunciation is horrible. So if I did everything about pronunciation, it would be awful. Whereas then I might have a heritage speaker whose French is beautiful when they're talking, but they ... Half the letters that you write in French are not pronounced. So those students don't know how to spell often at all.
(00:54:12):
Giving multiple ways of demonstrating knowledge, when communication is one of the long-term goals, written and spoken communication, then it just allows different people a chance to find a place where they feel strong and demonstrate that.
(00:54:28):
And then getting students outside their comfort zone too, because some of them will stay where they're comfortable. Making multiple different types of assessments, some of which are uncomfortable, but none of which are make or break for the course is a way of assuring a better sense of equity across the class.
Laura Pipe (00:54:47):
I also spend some time helping them dismantle the myth of learning styles because they'll tell you, "Well, I'm an oral learner," or, "I only am a visual learner." And we talk about that's not a growth mindset. So we spend time in that space as well so that when we get to things like the oral exam, the student who says, "I'm not really strong at presenting, I think I can really convey my ideas better in writing," we talk about strategies for that and how they're going to grow. That's part of the reason why I give them questions upfront or I give them context upfront.
(00:55:21):
And we talk about a strategy is writing out your entire response, practicing it. Don't read it to me, but you'll have that opportunity to work within the one that you're most comfortable with. But we're also working on growing and that's part of the learning space and that's what we do as an institution of learning.
(00:55:40):
So helping them really break away from that idea that I can only demonstrate my learning in this one way is really helpful at helping them understand that a learning style was an idea. It's really you have a preference or something that you're more skilled in, and now we're working on building skills over here.
Maddie Fayne (00:55:58):
I definitely fall victim to considering myself to be a visual learner just because I think that most of the assessments we have throughout education is being able to read something and then write it down. So it's not that I'm a visual learner, it's just that's what I have experience in. So give me experience in some other assessment and then I'll be able to grow and develop that skill as well. But that's something I definitely fall victim to.
Matt Wittstein (00:56:22):
This has been a really excellent conversation. So thank you all so much for taking the time to share your experience with us. I'm really excited to go back to Mark and share what you have talked about today with him.
Maddie Fayne (00:56:35):
Thanks for having me.
Laura Pipe (00:56:37):
Thanks for letting me hang out with y'all. I learned a lot.
Olivia Choplin (00:56:41):
Thanks, Matt.
Matt Wittstein (00:56:51):
Mark, welcome back to the show.
Mark Burnham (00:56:54):
Hi, Matt. Thanks for having be back. I'm excited to hear what the panel had to say.
Matt Wittstein (00:56:57):
Well, I hope our panel is able to support you in your classes. And I want to start off by saying there was definitely consensus that you're doing a lot of things in really smart ways, and it's obvious the care that you have for your students as you help them prepare for their graduating exams.
(00:57:12):
So we had Dr. Laura Pipe From UNC Greensboro, Olivia Choplin from Elon University, and Maddie Fayne, our student from Elon University. Laura and Olivia are both practitioners in the classroom as well as leaders in their university's teaching and learning centers. And Maddie is a graduating psychology student with minors in early childhood education and French.
(00:57:34):
And we had a fantastic conversation about oral exams, and I would say my biggest takeaway was that an oral exam is not serving the same purpose as a written exam. So don't just make an interview version of your normal midterm. You have to do something different and think about how those learning goals are different.
(00:57:52):
Going a little bit deeper into that, we really focused mostly on two pieces. One, how to prepare your students for their exam, and two, just some helpful tips for assessment. So Maddie shared out that one of the most important pieces for her to feel confident going into an oral exam was being able to simply trust her professor, and that starts with relationship building.
(00:58:13):
As we talked a little bit more about how to prepare students, the panelists expressed multiple ways to help set expectations, rubrics, examples of what is excellent versus what is insufficient, and giving students opportunities to practice.
(00:58:27):
In the end though, the biggest challenge may really be helping students overcome the very real fear that they will have. Laura reminded us that learning itself is scary and being open to new information or new ways to express understanding is an intellectual risk.
(00:58:43):
And Olivia reminded us that failure is actually a really important part of learning. So maybe sharing a personal story might humanize you and help you build some trust with your students and allow them to flourish as you implement these oral examinations.
(00:58:57):
Shifting a little bit towards assessment, again, those rubrics will be very helpful for both you and the students. Rubrics came up a few times in our conversation. Before you make them though, it is really important to consider those learning outcomes of the exam. Is it about communication or about content knowledge? Is it about applying knowledge or thinking on your feet? Those learning outcomes can and should vary between your lower and upper level courses, just like you're already doing. Asking and answering the why am I doing an oral exam question might be a really simple starting place to determine what you should really be grading.
(00:59:35):
To me, the most salient piece is that the exam itself should be a learning experience. Build in formative feedback that highlights students' strengths and opportunities for improvement. Moreover, we really support coupling this work with lots of reflection and metacognition, especially as it aligns with the professional workplace experience where you aren't likely to get feedback from the boss after every single water cooler conversation that you have.
(01:00:00):
A really interesting point came up in that by offering lots of different ways to demonstrate learning, we also have the opportunity to be more equitable in our teaching practices. Each assessment, written or oral exam, group projects, writing assignments, presentations, learning journals, whatever you're using to assess your students, is a chance for us as teachers to gather a piece of evidence that our students are actually learning. And what comes with that is that none of those assessments should be all or nothing. There shouldn't be any super major high stakes, just a new opportunity to learn how our students are learning.
(01:00:35):
So Mark, I hope this affirms some of the decisions that you've made, but what are your first impressions of the panel's advice?
Mark Burnham (01:00:41):
Yeah, thanks, Matt. That definitely is very helpful and it affirms some of the decisions and also gives me some other things to think about. Some of the additional pieces that I haven't done an adequate job of implementing so far are things like rubrics. And I think that that's a major piece that I need to think about more going forward.
(01:01:03):
Overall, I think their advice is really strong and it gives more diverse viewpoints of the approach that I'm taking on this. It gives me more things to think about as I'm working on developing them and improving them in future classes.
Matt Wittstein (01:01:19):
And I think with that rubric piece, I mean the same way in my classroom, I iterate every semester. The first rubric I ever wrote, I will just tell you it was not good. And I've steered away from very strict rubrics and used them more as guideposts to be like, "Hey, this is what I'm looking out for." So that might be a tip that you can play around with.
(01:01:40):
Are there any changes you're thinking about making with what you're implementing in this semester based on some of this feedback?
Mark Burnham (01:01:46):
Yeah, I think that I definitely could make some changes with how I'm doing it this semester. In my upper level class, I haven't actually started the orals for that particular class. So we have our first one in a week or so, and then our next one in about a month. And I think what I'll try to do is at least put together at least just a bare bones rubric to start to clearly communicate my expectations to the students.
(01:02:12):
Another piece that I think is really valuable from the panel is sharing a personal story. I definitely have personal stories of places that I struggled in oral examinations as a graduate student even, and I think that that would really help them move forward in preparing for theirs and realizing that they don't have to be perfect. It's okay to get to points where you say you're not sure about something or you don't know, and that is in and of itself, a learning experience that will be valuable going into the workplace.
(01:02:41):
So there are a couple changes, I think. Mainly starting to put together some early rubrics and incorporating some of that sharing a personal story. I think even more valuable as I'm also learning at the same time with implementing these oral examinations this semester and kind of combining the advice the panel's given as I'm working through the rest of the orals this semester and incorporating those going forward into future semesters is going to be really important. So that's super valuable.
Matt Wittstein (01:03:18):
I think the one thing I do want to emphasize is we talked a lot about reflection as part of this process and those metacognitive practices, I know I mentioned it earlier. But there was also this idea of sharing forward what the reflection your past students have done. So that might be an easy thing to add to your end of semester evaluations to share some of their reflections, just so you could share those forward with your future students of what your students thought.
(01:03:43):
Mark, thank you for joining our show and I hope that your first run with oral exams goes well and your students and you learn something from trying it out this semester. Thank you.
Mark Burnham (01:03:53):
Thank you for having me on, Matt. That's really valuable information. I look forward to incorporating some of the panel's advice going forward.
Matt Wittstein (01:04:09):
Limed: Teaching with a Twist was created and developed by Matt Wittstein, associate professor of exercise science. Dhvani Toprani is an instructional technologist and serves as a producer for the show. Music for the show was composed and recorded by Kai Mitchell, a class of 2024 music production and recording arts student at Elon University.
(01:04:29):
Limed: Teaching with a Twist is published by and produced in collaboration with the Center for Engaged Learning at Elon University. For more information including show notes and additional Engaged Learning resources, visit www.centerforengagedlearning.org. Thank you for listening and please subscribe, rate, review, and share our show to help us keep it zesty.