Question Clinic – The inaccurate question

In a quest to make the best exams possible, I will discuss different types of questions and how to improve them. Today’s topic is the inaccurate question.

They happen to the best of us. We write a question, and it seems straightforward. It isn’t until later (sometimes much later) that we learn that the question is actually inaccurate. It includes wrong information or is multiple choice but does not show the right answer. It confuses the students who know the material, and can even lead students to answer other questions incorrectly.

If you notice a question is inaccurate, the first thing to do is remove it. If students have already taken the test, give them credit for a correct answer without regard to which answer they gave.

You may not even realize that you have a bad question until a student confronts you about it. One reason we leave inaccurate questions in a test is because the nature of how we teach introductory subjects.

When one is learning a subject for the first time, we tend to tell the students flat obvious rules about how things work, even when we know that there are exceptions. We use words like “all” and “never” that are absolute. We also generalize, knowing that the student hasn’t been taught about any of the exceptions yet. This tends to penalize students who read ahead, or who have outside knowledge about the subject.

Example Question:

What is the pH of the small intestine?

1. acidic
2. neutral
3. basic

I found this question in an old test. There was probably a reason for this question. Perhaps it was meant to show that chemicals were released into the small intestine that counteract the acidity of the stomach. Perhaps it was meant in comparison to the stomach, I don’t know. But the question is inaccurate because none of the answers is strictly true.

Why it’s wrong

The pH is a number that tells how many hydrogen ions are in a solution. Numbers below 7 are acidic, above 7 are basic. Distilled water has a pH of 7 and is called neutral. In his article “Intraluminal pH of the human gastrointestinal tract”, J. Fallingborg writes that the pH changes along the length of the small intestine. After coming from the stomach, the pH is rapidly increased from very acidic (pH 2) to about pH 6 which is slightly acidic. As it travels through the small intestine, the pH rises to about 7.4 which is slightly basic before dropping down to 5.7 as it enters the large intestine.

The question is inaccurate because the pH of the small intestine could be said to be acidic, basic, and neutral. “About neutral” would be the most accurate answer to the question as the pH stays within two points of 7, and yet, for the majority of the time it is acidic. The question for the student becomes, ‘which answer does my instructor want me to give?’, not ‘which answer is correct’, since they all are.

How to improve the question?

1.Be more specific

Ask the pH of the duodenum or the illium or the caecum. And give specific pH number’s like 6 or 5.7, that might spur memories of a particular passage in the notes. One could also ask about the mechanism. What enzyme spurs the production of sodium bicarbonate raising the pH of the lumen? (Secretin). Or How does the pH change after entering the small intestine from the stomach? (It increases).

2. Change the type of question.

If the question were an essay question, this would not be as much of a problem. It would still be a bit too unspecific. Will every pH that is possible to find in the small intestine be counted as a correct answer? What if the person had inflammatory bowel disease that could cause fluctuations in intestinal pH?  A better question would be…

Describe the pH of the intestine, and explain what factors affect it.

This allows the student to discuss what they know about intestinal pH and encourages them to add more information if possible. It encourages them to go deeper. Many teachers will call such a question harder than the first one because it requires more effort to answer, but if your student knows the answer, the first question is harder because it does not allow for an answer that is completely correct and therefore requires guessing what the teacher wants.

Inaccurate questions penalize your best students. Students who guess the answer do just as well or better than the students who know the answer. Inaccurate questions derail your good students and make the test seem more unfair and superficial.

Do you have a different opinion, or do you remember an inaccurate question that you observed on a test? Leave a comment below.

REFERENCE:

Fallingborg J. Intraluminal pH of the human gastrointestinal tract.
Dan Med Bull. 1999 Jun;46(3):183-96. https://www.ncbi.nlm.nih.gov/pubmed/10421978

Advertisements

Testing Philosophy

Whether you thought of it or not, if you’ve ever given a test, you have a philosophy of testing. In your head, you have an idea of what you expect of the test. Do you expect all of your students to pass it? If not, about how many do you expect to pass? Do you expect them to be able to recall exactly what you told them, or will you show them situations they have never seen to see if they can figure out the correct answer? Are all of the questions equally hard, or do you put in  a few very hard questions to “sort the As from the Bs”? How you answer these questions reveals your philosophy of testing. It shows what you expect from a test, and will drive the questions that you will write.

One thing that I think you should consider, is whether the test you make is actually testing for what you want your students to learn. It is common for a new teacher to depend on existing test banks, and even more common for a long time teacher to use the same test questions over and over without really thinking about why.

When should you reevaluate your tests?

• When most of your students seem to understand the material in class, but fail the test.
• When students are frustrated and upset after the test is over.
• When grades on the test do not seem to be correlated to activity in course.
• When students drop in mass after the first exam.
• When no one gets any of your essay or fill in questions right.
  

It is easy to blame the students, but is it their failing, or is your test to blame?

What makes a good or a bad test question? How do I improve my questions so that they are understood?  What kind of test is best for my subject?

In the next series of blog posts, I will evaluate tests and testing and give some thoughts on how to make your tests better.

Demanding Students

Teaching anatomy means that you spend time around premed and prenursing students. These students tend to be some of the most demanding when it comes to grades, because they are planning to enter a highly competitive educational application process where the difference between an A and a B may matter in whether they are accepted or not.

These students often are picky about every single point that they get in the class. This is difficult for the teacher, because much of teaching is subjective. Sometimes an answer to a question is slightly wrong, how wrong is wrong enough not to get the point. Students will often argue points from old tests, or even try to jump above the teacher and involve the administrators in the grading process.

Here are some tips for teaching demanding students.

1. Have a very clear policy on grades and write it in the syllabus.

The syllabus is what your college dean will look back on to see the policies of your class. If you didn’t state it at the beginning, it can be argued. Statements such as “No late homework will be accepted.” or “Bonus points will be awarded at the discretion of the instructor.” Can make the difference between a clear refusal to change a grade, and making a new assignment because someone skipped it and then decided later that they wanted that grade.

2. Use Rubrics for papers and more subjective assignments.

Rubrics make clear how you are grading. You probably have a rubric in your head when you are grading anyhow, just make it explicit. If possible, make the grading rubric available ahead of time.

Rubrics can contain subjective judgements. For example you could say that 10 percent of a paper is the quality of the introduction. I would further divide that into more specific points if possible.

3. Be consistent

If you say that you don’t take late work, and you accept if from one student, expect the avalanche of requests and vitriol that will follow.

If you decide to extend a deadline for one student, consider extending the deadline for all of the students.

4. Consider Bonus Points

When students are close to the line, consider offering bonus points. Bonus points are a way to separate students who are trying from students who are coasting.

Students with a sincere effort to pass will do extra work to improve their grade.  If a student does not participate in extra activities and they ask why you gave them a C with a grade of 78 instead of raising it to a B, you can point to bonus points and say that the student had an opportunity to get an higher grade if they only did the extra work.

With care, motivated students can make the class more challenging for everyone with a minimum of stress for you and them.

Dec 6, 2018

Who makes the best teacher?

Conventional wisdom in Academia is that the best teacher is the one with the most knowledge about a subject, but that is not what I have found. In my experience, the best teachers for introductory courses are the ones who struggled in the subject.

Why?

Because those teachers that are naturals in the subject, the ones who had no trouble understanding, usually don’t know why they know what they know. It has been so long since they had to learn the basics, that these facts come almost unbidden.

It is the people for whom it did NOT come naturally, those that had to consciously come up with ways to remember the material, that can help the students get past their confusion and understand the material better. The cognitive tricks that they use to learn the subject are often of the most use to beginning learners.

A new subject can be overwhelming. A teacher who struggled can point out the obvious connections without making the student feel ashamed for asking. They won’t get angry when someone asks them to slow down or repeat what they said, because they remember how many times they had to go over the material to remember it.

So what should teachers do if they suspect that they are just too good at their subject?

Consider introducing peer teaching. This will allow students who are a little ahead to do the hard work of answering the most obvious  questions so that the teacher can discuss the deeper ones.

So if you find yourself out of your depth, unsure about what you have to teach, remember, that very uncertainty is your greatest strength.

11/20/18

The Procrastinators

So we have reached the end of the semester, and now I am coming face to face with… the procrastinators.  These are the students who wait until the last minute to realize that they haven’t done lab 2, or 3, or 5 or 7, or 10 for that matter. Now I would be much more judgemental about them if I wasn’t known to procrastinate myself. The only difference between me and those students is, I don’t miss class, and I am brilliant at the last minute save.

The students come to me with mouths full of excuses that I don’t listen to. The only question I ask is do they actually plan to complete the course, and are they willing to do the work to get grades on the board?

Most students don’t realize that I too have deadlines. My due dates are setting tests up on time, and turning back papers before grades are due. I have deadlines for responding to students emails and messages, and for showing up to class at all.

I know all about procrastination.

But what I’ve recently noticed is what the desperation after procrastination can teach you about studying.

• That deadlines are useful.
• That without consequences there is no action.
• That frequent reminders are useful.
• That simple instructions on how to begin can allow the dam to burst and the work to come out.
• That confusion stifles action.
• That bad structure and even bad font can discourage students from reading on.
• That sometimes people lie when you ask them if they understand. (Sometimes you need to ask two or even three times and then show them anyway.)
• That it is not shameful to ask for help.

Talking with procrastinators reminds me that we should not feel irritated when students ask for help. No one signs up for a class with the intent to fail.

Instead, be patient and help the students get through those late nights, even if we think they won’t make it, because even if they don’t complete the assignment, the struggle is a learning experience in itself. Maybe next time, they will start a week  or two earlier, and make it.

Nov 17, 2018

Recommended reading – How a bench and a team of grandmothers can tackle depression

This article from the BBC has some very good advice, and an interesting story about using compassionate lay people to ease the problem of not enough mental health professionals.

http://www.bbc.com/future/story/20181015-how-one-bench-and-a-team-of-grandmothers-can-beat-depression

It reminds me of spaces on the internet where people vent their emotions and others give them likes and comments like virtual pats on the back. These interactions can keep people going, so I would think it would work even better in person.

Here is a quote:

“At first, he tried to adhere to the medical terminology developed in the West, using words like “depression” and “suicidal ideation”. But the grandmothers told him this wouldn’t work. In order to reach people, they insisted, they needed to communicate through culturally rooted concepts that people can identify with. They needed, in other words, to speak the language of their patients. So in addition to the formal training the received, they worked together to incorporate Shona concepts of opening up the mind, and uplifting and strengthening the spirit.”

So what do you think about the idea. Would it work here?

Lecture and Variation

Every time I do a lecture, I change it a bit. That way I can see what works and what doesn’t. I usually do the same lecture several times a week. I erase the board and draw it fresh. Today, I left the old notes on the board and did the lecture by pointing to the old notes. The students didn’t get it.

There is something about showing the steps one at a time that makes it easier to understand. It is less confusing, perhaps.

I think that the students learn better if you draw each part one at a time and assess understanding as you go along.