66% of the Time, Every Time

When I began teaching economics something struck me during the first week. I knew a fair amount about economics -- much less than I thought -- but I had received not even a minute's worth of instruction on teaching. All I could think to do was read the book, more or less explain it in my own words using examples not in the book, and answer questions. There were no war stories for a first year teacher of microeconomic theory. One thing that gradually occurred to me is that a knowledge of economics, and then later of law, only accounted for about 66% of what I did as a teacher. And it also occurred me that while students see the professor while he or she is teaching, they only witness about 66% of what goes into teaching.

Other courses, common sense, and day to day experiences inform teaching yet their importance remains behind the scenes. One of the most useful courses I took was a required freshman level course in logic. I am not sure it is required or even offered any more but it did mean that I do not confuse causation and correlation. It also meant that I do my best to correct students who reason like this: "The professor does not need to take role because I attend regularly" Bizarre, right? But I have heard the very same "reasoning" from law professors. For example, "There is no need to have a rule requiring professors to take role because I already take role." I assume professors finding this acceptable also find it acceptable in class.

And then there was statistics. There I learned the difference between reliability and validity. Reliability means the tool you are using when applied to the same data gives you the same result. Validity means the tool is actually testing what you are intending to test. If you've got a tape measure that has been stretched it may consistently measure your waist at 32 inches even though is is 36 inches around. It's just not a valid test of your girth, although can surely be a source of great happiness.

The meaning of a normal distribution also came up and can be understood in the context of reasoning I have heard twice lately: "My method of testing is valid because it produced a normal distribution." I most recently heard this from someone administering a law exam to people with widely varying knowledge of English. The normal distribution means nothing about the validity of the test. My guess is that what she was testing was the ability to understand English. The normal distribution fixation is particularly odd. If the students in the class are normally distributed then, hopefully, the test result will reflect that. On the other hand, getting a normal distribution does not mean the same is true of the class itself. In fact, a normal distribution could just as easily cause concern about the test. Normal distributions are, however, convenient when grades must be assigned.

And now back to logic. Remember your high school math classes. Some teachers said to show your work and then gave you credit if you got everything thing right except, say, the final step. Others just machine graded.The problem is this. In most complex math problems there are many ways to get a wrong answer. Some reveal that the test taker did not have a clue. Some reveal that the test taker forgot to carry the one on the last step. The machine grader gives them the same credit although their knowledge and understanding are quite different. The teacher who requires the student to show his or her work makes a distinction because there is a distinction. Of course, the same is true in law where the issues are not simply complex but more nuanced.

This also relates to the point that students see only about 66% of what goes into teaching. Suppose you give a machine graded exam and there are 10 reasons that could explain a wrong answer. If most of the students are getting it wrong for the same reason, it suggests an opportunity to improve one's teaching the next term. (Unless, of course, the goal is not really to teach but to get a good distribution.) I assume the machine graded test givers just plow along without pin pointing the problem which may reflect their teaching as much as student diligence.

The all time prize for irrational testing actually goes to essay test givers who say something like "Answer 3 of the next 5 questions." There are many combinations of 3 out of 5 and each one represents a different test. In addition, a student could get an 80 of 100 on all five and do worse than a student who scores and 85 on three but would have scored a 60 on the other two. Pretty simple, right? This is, however, popular with the students and you know where that can lead.

I would not want to confuse causation and correlation but there is pattern. All of the reasoning that, at least to me, seems in error does make the lives of those making the errors easier. Could it be that reasoning is driven by convenience and self-interest?