Right Wing Nation

From Rory, whose daughter’s teacher said the following:

You daughter is a good reader, she just has problems with her decoding

One more time: The fundamental problem with education is that idiots like this become teachers.

Posted early this week. Here.

I posted this last year. However, I have edited it, and expanded my discussion of how evaluations might be used in the public schools, as well as at the university. Of course, that means an already long article is now longer. Sorry abut that.

Let’s talk evaluations, starting with this article from Inside Higher Ed:

But what if the much derided Web site’s rankings have a high correlation with markers that are more widely accepted as measures of faculty performance? Last year, a scholarly study found a high correlation between RateMyProfessors.com and a university’s own system of student evaluations. Now, a new study is finding a high correlation between RateMyProfessors and a student evaluation system used nationally.

A new study is about to appear in the journal Assessment & Evaluation in Higher Education and it will argue that there are similarities in the rankings in RateMyProfessors.com and IDEA, a student evaluation system used at about 275 colleges nationally and run by a nonprofit group affiliated with Kansas State University.

What is notable is that while RateMyProfessors.com gives power to students, IDEA gives a lot of control over the process to faculty members. Professors identify the teaching objectives that are important to the class, and those are the measures that count the most. In addition, weighting is used so that adjustments are made for factors beyond professors’ control, such as class size, student work habits and so forth — all variables that RateMyProfessors doesn’t really account for (or try to account for).

The study looked at the rankings of 126 professors at Lander University, in South Carolina, and compared the two ratings systems. The findings:

• Student rankings on the ease of courses were consistent in both systems and correlated with grades.
• Professors’ rankings for “clarity” and “helpfulness” on RateMyProfessors.com correlated with overall rankings for course excellence on IDEA.
• The similarities were such that, the journal article says, they offer “preliminary support for the validity of the evaluations on RateMyProfessors.com.”

It has been my experience that those who discount evaluations are bad teachers. In fact, I can think of many examples of faculty I know who discount evaluations and who are bad teachers, but I cannot think of even one counterexample. I do not know even one faculty member who is a good teacher and discounts evaluations. Not one. But I know lots of bad teachers who discount evaluations.

The most common statement about evaluations is that students care only about how easy the course is and what grade they get. But it takes little more than a quick perusal of professor ratings on RateMyProfessors.com to see that the assumption is a fallacy.

Let’s look at the categories RateMyProfessors.com uses to evaluate professors (1-5):

Easiness - Some students may factor in the easiness or difficulty of the professor or course material when selecting a class to take. Is this class an easy A? How much work do you need to do in order to get a good grade? Please note this category is NOT included in the “Overall Quality” rating.

Helpfulness - Helpfulness is defined as a professor’s helpfulness and approachability. Is this professor approachable, nice and easy to communicate with? How accessible is the professor and is he/she available during office hours or after class for additional help?

Clarity - A professor’s organization and time management skills can make a great difference on what you get out of the class. How well does the professor teach the course material? Were you able to understand the class topics based on the professor’s teaching methods and style?

Overall Quality - The Overall Quality rating is determined by the average rating of the Helpfulness and Clarity given by all users. An overall rating of 3.5 to 5 is considered good (yellow smiley face). An overall rating of 2.5 to 3.5 is considered average (green smiley face). An overall rating of 1 to 2.5 is considered poor (blue sad face). The Easiness rating is NOT included when calculating the Overall Quality rating.

Rater Interest - There is always that one class everyone recommends taking before graduating. As a student, how interested were you in the class, BEFORE taking it? Or how interested were you in taking this course from this specific professor.

So easiness is included, but crucially, it is not included in the overall rating. There goes the cornerstone of that major assumption about evaluations, that they are nothing more than evaluations of the ease of a course. My university evaluations are admittedly superior as rating tools: There are more questions, half of which relate to the course and the other half to the instructor, and they are more detailed. RateMyProfessors.com is by no means a rigorous tool for rating teacher effectiveness, but then, it’s a web application, and if there were twenty questions on it, nobody would use it.

I said above that a quick perusal of the evaluations would destroy the major assumption about RateMyProfessor.com, and here is an example:

We may assume that this student is a history major. Note that the student calls the professor “demanding,” and gives him only a 2 on easiness. Yet, this professor gets a glowing review. Here is another, from a different professor, at a different university, in a different field:

The first evaluation was (presumably) from a major, and an upperclassman. This evaluation is from a student who dislikes the course subject, and admits that he has a hard time with it. It also looks like this student is most likely a freshman or a sophomore, to judge from the course number (I replaced the letters, but not the first course number, because it indicates the level of the course). This student gives this professor the lowest possible score for easiness (1), yet like the last evaluator, gives the professor a glowing review. How is this possible, if evaluations are nothing more than a popularity contest, and students rate primarily on how easily they can get As?

But what this article misses is that one hears exactly the same complaints among faculty about university student evaluations as RateMyProfessors.com. It’s one of the Laws of the Faculty Lounge that if there are tenured faculty present and student evaluations come up, at least one professor will make a sneering comment. As a group, tenured faculty give little concern to teaching (which renders the whole “I want a real professor to teach my class!” argument ludicrous.). Don’t believe me? Here it is from the Chronicle of Higher Education:

Colleges are quick to argue that a college education is more about enlightenment than employment. That may be the biggest deception of all. Often there is a Grand Canyon of difference between the reality and what higher-education institutions, especially research ones, tout in their viewbooks and on their Web sites. Colleges and universities are businesses, and students are a cost item, while research is a profit center. As a result, many institutions tend to educate students in the cheapest way possible: large lecture classes, with necessary small classes staffed by rock-bottom-cost graduate students. At many colleges, only a small percentage of the typical student’s classroom hours will have been spent with fewer than 30 students taught by a professor, according to student-questionnaire data I used for my book How to Get an Ivy League Education at a State University. When students at 115 institutions were asked what percentage of their class time had been spent in classes of fewer than 30 students, the average response was 28 percent.

That’s not to say that professor-taught classes are so worthwhile. The more prestigious the institution, the more likely that faculty members are hired and promoted much more for their research than for their teaching. Professors who bring in big research dollars are almost always rewarded more highly than a fine teacher who doesn’t bring in the research bucks. Ernest L. Boyer, the late president of the Carnegie Foundation for the Advancement of Teaching, used to say that winning the campus teaching award was the kiss of death when it came to tenure. So, no surprise, in the latest annual national survey of freshmen conducted by the Higher Education Research Institute at the University of California at Los Angeles, 44.6 percent said they were not satisfied with the quality of instruction they received. Imagine if that many people were dissatisfied with a brand of car: It would quickly go off the market. Colleges should be held to a much higher standard, as a higher education costs so much more, requires years of time, and has so much potential impact on your life. Meanwhile, 43.5 percent of freshmen also reported “frequently” feeling bored in class, the survey found.

Ha! Students are idiots! What do they know? Read on:

College students may be dissatisfied with instruction, but, despite that, do they learn? A 2006 study supported by the Pew Charitable Trusts found that 50 percent of college seniors scored below “proficient” levels on a test that required them to do such basic tasks as understand the arguments of newspaper editorials or compare credit-card offers. Almost 20 percent of seniors had only basic quantitative skills. The students could not estimate if their car had enough gas to get to the gas station.

Unbelievably, according to the Spellings Report, which was released in 2006 by a federal commission that examined the future of American higher education, things are getting even worse . . .

There follows a lengthy discussion of what to do about the problem. But one thing that can be done, the one thing that is relevant here, is to start taking student evaluations seriously, and not just for teaching faculty.

Student evaluations are by no means perfect. Yes, there are students who care about nothing but how easily they can get As. But at least our evaluations, and the ones at RateMyProfessors.com, can be filtered. Our evaluations are done on a scantron, and on the back are four short answer questions:

1. What did you like most about this course/instructor?
2. What did you like least about this course/instructor?
3. What could the instructor do to improve the course or his or her teaching effectiveness?
4. Other comments?

The comments are linked to the evaluation scores on the other side of the scantron. Non-serious comments indicate a non-serious evaluation. In other words, one can safely ignore the evaluation of a student who answered, say, the second question with something like, “Give us all As.” And yes, you do get non-serious comments, because as I admitted, there are students who care about nothing but how easily they can get As.

However, that some students care about nothing other than the ease with which they can get As does not imply that all or even most students care about nothing other than the ease with which they can get As, nor does it discount the student evaluation as a rating tool. One would assume that somebody with a PhD would have enough basic intelligence to grasp that, but far too many do not — or they do, but would rather discount evaluations rather than face up to the fact that they might not be the brilliant teachers they think they are.

Tenured or teaching faculty, if you are a good teacher and get good evaluations, you have no reason to sneer at them. In fact, if you get good evaluations, yet claim that evaluations are nothing more than a popularity contest, then you are admitting that you are a poseur who does not take your career seriously and that you hand out As like candy. And how stupid would you have to be to do that?

Again, some evaluations are crap. But many of them can be filtered, because they are accompanied by crap comments. This is a useful, but sloppy filter, since not all students bother to write comments on the back. Still, one may filter some of the, ahem, less than useful evaluations.

Note that the same is true on RateMyProfessors.com: Each evaluation is accompanied by comments, and non-serious comments indicate non-serious evaluations. But also note that by no means are all comments non-serious. Neither are the comments on our evaluation scantrons.

At my university, student evaluations were required to be given in all classes. However, the results were largely ignored for tenured and tenure-track faculty, but used as a major employment criterion for adjunct, or teaching, faculty. There is a certain amount of logic to this, except that it relieves tenured faculty of any review of their teaching skills. And unsurprisingly, it is always tenured faculty who sneer at evaluations. Teaching faculty do not, because if they get bad evaluations, they are not re-hired — and they shouldn’t be.

Of course, neither should the university harbor tenured faculty who consistently get bad evaluations, but that’s another topic, and addressing that problem would require at least a major redefinition of tenure, if not abolishing it altogether.

But back to student evaluations. They serve two major functions: They provide teaching effectiveness feedback for the department, school/college, and university; and they provide teaching effectiveness feedback for the faculty member. I, at least, would hold that both are equally important. At the university, they provide yet a third function.

Evaluations as school/university feedback

If evaluations are to be used administratively as teaching effectiveness rating tools, a number of issues must be addressed. Evaluations are subjective, after all. As I said earlier, some students care about nothing other than ease. Also, a statistician should design the evaluation, and not, say, a committee of teachers or the school board. Some outside agency should administer, or at least design and process, the evaluations.

Because of the nature and purpose of student evaluations, there should be no “undecided” option available to students on any question. That is, the available options should force students to make either a positive or negative response of some degree. Student evaluations are not political polls, where “undecided” can often give useful information. Student evaluations are rating tools, and as such, should force a rating.

Alternatively, including an “undecided” option can be used as a filter. After all, a student who, after a whole semester, is “undecided” about the difficulty of the course or some aspect of the teacher’s effectiveness obviously doesn’t have much of value to say about the course. Evaluations with “undecided” checked can then be discarded from the results.

Evaluations have to be controlled for non-serious ratings. As I said above, one method (admittedly sloppy) is to use the comments as a context for the evaluation. This is not really possible when the administration is using the evaluations to rate teacher effectiveness, however, and not all students write comments, so it only acts as a defective filter. Another option would be to trim the evaluations, that is, remove the top and bottom scores from a teacher’s evaluations when calculating overall scores. The problem with this method is trimming too much, easy to do if one is teaching small courses of, say, only forty students (removing the top two and bottom two from forty evaluations is the maximum). The problem is that with smaller classes — say again forty students — you aren’t trimming enough data to control for the variable (but for larger classes, like 240 students, this works well).

Perhaps the best solution is to rate all faculty after all of the evaluations are in and analyzed. Calculate population means and confidence intervals for the different criteria, and compare faculty member scores not to other faculty member scores, but the population means. This will neutralize the “lazy/disaffected student factor” and rate all faculty members on the same scale.

But because evaluations are subjective, it is also important not to use them as a sole measure of teaching effectiveness. Use them, yes, but use them along with another, more objective, measure, such as test scores. If you are an administrator and you do this, I suspect you will find, as our department did, that there will be a high correlation between student evaluations and test scores.

Evaluations should be processed individually by class, and should not be processed by instructor. John Smith may be teaching Algebra and Geometry, and may get very different evaluations in the two classes. He should therefore receive evaluations for each of his classes, instead of an overall evaluation. Even if he is teaching three Algebra courses, and all are the same content, the three should be processed individually. Excessive aggregation masks differences in data, and obscures the results of the evaluations. Aggregating evaluations by class also allows the administration to spot faculty members’ strengths and weaknesses.

Evaluations as faculty feedback

Student evaluations can provide faculty with crucial information. Remember that perceptions differ on either side of the desk. You may believe that you are being clear, but that doesn’t mean your students find you clear. The same holds for almost any criterion you can put on an evaluation.

The biggest problem with student evaluations is their infrequency. Only once at the end of the semester really isn’t helpful for your current classes, although you can use the information to improve your teaching for future classes. Another problem is that they provide too little data. As an example, let’s say you read your evaluations and find that students don’t think you’re as clear as you could be. The evaluations do not tell you much about how you could become clearer.

For these reasons, I administered informal, short-answer only, evaluations three times throughout the semester: A quarter of the way through, around midterms, and three-quarters of the way through. I explained to students that the evaluations were only for me and them, so I could find out from them how I could better present the material, and I encouraged them to be as detailed as they felt they needed to be. I got some crap, certainly, but I also got some valuable feedback about where and how I was being unclear or unhelpful, and adjusted my teaching accordingly.

I also always asked my students when they came to office hours (after we had dealt with their problems) how I could have better presented the material they had come to ask about. After all, if they didn’t have a problem, they would not have come to my office for help, and if they have a problem, then I assume I could have done more to prevent it in the first place. Never miss the opportunity to ask your students how you’re doing. Your students are not idiots. If you ask, they will have some important things you need to hear. Stop and listen.

Also, take evaluations into account along with your students’ test or assignment scores. If your students’ grades are consistently low, even if you teach a difficult course, you could be more effectively presenting the material. Poll your students even more frequently, and find out what they think you could be doing. Use the test or assignment results to tell you what your students are not getting. Keep your thumb on the pulse of the class — and you can’t do that if you’re not paying close attention.

Evaluations as consumer information

The university differs in one crucial way from the elementary, middle, or high school: University students have the luxury of choice. The student can choose his major and degree program. The student can choose his courses based in part on which faculty members are teaching them. After all, if you have to take M125 and three faculty members are teaching it, why not take it with the best of the three, or at least not take it with the worst?

When I was an undergraduate, that choice was mitigated by a lack of technology, and we were limited to word of mouth. Now we have the web, and RateMyProfessors.com, and many universities have their own forums for rating faculty. Today’s university students is far more informed about faculty than we were, and can make better choices.

Of course, there is that silly objection to students as consumers, but face it, that’s exactly what they are. The course is for them, after all, not your ego. That alone makes them consumers, and you, the provider. If you’re a godawful teacher, but your colleague is a really good teacher, then students have every right to avoid you and take your colleague’s class instead, and you would do exactly the same.

Student evaluations are, if anything, underused. They are certainly underused in public schools, particularly where teachers’ unions block any kind of teacher evaluation, and just as underused by universities with respect to tenured faculty who should be seriously evaluated on their teaching skills, but even more importantly, they are underused by well-meaning teachers, who could be using them as a powerful tool for improving their own effectiveness. If you teach, and if you care whether you are a good teacher, start asking your students for guidace, and take their answers seriously.

But how useful would student evaluations be in the public schools? Certainly not as useful as they are at the university. The older a student is, the more capable he is of judging how well he is learning the material. Student evaluations would be of very little use in the elementary grades, but more useful in high school. Still, they could be used, along with other, more heavily weighted measures, like scores or even peer or adminstrative review, to give a more complete picture of a teacher’s effectiveness.

As with any measure, the larger the sample space, the more reliable the results. This points up one difference between the public schools and the university: Class size. While there are a great many smaller university classes, particularly at the upper levels, there are also a great many large classes. Few, if any, public school teachers have over 200 students in a class.

This implies that student evaluations could be used more effectively as filters in the public schools, since smaller classes are more likely to exhibit non-normally distributed learning than large classes (the same is true at the university). That is, a teacher who teaches five small classes (40 students or fewer) is more likely to have, say, one class significantly below the others than is a teacher who has five lecture halls full of students (200 or more in each). Properly designed, student evaluations could be used to identify non-serious students who would otherwise pull down a teacher’s rating.

I dare say there are many more ways in which student evaulations could effectively be used. They are undoubtedly underused, and even where they are used, used poorly. If we are to become better teachers, and if our schools and universities are to become more effective places of learning, student evalutaions provide a unique and powerful assessment tool.

is that unless you’re a full-time blogger on the specific topic, you may not have something topical to submit when the deadline rolls around. And me, I’m just not the kind of person to torment myself trying to think up some crap to send in just because the deadline is nearing.

Anyway, I watched Hard Times at Douglass High yesterday. All I can say is alien environment. What we at the university call discipline problems, well, the teachers in the documentary would just laugh. Sure, we get disengaged students (although most of those don’t come to class, so we don’t have to put up with it), but for the most part, the considerable amount of misbehavior on university campuses happens outside the classroom.

And that started me thinking. I always had a policy: Any time any colleague in the department needed a sub, I volunteered. I didn’t do this because I wanted to be altruistic. My motivation was purely self-interest. You see, I had everybody in the department owing me big-time and when I did go out of town — and while rare, those trips tended to be for a full week — I would always be able to get subs.

As a result of this policy, I have more experience than most in seeing other people’s classes. What was striking was how some people (no names) always had unruly classes. I never could figure out why.

I’m no storm trooper in the classroom. I don’t yell at students (even when they are being unruly), and I don’t get nasty. The thing is, my classes were never unruly. Never. Here’s my untested hypothesis to explain it.

Way back when I started teaching, I was overly familiar. That was no doubt because I wasn’t that much older than my students. That didn’t work, or at least, I wasn’t at all effective at controlling the classroom.

As the age difference between me and my students increased, I became more formal and authoritarian. I swung way too far in that direction, and found that it didn’t help me control the class. I also wasn’t comfortable with it.

Over the years, I found an effective balance between the two extremes. I am warm and cordial with students, but I maintain a professional distance. I am not on a first-name basis with them (although I don’t insist that they address me in any specific way). I address all of my students as Mr. Jones and Ms. Wilson — not to be cold or formal, but respectful. I never talk down to students, even when they are being stupid (and yes, they are sometimes incredibly so). I made sure that they understood that my highest priority was not only to teach the material, but to do everything I could to see that they learned it, and succeeded.

I treat my students like adults. Not peers, because we are not peers, but adults.

The surprising thing is that they respond to that. When I walk into the room, students stop talking. Well, that’s my classroom. When I walk into many of my colleagues’ classrooms, the talking continues until I have said, “Let’s start,” four or five times. In my classroom, when I ask a question, students respond. In those other classrooms, I have to ask repeatedly, and often have to threaten to “volunteer” somebody.

Students come to my office hours, sometimes just to talk. They greet me in the hall or the library. Until I left the university, ex-students frequently dropped by my office to chat and let me know what was going on in their lives.

My hypothesis is that while students may not be adults in the fullest sense of the word, they appreciate being treated like adults, and they appreciate being treated with respect. If you set a respectful tone, students (in my experience) will rise to it. I had only a very few students that could be termed discipline problems. Some of my colleagues, on the other hand, at least to judge from daily conversation, had troublesome, disrespectful students on their hands all the time.

At meetings, they would ask things like, “How do I keep students from walking out early?” I could not imagine one of my students walking out early, unless he had asked me if he could before class started. It just never happened. Not once.

Note that my colleagues ribbed me a lot about the way I addressed my students. The thing is, they were the ones with the problems, not me.

Thats my hypothesis, anyway.

I had a troll a couple of months ago who left incoherent, ungrammatical, nonsense comments, which fortunately went into the moderation queue. Well, he’s suddenly back, and just to show you what an idiot he is, I decided to quote him here (but not approve the comment).

None of the cool kids want tthe repuiblicansz to sit with them anymore, sorry

Your public school system at work.

Remember that pure, unsullied by the outside world tribe they supposedly found in the Amazon last month? And what did I say?

Tasaday.

Guess what? I was right.

‘Lost’ Amazon tribe a publicity stunt
By staff writers
June 23, 2008 08:26pm

THE man behind photos of warriors from an “undiscovered” Amazon tribe that were beamed around the world has admitted it was a publicity stunt aimed at raising awareness of logging.

Indigenous tribes expert, José Carlos Meirelles, said the tribe had been known of since 1910, and had been photographed to prove that they still existed in an area endangered by logging, The Guardian reported.

Mr Meirelles, who was working for Funai, the Brazilian Indian Protection Agency dedicated to finding remote tribes and protecting them, said he spent three years gatheiring “evidence” about the tribe, and then planned the publicity to protect them from losing their habitat.

One more time, folks, with feeling.

The Intellectual Dishonesty of Anthropologists

More Intellectual Dishonesty

There’s been a great deal of talk lately about universities’ making SAT scores optional for admission or dropping them entirely; today, Kimberly Swygert pointed to this article in the WSJ. Many of the articles about this topic have been of the moaning variety, largely because the authors are making unwarranted assumptions.

Let’s get a couple of things out of the way first, points I will expand in a moment. Dropping SAT scores as an admission requirement is distinctly different from lowering standards. Dropping the SAT is not in any way comparable to “education gaps” in the public schools. Finally, it makes no difference how well the SAT predicts student performance at the university.

The dirty little not-so-secret secret on campus is this: Underprepared students drop out, and do not complete degrees. I call it a not-so-secret secret because you almost never see reference to this anywhere but on campus. Whenever you see an article about “diversity among the student body,” it is always about admissions.

This is because, unlike the public schools, universities are not responsible for failed students: Going to college is your chance, and if you blow it, you blow it. I dare say that on every university campus every year, there are a couple of moaning, liberal, hand-wringing articles or editorials in the campus paper about retaining minority students. Administrations set up task forces, schools and colleges and departments create committees, all to study retention, but these task forces and committees don’t accomplish any policy and affects retention, for the one, painfully obvious reason: Students drop out because they are overwhelmed, and they are overwhelmed because they are not prepared for university work.

But in “The College Paradox: Not Everyone Gains By Higher Education,” an article about egalitarian v. tracking in education and not SAT scores, Steve Sailer puts his finger on the dirty little secret secret:

The prestige of Harvard and the other apex predators at the lofty pinnacle of the American educational pyramid means that the vast K-12 bottom has been infected with Harvard’s values (such as abstraction and abstruseness) and rhetoric (equality uber alles)…but not, alas, Harvard’s brains. Most of the K-12 educators, much less their students, aren’t smart enough to get the joke. They don’t understand that the IQ elitists of America are pulling the wool over their eyes when they rattle on about their purported liberal beliefs about how everybody should go to college.

They don’t understand it’s all a big pyramid scheme. The Harvard professors’ graduate students become the UCLA professors whose graduate students become the Cal State LA professors whose students become the schoolteachers who browbeat their more gullible pupils into believing that everybody should go to college, no matter how obvious a waste of money and time it will turn out to be.

Students with below average IQs [or underprepared students, regardless of IQ] are just the cannon fodder that keeps the system churning along for the professors.

Exactly. Dropping the SAT as an admission requirement is a win-win situation for the university. It’s a win-win because the university wins on two fronts: PR (and latte liberal feel-good-ism), and financial.

On the PR front, spewing nonsense about “diversity” accomplishes two things. First, it gives everybody something to feel guilty about, and as we all know, guilt is the primary motivator for upper middle class over-educated latte liberal academics, and as with any other source of guilt, it gives them something to “study” in those pointless task forces and committees. Everybody can pat himself on the back for “making a difference.” Second, it brings in lots of contributions from external sources, one of them guilty, wealthy latte liberal alumni, who give their alma mater more money because they are “making a difference,” and of course, because the contribution itself is “making a difference.” An extremely liberal alumnus of one Ivy League university who started a software company and did very well for himself donated several million dollars to his alma mater for setting up a program to study how the university could increase “diversity” in the student body (no names, of donors or universities).

The university also wins financially. Students are revenue. The more students a university enrolls, the more revenue the university rakes in — and the university always gets their revenue. Yes, most of the underprepared students will drop out, but the university has a freshman class every year. As Sailer points out, although talking about a completely different topic, it’s a pyramid scheme. It doesn’t matter that most of the underprepared freshmen won’t make it through the year (they will pay for the year, or somebody will), because there will be more the next year.

Sailer again:

Rather than follow CCNY’s disastrous route [open admissions], they made the cheaper choice of paying off minorities with affirmative action. Simultaneously, and paradoxically, they became even more IQ elitist in choosing mainstream applicants.

And it is exactly the same with dropping SATs from admissions, or almost the same. SATs required or not, the students who have the knowledge and are prepared will still apply. It doesn’t matter that a handful of qualified students will not be accepted in order to make room for “more diverse” students, because elite universities, with few exceptions, don’t make their reputations or most of their income from undergraduate education, but from graduate education and research.

And universities can always expand the number of undergraduate students they accept.

I’m not saying that universities don’t care about the quality of their undergraduate programs, and I’m certainly not saying that undergraduate education isn’t a big cash cow. But the undergraduate program takes a back seat to graduate programs, and here’s why.

Why do people desperately want to get into Harvard, say, or MIT? Because they’re top universities. And why are they top universities?

Not because of the quality of their undergraduate programs, but because of their reputations. And they have those reputations precisely because of their faculty who are big names in their fields and win things like Nobel Prizes and the PhD students they turn out who go on to become big names in their fields and win things like Nobel Prizes (as as anyone who has ever been in a PhD program knows, much of the time, the research that got Professor Smith that Nobel Prize is done at least in part by his PhD students).

“Diversity” is an investment. So are tools to achieve it (which never do), like affirmative action and dropping the SAT as an admissions requirement.

I call this a dirty little secret secret because from the talk about universities in the edusphere, you would think that all universities do is educate undergraduates. No doubt this is because most of the edusphere is made up of public school teachers and parents. But universities aren’t primarily in the business of educating undergraduates. I wouldn’t call it a sideline, but it does fall beneath research and graduate education, for good reason.

Universities don’t have to downgrade their education standards for these students, because they drop out, and there will be a whole new crop next year. Universities throw them bones, like African-American Studies Programs, Women’s Studies Programs, etc., but at many universities, these programs don’t offer bachelors’ degrees, and even if they do, there’s very little that student can do with that degree if he graduates. Standards are being downgraded in some fields, certainly, but that has absolutely nothing to do with “a more diverse campus,” and is unrelated to this discussion.

Sailer is correct. The underprepared students are cannon fodder for the pyramid scheme. They drop out, and the university gets more money.

As a final note, I suppose this sounds cynical, but it’s not meant to be cynical. It’s realistic. Universities, unlike the public schools, are businesses. Dropping the SAT is good business. Dropping the GRE (or GMAT, etc.) certainly is not, and that’s why you see very few universities lowering admission standards for graduate school (Ann Althouse last month referred to a university law school that’s dropping the LSAT, but I gathered from the discussion that it is not in any way prestigious, and seems to be trying to make a name for itself solely by scoring political points).

rather, another reason why learning them (as opposed to mindlessly punching a calculator) is a good idea: On one of those house flipping shows, this woman was told that fifty gallons would be enough, so she bought fifty five-gallon buckets — thinking she was buying fifty gallons.

Dr Mercury of Maggie’s Farm fame has made all of the episodes of James Burke’s The Day the Universe Changed for download. They’re wmv files. I downloaded them and used Nero to transcode them onto DVD; we’re watching the first episode on TV now. You could just as easily watch the wmv files on your computer, of course. Dr Mercury has instructions posted.

James Burke is, of course, the creator and narrator of Connections, one of the most fascinating educational series to be televised. Burke’s Knowledge Web is a valuable resource of technological development and how technology molds us.

Burke is a science historian who takes a unique and fascinating approach to his topic. His approach to history is the world wide web to the encyclopedia. He sees history not so much as a chronological step-by-step development, but as interconnected seemingly unrelated developments that spur yet other seemingly unrelated developments. For example, how was Napoleon important to the development of the modern computer?

Napoleon’s troops in Egypt buy shawls and start a fashion craze.

In Europe the shawls get made on automated, perforated-paper control looms.

This gives an American engineer Herman Hollerith the idea to automate calculation using punch cards.

Which get used to control ENIAC, the first electronic computer.

Get em while you can. They won’t be online forever. Most of the episodes of Connections, by the way, are on youtube. Just search on connections and burke.

This is the best argument for home-schooling I’ve seen yet:

Danny belongs to a school club called Kids Against Pollution … For five days, club members car-pooled to the school by 6:45 a.m. and counted the number of cars entering and leaving the parking lot. They held up signs with slogans like “Hop on the Bus, Gus” and “Make a New Plan, Stan.”

[ . . . ]

Most members said they were worried about global warming. Aaron Kohn, 11, said that he had watched the movie “Waterworld,” about a future in which the polar ice caps have melted and most of the planet is underwater, and then researched on the Internet reasons the earth could flood.

When he read about global warming, Aaron said, he got scared.

Psssst, Aaron, it’s a movie, you know, fiction.

The public schools: Turning our youth into wacked-out jackbooted enviro-thugs.

Joanne Jacobs points to this interesting article about a school that is teaching students how to behave like students, and getting excellent results:

Top achievers, they found, had mastered a behavioral code that equaled school success. They spoke up in class. They balanced when to speak and when to listen. They turned toward the speaker. Those behaviors — not their brightness — separated them from their lower-achieving peers and enabled them to absorb information. If the school explicitly taught students those behaviors, White reasoned, wouldn’t they do better?

It’s an interesting article, and something that hadn’t occurred to me before — but we kind of assume students will have mastered those things by the time they get to college. It did remind me of something I noticed for years, though.

I have always been amazed — and a bit depressed — by the number of students who have no test taking skills. They’re the ones chewing on their pencils, the ones who never look at their watches, the ones who plod their way through and rarely get to the end of the exam. It’s like they’ve never taken a test in their lives.

One might think with increased emphasis on testing in the schools, we’d see the numbers of these students decrease. But no, we haven’t. I’d say anywhere from 1/3 to 1/2 of the students in any test administration have no test taking skills — and I don’t sit at the front of the class, I actively proctor, walking through the students and watching what they do closely.

In fact, there are so many that I started covering basic test taking skills in class whenever I had a few free minutes.

Why are the things I tell them news? Does nobody in the schools cover test taking skills, and if not, why not?

Look over the exam

Before you get your pencil anywhere near the scantron, scan (sorry) the exam. How many questions are there? Do some quick math and figure out how much time you can allot to each question. Never forget that you’re taking a timed exam, and keep an eye on your watch, or the clock at the front of the room.

Don’t rush

Yes, it’s a timed exam, but when you rush, you get careless. It’s easy to miss the “not” in the question (bad test design alert!), or just misread it. Mind the time, but don’t hurry yourself, and for God’s sake, when you read, read carefully.

KISS

You know what that acronym means, right? Keep it simple. If I had a dollar for every question a student missed because he over-analyzed it, or the distractors, I’d own my own island in the Caribbean. Interpret the question exactly as it’s written. If you start asking youself, “Do you mean in this circumstance, or in that circumstance?” then you’re over-analyzing it. Stop it. If the professor had meant that, he would have put it in the question.

Of course, you will run into poorly-worded questions from time to time. And if the question only makes sense in one or another specific circumstance, ask. But as a general rule, don’t try to out-think the questions.

Question types

There are three types of questions: Questions you can immediately answer, questions you think you could probably answer if you could think about them for a minute or so, and questions you will never be able to answer. If you don’t know now, you won’t know fifty minutes from now, unless there’s something nagging at you in the back of your head, and in that case, the question is the second type, not the third.

How you approach the question depends on which type it is.

A lot of people will tell you not to skip around. Nonsense. That’s exactly what you should do.

First, answer all of the questions of the first type (those you can immediately answer). When you encounter a question of the second type (you think you could answer it if you could think some more about it), mark it, and move on. When you encounter a question of the third type — you have absolutely no idea what the question or the distractors mean — answer B, statistically the most frequent correct answer on a multiple-choice exam. Go through the exam and answer all of the first-type and third-type questions.

Check your time, and calculate how much time you have left. Again, keep an eye on the clock.

Tackle the second-type questions. Answering them can involve a number of strategies, for example, try using the process of elimination, so you know which distractors are not correct, then focus on the remaining ones. If more than one distractor seems that it could be the answer, re-read the question and ask yourself which of the two is the best answer to that specific question. So if it’s a general question and one of the two possible answers could be true, but only in a specific set of circumstances, the other answer is most likely correct. Also, look for keywords in the question and the possible distractors. Absolutes (all, always, none, never, etc.) usually indicate incorrect answers (bad test design alert!)

Substitute. If the question is, “Which of the following is used for forecasting?” and the two options are “Exponential smoothing,” and “Pearson correlations,” substitute the possible answers in the question.

Associate. Memory is visual. Using the above question, if substitution didn’t work, visualize your notes. What did “Exponential smoothing” appear with? Moving averages? Aha! You know moving averages are used for forecasting, so you’ve just answered the question.

The “trick question” is largely a myth. There are lots of poorly written questions, but there are few faculty members with a sadistic streak who want to see you stumble. There are a few, but a lot fewer than students believe.

If the question reeks, ask

Even on a carefully written exam, there are poorly written questions. If you find a question that doesn’t make sense, or distractors that don’t make sense, raise your hand and ask the proctor. Many faculty don’t analyze their test results, so they don’t always know if there are bad questions unless you call their attention to them.

To second-guess or not to second-guess

If you did not rush, don’t second-guess your answers to questions of the first type. However, you’ll most likely have time left, so carefully go over some of those second-type questions, particularly the ones that gave you more trouble.

I said second-guess, which isn’t checking your answers. Yes, if you have the time, check your answers. If you feel good about your performance on the exam, leave. If you feel shaky, you may want to look over some of those second-type questions again.

And make sure you filled in all of the bubbles to the edge. Dark.

About those Bs

If each of the four distractors (we’re assuming four here) is equally likely to be the correct answer, and if there were (out of 100 questions) 16 you had absolutely no idea about, you would most likely get 4 of them correct. However, all four distractors are, in fact, not equally likely to be correct (unless you have a really anal professor who runs frequency analyses on his exams before he sends them to be printed, and not even I am that anal), and B is more likely to be correct than A, C, D, or E (on five-distractor exams). I don’t recall the overall probabilities for the four distractors, so I can’t be more specific, but the odds are you’ll get at least 4 of those questions correct if you answered B for all of them.

And since you have no idea what the answers to those 16 questions are, 4 or 5 correct beats 0 correct, doesn’t it?

However, if your professor uses guessing penalties, you’ll have to do a little math to calculate the risk, since the best strategy depends on what the penalty is. If you have more to lose by leaving the questions blank, use the B strategy (and vice versa). By the way, it’s best to do this before you go take the exam.

I think guessing penalties are a very bad idea, by the way, but some professors use them.

Anyway, those are some general test-taking strategies. I hope everybody is enjoying finals week!

and timely, considering that I just published an article on student evaluations yesterday. This over-educated idiot masquerading as a professor obviously needs a crash course in ethics:

The D reported yesterday on lecturer Priya Venkatesan (also undergrad ‘90 and a Med School researcher) who, in a series of strangely passive-aggressive group emails, announced a plan to sue her students for workplace harassment based on “intolerance of ideas.”

Now if you haven’t seen this yet, even if you’re as cynical as I am, you’ll be picking your jaw up off the floor when you find out what “intolerance of ideas” means.

Few of Venkatesan’s students deny disliking her; they just say it had nothing to do with race, gender, or any other federally-protected characteristic. Rather, the lecturer embodied that special brand of neurotic pedagogical tyranny that includes making rules against questions, refusing to interact with students, and, according to the D,

cancelation [sic] of class for a week after the class applauded a student who contradicted Venkatesan’s opinions about post-modernism

Spontaneous applause during a class on literary criticism? Obviously, there is something very wrong with this picture, so outrageously shocking as to shake Venkatesan to her very core: In a class at an Ivy League university, students were paying attention. Worse: They were engaged, and they cared.

“I was horrified,” Venkatesan said. “My responsibility is not to stifle them, but when they clapped at his comment, I thought that crossed the line … I was facing intolerance of ideas and intolerance of freedom of expression.” …She canceled class because the incident caused her “intellectual and emotional distress,” she said.

Then again, being outsmarted by a room full of eighteen-year-olds must be pretty humiliating. A kinder choice would have been emitting a spontaneous snore or two, then preoccupying themselves with a more innocuous form of disrespect, like text messaging during class or ostentatious yawning.

And what does this have to do with evaluations? Well, apparently student evaluations were what pushed her over the edge into drooling idiocy. Here are a couple:

Professor Venkatesan refuses to answer questions, does not respond to questions, and lectures by reading off her notes in front of her. She did not make me a better writer, she did not explain the concepts well, but she did manage to make my life a living hell.

She offered no help in class or in office hours for papers. When handed a hard copy she read the paper, said it was great, but then gave terrible grades to many students. Later on she began refusing to grade papers and gave the reason that judging by our peer editing abilities we didn’t need her help on papers. She missed/cancelled 5 or 6 classes and as a result the syllabus was squished into 3 weeks and she changed the final project about 4 times. A TERRIBLE CLASS.

I can see how being such a slobbering moron could cause one “intellectual and emotional distress.” Of course, the lawsuit will go nowhere, no matter how much “distress” she suffered, but the degree of oblivion this idiot exhibits is, frankly, breathtaking.

I might add that I had a student a few years ago who really hated my guts — I was never sure why, but she did. She hated me like poison. She really let loose on her evaluation too (yes, they’re anonymous, but it was obvious), but it never occurred to me to sue her. Or even raise hell about it. Life goes on.

But in quite a pleasant contrast to the above idiot, we have this, from (of all places) Inside Higher Ed:

What does it mean to be an American citizen?

From all the heat generated of late over immigration, one might have supposed that some light would have been cast on this crucial question. Given the need to elevate our national dialogue over this issue, it is disheartening that this has yet to happen. It appears that the idea that is American citizenship is all but lost on America’s citizens themselves. Here our universities can be of invaluable assistance, through introducing their students to the perennial questions and issues that define American democratic theory and practice.

Any attempt to perform this task ought to begin at the beginning, with the very justification for our existence as a country—the Declaration of Independence. Its claims are meant to be universal, addressed not only to King George III, but to a “candid world.” The Declaration argues that, in the new American order, blood, creed, and national origin—the constituents of citizenship throughout history—have been dethroned. Instead, U.S. citizenship entails adherence to moral and political principles the truth of which, says the Declaration, is “self-evident” to those who reason rightly. These principles, which form what can be called the “American theory of justice,” argue for human equality; for the inalienable rights to life, liberty, and the pursuit of happiness; for government established by popular consent; and for the right of the people to rebel should government cease to fulfill the purposes for which it was instituted. On this basis, the United States is more than a mere address, more than its history, and more than its demographics. It is, in its essence, an idea.

Yet how many of us today, native-born no less than newly arrived immigrants, can recount the Declaration’s four self-evident truths? More crucial, how many of us have even a rudimentary grasp of the moral and intellectual foundations of the “American theory of justice”? For years, surveys have told us that the answer to both questions is precious few. This cannot help but alarm those of us who believe, with the Declaration’s author, Thomas Jefferson, that no nation can expect to be “both ignorant and free.” But neither should we be surprised at the surveys’ results, says Derek Bok. The former president of Harvard University argues in his recent book, Our Underachieving Colleges, that American higher education is not providing the democratic or civic education on which he and Jefferson deem democratic health to depend.

[ . . . ]

For the sake of the integrity of both our universities and our politics — for our citizens both newly arrived and native-born — let us begin this quest, and let us do so in the civil, fair-minded, and magnanimous manner that defines university life at its noblest.

Do read the whole thing, because it’s really good — and coming from an academic, refreshingly sane. And you might be surprised at how supportive the comments are.

The Carnival of Ed is up.

Let’s talk evaluations, starting with this article from Inside Higher Ed:

But what if the much derided Web site’s rankings have a high correlation with markers that are more widely accepted as measures of faculty performance? Last year, a scholarly study found a high correlation between RateMyProfessors.com and a university’s own system of student evaluations. Now, a new study is finding a high correlation between RateMyProfessors and a student evaluation system used nationally.

A new study is about to appear in the journal Assessment & Evaluation in Higher Education and it will argue that there are similarities in the rankings in RateMyProfessors.com and IDEA, a student evaluation system used at about 275 colleges nationally and run by a nonprofit group affiliated with Kansas State University.

What is notable is that while RateMyProfessors.com gives power to students, IDEA gives a lot of control over the process to faculty members. Professors identify the teaching objectives that are important to the class, and those are the measures that count the most. In addition, weighting is used so that adjustments are made for factors beyond professors’ control, such as class size, student work habits and so forth — all variables that RateMyProfessors doesn’t really account for (or try to account for).

The study looked at the rankings of 126 professors at Lander University, in South Carolina, and compared the two ratings systems. The findings:

• Student rankings on the ease of courses were consistent in both systems and correlated with grades.
• Professors’ rankings for “clarity” and “helpfulness” on RateMyProfessors.com correlated with overall rankings for course excellence on IDEA.
• The similarities were such that, the journal article says, they offer “preliminary support for the validity of the evaluations on RateMyProfessors.com.”

It has been my experience that those who discount evaluations are bad teachers. In fact, I can think of many examples of faculty I know who discount evaluations and who are bad teachers, but I cannot think of even one counterexample. I do not know even one faculty member who is a good teacher and discounts evaluations. Not one. But I know lots of bad teachers who discount evaluations.

The most common statement about evaluations is that students care only about how easy the course is and what grade they get. But it takes little more than a quick perusal of professor ratings on RateMyProfessors.com to see that the assumption is a fallacy.

Let’s look at the categories RateMyProfessors.com uses to evaluate professors (1-5):

Easiness - Some students may factor in the easiness or difficulty of the professor or course material when selecting a class to take. Is this class an easy A? How much work do you need to do in order to get a good grade? Please note this category is NOT included in the “Overall Quality” rating.

Helpfulness - Helpfulness is defined as a professor’s helpfulness and approachability. Is this professor approachable, nice and easy to communicate with? How accessible is the professor and is he/she available during office hours or after class for additional help?

Clarity - A professor’s organization and time management skills can make a great difference on what you get out of the class. How well does the professor teach the course material? Were you able to understand the class topics based on the professor’s teaching methods and style?

Overall Quality - The Overall Quality rating is determined by the average rating of the Helpfulness and Clarity given by all users. An overall rating of 3.5 to 5 is considered good (yellow smiley face). An overall rating of 2.5 to 5 is considered average (green smiley face). An overall rating of 1 to 2.5 is considered poor (blue sad face). The Easiness rating is NOT included when calculating the Overall Quality rating.

Rater Interest - There is always that one class everyone recommends taking before graduating. As a student, how interested were you in the class, BEFORE taking it? Or how interested were you in taking this course from this specific professor.

So easiness is included, but crucially, it is not included in the overall rating. There goes the cornerstone of that major assumption about evaluations, that they are nothing more than evaluations of the ease of a course. My university evaluations are admittedly superior as rating tools: There are more questions, half of which relate to the course and the other half to the instructor, and they are more detailed. RateMyProfessors.com is by no means a rigorous tool for rating teacher effectiveness, but then, it’s a web application, and if there were twenty questions on it, nobody would use it.

I said above that a quick perusal of the evaluations would destroy the major assumption about RateMyProfessor.com, and here is an example:

We may assume that this student is a history major. Note that the student calls the professor “demanding,” and gives him only a 2 on easiness. Yet, this professor gets a glowing review. Here is another, from a different professor, at a different university, in a different field:

The first evaluation was (presumably) from a major, and an upperclassman. This evaluation is from a student who dislikes the course subject, and admits that he has a hard time with it. It also looks like this student is most likely a freshman or a sophomore, to judge from the course number (I replaced the letters, but not the first course number, because it indicates the level of the course). This student gives this professor the lowest possible score for easiness (1), yet like the last evaluator, gives the professor a glowing review. How is this possible, if evaluations are nothing more than a popularity contest, and students rate primarily on how easily they can get As?

But what this article misses is that one hears exactly the same complaints among faculty about university student evaluations as RateMyProfessors.com. It’s one of the Laws of the Faculty Lounge that if there are tenured faculty present and student evaluations come up, at least one professor will make a sneering comment. As a group, tenured faculty give little concern to teaching (which renders the whole “I want a real professor to teach my class!” argument ludicrous.). Don’t believe me? Here it is from the Chronicle of Higher Education:

Colleges are quick to argue that a college education is more about enlightenment than employment. That may be the biggest deception of all. Often there is a Grand Canyon of difference between the reality and what higher-education institutions, especially research ones, tout in their viewbooks and on their Web sites. Colleges and universities are businesses, and students are a cost item, while research is a profit center. As a result, many institutions tend to educate students in the cheapest way possible: large lecture classes, with necessary small classes staffed by rock-bottom-cost graduate students. At many colleges, only a small percentage of the typical student’s classroom hours will have been spent with fewer than 30 students taught by a professor, according to student-questionnaire data I used for my book How to Get an Ivy League Education at a State University. When students at 115 institutions were asked what percentage of their class time had been spent in classes of fewer than 30 students, the average response was 28 percent.

That’s not to say that professor-taught classes are so worthwhile. The more prestigious the institution, the more likely that faculty members are hired and promoted much more for their research than for their teaching. Professors who bring in big research dollars are almost always rewarded more highly than a fine teacher who doesn’t bring in the research bucks. Ernest L. Boyer, the late president of the Carnegie Foundation for the Advancement of Teaching, used to say that winning the campus teaching award was the kiss of death when it came to tenure. So, no surprise, in the latest annual national survey of freshmen conducted by the Higher Education Research Institute at the University of California at Los Angeles, 44.6 percent said they were not satisfied with the quality of instruction they received. Imagine if that many people were dissatisfied with a brand of car: It would quickly go off the market. Colleges should be held to a much higher standard, as a higher education costs so much more, requires years of time, and has so much potential impact on your life. Meanwhile, 43.5 percent of freshmen also reported “frequently” feeling bored in class, the survey found.

Ha! Students are idiots! What do they know? Read on:

College students may be dissatisfied with instruction, but, despite that, do they learn? A 2006 study supported by the Pew Charitable Trusts found that 50 percent of college seniors scored below “proficient” levels on a test that required them to do such basic tasks as understand the arguments of newspaper editorials or compare credit-card offers. Almost 20 percent of seniors had only basic quantitative skills. The students could not estimate if their car had enough gas to get to the gas station.

Unbelievably, according to the Spellings Report, which was released in 2006 by a federal commission that examined the future of American higher education, things are getting even worse . . .

There follows a lengthy discussion of what to do about the problem. But one thing that can be done, the one thing that is relevant here, is to start taking student evaluations seriously, and not just for teaching faculty.

Student evaluations are by no means perfect. Yes, there are students who care about nothing but how easily they can get As. But at least our evaluations, and the ones at RateMyProfessors.com, can be filtered. Our evaluations are done on a scantron, and on the back are four short answer questions:

1. What did you like most about this course/instructor?
2. What did you like least about this course/instructor?
3. What could the instructor do to improve the course or his or her teaching effectiveness?
4. Other comments?

The comments are linked to the evaluation scores on the other side of the scantron. Non-serious comments indicate a non-serious evaluation. In other words, one can safely ignore the evaluation of a student who answered, say, the second question with something like, “Give us all As.” And yes, you do get non-serious comments, because as I admitted, there are students who care about nothing but how easily they can get As.

However, that some students care about nothing other than the ease with which they can get As does not imply that all or even most students care about nothing other than the ease with which they can get As, nor does it discount the student evaluation as a rating tool. One would assume that somebody with a PhD would have enough basic intelligence to grasp that, but far too many do not — or they do, but would rather discount evaluations rather than face up to the fact that they might not be the brilliant teachers they think they are.

Tenured or teaching faculty, if you are a good teacher and get good evaluations, you have no reason to sneer at them. In fact, if you get good evaluations, yet claim that evaluations are nothing more than a popularity contest, then you are admitting that you are a poseur who does not take your career seriously and that you hand out As like candy. And how stupid would you have to be to do that?

Again, some evaluations are crap. But many of them can be filtered, because they are accompanied by crap comments. This is a useful, but sloppy filter, since not all students bother to write comments on the back. Still, one may filter some of the, ahem, less than useful evaluations.

Note that the same is true on RateMyProfessors.com: Each evaluation is accompanied by comments, and non-serious comments indicate non-serious evaluations. But also note that by no means are all comments non-serious. Neither are the comments on our evaluation scantrons.

At the university where I worked, student evaluations were required to be given in all classes. However, they were largely ignored for tenured (and tenure-track) faculty, but used as a major employment criterion for adjunct, or teaching, faculty. There is a certain amount of logic to this, except that it relieves tenured faculty of any review of their teaching skills. And unsurprisingly, it is always tenured faculty who sneer at evaluations. Teaching faculty do not, because if they get bad evaluations, they are not re-hired — and they shouldn’t be.

Of course, neither should the university harbor tenured faculty who consistently get bad evaluations, but that’s another topic, and addressing that problem would require at least a major redefinition of tenure, if not abolishing it altogether.

But back to student evaluations. They serve two major functions: They provide teaching effectiveness feedback for the department, school/college, and university, and they provide teaching effectiveness feedback for the faculty member. I, at least, would hold that both are equally important. At the university, they provide yet a third function.

Evaluations as school/university feedback

If evaluations are to be used administratively as teaching effectiveness rating tools, a number of issues must be addressed. Evaluations are subjective, after all. As I said earlier, some students care about nothing other than ease. Also, a statistician should design the evaluation, and not, say, a committee of teachers or the school board. Some outside agency should administer, or at least design and process, the evaluations.

Because of the nature and purpose of student evaluations, there should be no “undecided” option available to students. That is, the available options should force students to make either a positive or negative response of some degree. Student evaluations are not political polls, where “undecided” can often give useful information. Student evaluations are rating tools, and as such, should force a rating. After all, a student who, after a whole semester, is “undecided” about the difficulty of the course or some aspect of the teacher’s effectiveness obviously doesn’t have much of value to say about the course.

Evaluations have to be controlled for non-serious ratings. As I said above, one method (admittedly sloppy) is to use the comments as a context for the evaluation. This is not really possible when the administration is using the evaluations to rate teacher effectiveness, however. Another option would be to trim the evaluations, that is, remove the top and bottom scores from a teacher’s evaluations when calculating overall scores. The problem with this method is trimming too much, easy to do if one is teaching small courses of, say, only forty students (removing the top two and bottom two from forty evaluations is the maximum). The problem is that with smaller classes — say again forty students — you aren’t trimming enough data to control for the variable (but for larger classes, like 240 students, this works well).

Perhaps the best solution is to rate all faculty after all of the evaluations are in and analyzed. Calculate population means and confidence intervals for the different criteria, and compare faculty member scores not to other faculty member scores, but the population means. This will neutralize the “lazy/disaffected student factor” and rate all faculty members on the same scale.

But because evaluations are subjective, it is also important not to use them as a sole measure of teaching effectiveness. Use them, yes, but use them along with another, more objective, measure, such as test scores. If you are an administrator and you do this, I suspect you will find, as our department did, that there will be a high correlation between student evaluations and test scores.

Evaluations should be processed individually by class, and should not be processed by instructor. John Smith may be teaching Algebra and Geometry, and may get very different evaluations in the two classes. He should therefore receive evaluations for each of his classes, instead of an overall evaluation. Even if he is teaching three Algebra courses, and all are the same content, the three should be processed individually. Excessive aggregation masks differences in data, and obscures the results of the evaluations. Aggregating evaluations by class also allows the administration to spot faculty members’ strengths and weaknesses.

Evaluations as faculty feedback

Student evaluations can provide faculty with crucial information. Remember that perceptions differ on either side of the desk. You may believe that you are being clear, but that doesn’t mean your students find you clear. The same holds for almost any criterion you can put on an evaluation.

The biggest problem with student evaluations is their infrequency. Only once at the end of the semester really isn’t helpful for your current classes, although you can use the information to improve your teaching for future classes. Another problem is that they provide too little data. As an example, let’s say you read your evaluations and find that students don’t think you’re as clear as you could be. The evaluations do not tell you much about how you could improve.

For these reasons, I administered informal, short-answer only, evaluations three times throughout the semester: a quarter of the way through, around midterms, and three-quarters of the way through. I explained to students that the evaluations were only for me and them, so I could find out from them how I could better present the material, and I encouraged them to be as detailed as they felt they needed to be. I got some crap, certainly, but I also got some valuable feedback about where and how I was being unclear or unhelpful, and adjusted my teaching accordingly.

I also always asked my students when they came to office hours (after we had dealt with their problems) how I could have better presented the material they had come to ask about. Never miss the opportunity to ask your students how you’re doing. Your students are not idiots. If you ask, they will have some important things you need to hear. Stop and listen.

Also, take evaluations into account along with your students’ test (or course assignment) scores. If your students’ grades are consistently low, even if you teach a difficult course, you could be more effectively presenting the material. Poll your students even more frequently, and find out what they think you could be doing. Use the test or assignment results to tell you what your students are not getting. Keep your thumb on the pulse of the class — and you can’t do that if you’re not paying close attention.

Evaluations as consumer information

The university differs in one crucial way from the elementary, middle, or high school: University students have the luxury of choice. The student can choose his major and degree program. The student can choose his courses based in part on which faculty members are teaching them. After all, if you have to take M125 and three faculty members are teaching it, why not take it with the best of the three, or at least not take it with the worst?

When I was an undergraduate, that choice was mitigated by a lack of technology, and we were limited to word of mouth. Now we have the web, and RateMyProfessors.com, and many universities have their own forums for rating faculty. Today’s university students is far more informed about faculty than we were, and can make better choices.

Of course, there is that silly objection to students as consumers, but face it, that’s exactly what they are. The course is for them, after all, not your ego. That alone makes them consumers, and you, the provider. If you’re a godawful teacher, but your colleague is a really good teacher, then students have every right to avoid you and take your colleague’s class instead, and you would do exactly the same.

Finally, I think that student evaluations are, if anything, underused. They are certainly underused in public schools, particularly where teachers’ unions block any kind of teacher evaluation, and just as underused by universities with respect to tenured faculty who should be seriously evaluated on their teaching skills, but even more importantly, they are underused by well-meaning teachers, who could be using them as a powerful tool for improving their own effectiveness. If you teach, and if you care whether you are a good teacher, start asking your students for guidace, and take their answers seriously.

Middle school students in Texas beat a student, and upload it to youtube. We could go on forever about the incident itself, and how many years these scumbags should spend in prison, but let’s look at what the administrators say in response:

RAYMONDVILLE - Administrators of the Raymondville school system said they will review the district’s cell phone policy after a report of an assault on a middle school student that was recorded with a cell phone and then displayed on YouTube.

It was at least the second time this year students from the school district uploaded violent videos to YouTube, said school board president John Solis. Early this year, a Raymondville High School student used a video to solicit someone to beat another student.

Somehow, I don’t think the cell phone policy is the problem here, do you?

Jaw-droppingly lame?

Anyway, the asskickinest new show on TV is DEA on Spike. The cameramen put on bullet-proof jackets and follow a DEA team around Detroit, and documenting them busting scumbags. That all by itself would be asskickin, but one of the agents, Woody, is one of the smoothest guys you’ve ever seen as he talks the bipedal excrement into flipping on their suppliers.

So where does jaw-droppingly, speechlessness-inducingly, fry-your-brain lame come in? In this, ahem, “review” I found while looking for clips. Oh, wait. Before I quote this . . . well, st00pidity, let’s note who wrote it:

Spike’s ‘DEA’ fails to capture Detroit
Mekeisha Madden Toby / Detroit News Television Critic

Got that? This drooler actually gets paid to write this drivel. Think about that as you read:

Spike’s ‘DEA’ fails to capture Detroit
Mekeisha Madden Toby / Detroit News Television Critic

The one truly authentic Detroit moment in Spike TV’s new reality crime series “DEA” comes neither from the endless obligatory shots of the city’s skyline nor the occasional pan of dilapidated buildings along Hamilton Avenue.

The essence comes through when Detroit DEA agents are on a stakeout waiting for a confidential informant to arrive and a man from the neighborhood comes up and asks what the men are doing.

First, the cops lie and say they are just checking out the buildings. When this doesn’t work, DEA agent Rick “Woody” Gatewood tells the man that he and his partner are from Channel 7 and they’re following up on a story.

A true Detroiter, the man clearly doesn’t believe him and asks Gatewood (who seems to own and wear every Detroit T-shirt in existence) for proof.

“You got a business card?” the man asks. Of course he doesn’t. Eventually, the man goes away and so, too, does the little Detroit flavor the series possesses.

While the narrator spends a great deal of time at the beginning of this Al Roker-produced series telling viewers how dangerous Detroit is, how it has the highest murder rate in the country and how many hundreds of thousands of illegal guns there are on the street, Roker and his producers fail to establish a sense of place. Our Motown is so much more than a list of frightening statistics.

Yes, we get it. Detroit is the big bad city and these DEA agents — who are shadowed for six episodes — risk their lives every time they bust a drug house or make an arrest. This is indisputable. The producers and camera people go to great lengths, themselves wearing bullet-proof vests, to prove to us how much is on the line.

Unfortunately, the same care isn’t taken when accurately portraying the most crucial character in the whole production — Detroit itself.

You can reach Mekeisha Madden Toby at (313) 222-2501 or mmad den@detnews.com.

How, exactly, does this moron come to believe that it’s somehow the show’s obligation to “capture Detroit”? Look, idiot, see the name of the show? DEA? The show could be filmed in Milwaukee and it wouldn’t make any difference, because the DEA is the point of the frakking show. Is it called Detroit? No? And why do you suppose that might be, idiot?

But if you look a bit beyond that, this “review” is even st00pider than it appears. What does she think this is, a National Geographic show? What, exactly, would “capture Detroit”? Quaint local savages in quaint local garb doing quaint local dances? Aging, hippies with grey hair and sagging breasts selling “organic” food at a farmers’ market? Insufferable latte-sipping, sushi-sucking liberals noshing at the local Starbucks? Your criminal mayor banging his ho’s in his office?

This, ladies and gentlemen, is what’s wrong not only with newspapers, but universities. Mindless mouthbreathers like this manage to get journalism degrees. And how much would you like to bet that this idiot “reviewer” faints and slobbers all over herself at the mere mention of Obama?

Anyway, catch DEA. It kicks serious ass — literally.

“The Democrats seem to have a living, breathing definition of the word filibuster.” And the pictures are, well, priceless.

and everything is assigned a number at the Carnival of Ed this week.

In my experience, too few teachers give enough thought to assessment — not standardized exams, but the assessments they design for their classes. If you’re one of these teachers who hasn’t really thought much about it beyond writing exams or quizzes, read on.

Curriculum, pedagogy, and assessment are the holy trinity of the classroom, and any must be designed in the context of the others. You shouldn’t spend your class time covering things not on the curriculum.

The same is true of assessment. But before we get into details, there are some issues that need to be addressed.

Assessment goals

There is more to assessment than just finding out how well your students have learned the material. Assessment is feedback. If you assign, grade, and return work regularly, students have a much better idea of how well they’re doing, and what their strengths and weaknesses are. A student who knows his strengths and weaknesses knows what he needs to work on, and how he can improve his performance. Assessment isn’t just for you, or your class. Assessment is also for your students. Keep that in mind as you design your class.

This is the reason I never design exam-only classes. Exams are good for finding out how well your students have learned the material, but they’re not good feedback mechanisms for students. I prefer that my students get feedback more or less weekly. I say more or less because the first week is a throwaway week (few students have the book the first day of class, you have to deal with administrative issues), and the weeks before the exams need to be reserved for review. With a 14-week semester, ten is a good number for some kind of assessment that will serve as student feedback.

Assessment should also serve as an incentive for students. If you give quizzes in class, but your quizzes are worth only 5% of the total, then those quizzes will not motivate your students to come to class (unless you have exceedingly anal high achievers).

In-class v. out-of-class work

In many ways, in-class assessment is superior to out-of-class assessment. You can control in-class assessment (provided that you’re paying attention), so you know students are doing their own work, and the assessment will reflect what they know, and not what their neighbors (or parents) know.

The disadvantage of in-class assessment is the constraint on complexity because of the time constraint. Your assessment cannot take more time than you have. If you need to test knowledge that exceeds the constraint, you have to use out-of-class assessment.

The major disadvantage of out-of-class work, of course, is that it is uncontrolled. Exams are a check on this: If the student got an A on the out-of-class work, but failed the section on the exam that tested the same knowledge, then somebody else did his out-of-class assignment for him. But you can assess more complex knowledge on out-of-class work than in-class work, which may or may not be a major consideration in your class. And you can minimize the inflation caused by out-of-class assignments.

I prefer to use a mix. Let’s take a stats class as an example. Two practical exams (done in a computer lab on computer files, not multiple-choice exams), ten in-class exercises, and ten out-of-class assignments. Students get two sources of feedback (exercises and assignments) more or less weekly, and the exercises act as an additional check on student knowledge (as opposed to parent or neighbor or friend down the street knowledge).

Fundamental Principle of assessment

Out-of-class assessment should never exceed 50% of the total. If it does, the assessment has become a joke. If your principal has some kind of idiotic rules that maximize the weight you may give exams, give in-class exercises or quizzes to make up the difference. And one more time, just in case you missed it.

Out-of-class assessment should never exceed 50% of the total.

This applies even to classes where subjective assessment is necessary or desirable. If you teach a history class and assign a paper, that’s great, but also assign in-class work. You can give essay or short-answer exams in class, but the paper should not exceed 50% of the total class assessment, or in many cases, that total assessment will not reflect student knowledge.

Assessment design

The midtem and final exams test the material covered in the first and second halves of the semester. The in-class exercises and out-of-class projects focus on a narrower field of knowledge, what was covered immediately previous to the assessment. I prefer that non-exam assessments be as tightly focused as possible, say an exercise and a project for each chapter. These assessments are for the students, so they can always have a clear idea of how they’re doing. The more tightly focused the work is, the easier it is for students to track themselves.

I always write previous knowledge into these assessments, however, just as I refer in class back to something we’ve done previously as frequently as I can. I don’t teach students a series of discrete, unrelated skills, but a body of knowledge that unfolds over time. Each new topic is based upon older topics. I reinforce the relationships from topic to topic because the relationships are part of the knowledge students need.

For a stats class, practical exams are a more accurate knowledge assessment than written exams. Practical exams test whether students understand the material and can appropriately analyze data. Written exams remove this hands-on assessment to an abstract level. All of the work for this class is practical, and not abstract. The exercises and projects are problem sets. If you’re teaching a history class, you may very well want to assign short-answer and essay exams and papers. The principles, however, are the same. Keep your assessment accurate and honest.

Watch your wording. You’re assessing the class material, and unless you’re teaching English, avoid language that could complicate assessment by throwing up roadbloacks (class-related jargon, of course, is exempt from this): Don’t use "dearth," but "lack." You can avoid this directive when designing out-of-class work, because they have the time to work through the language, but on in-class work, keep your language direct, simple, and clear.

This isn’t "dumbing down" the class. It’s assessing the class material, and only the class material. If you feel your students are nearly illiterate, and many of us do, take it up in faculty meetings, not on your exams.

Weighting assessments

Too often, weighting assessments is nothing more than making sure the numbers come out. But there is far more to it than making sure your points add up. Let’s look at a few possible weights for the assessments in the stat class (1000 points possible).

There are a couple of problems here. First, the exams are weighted too heavily, and the exercises and projects too lightly. Students assign worth to class work based on how it’s weighted. A student who comes to class and does the exercises and takes the exams but doesn’t bother to do the projects (or doesn’t come to class, but does the projects and takes the exams) can theoretically do well in this system, as any student can see. This system sends a clear message, that you don’t value their attendance in class, and you don’t place much value on their out-of-class work.

There’s another problem. The exercises and projects are both ten points each, and a hundred points total. But exercises are in-class assessments, and because of time constraints, are simpler than out-of-class projects. In this system, the weighting does not reflect the complexity of the work.

Here’s another one.

This system solves the two problems with the last: Exams are half the grade, and exercises and projects are the other half; and the more complex projects are weighted more heavily than the in-class exercises. Yes, this system is better than the last, but it is also problematic as an assessment of student knowledge.

The problem is that out-of-class projects are weighted too heavily. With no control over these assessments, a student can get somebody else to do his projects and come out of the class with an inflated assessment. The exams are not weighted heavily enough to counter the potential grade inflation. Also, the project and exercise weights do not accurately reflect the difference in complexity. Projects are more complex, yes, but not that much more so than exercise or exam problems.

Here’s one more.

This system is nearly ideal. Exams are weighted more heavily than weekly assessments, but not so much more so that students will be tempted to blow off exercises or projects. Projects are 20% of the final assessment, twice as much as exercises, which accurately reflects the comparative complexity, and they are not weighted so heavily that a student whose parents or friends do his work for him can counter the more accurate knowledge