In my experience, too few teachers give enough thought to assessment — not standardized exams, but the assessments they design for their classes. If you’re one of these teachers who hasn’t really thought much about it beyond writing exams or quizzes, read on.
Curriculum, pedagogy, and assessment are the holy trinity of the classroom, and any must be designed in the context of the others. You shouldn’t spend your class time covering things not on the curriculum.
The same is true of assessment. But before we get into details, there are some issues that need to be addressed.
Assessment goals
There is more to assessment than just finding out how well your students have learned the material. Assessment is feedback. If you assign, grade, and return work regularly, students have a much better idea of how well they’re doing, and what their strengths and weaknesses are. A student who knows his strengths and weaknesses knows what he needs to work on, and how he can improve his performance. Assessment isn’t just for you, or your class. Assessment is also for your students. Keep that in mind as you design your class.
This is the reason I never design exam-only classes. Exams are good for finding out how well your students have learned the material, but they’re not good feedback mechanisms for students. I prefer that my students get feedback more or less weekly. I say more or less because the first week is a throwaway week (few students have the book the first day of class, you have to deal with administrative issues), and the weeks before the exams need to be reserved for review. With a 14-week semester, ten is a good number for some kind of assessment that will serve as student feedback.
Assessment should also serve as an incentive for students. If you give quizzes in class, but your quizzes are worth only 5% of the total, then those quizzes will not motivate your students to come to class (unless you have exceedingly anal high achievers).
In-class v. out-of-class work
In many ways, in-class assessment is superior to out-of-class assessment. You can control in-class assessment (provided that you’re paying attention), so you know students are doing their own work, and the assessment will reflect what they know, and not what their neighbors (or parents) know.
The disadvantage of in-class assessment is the constraint on complexity because of the time constraint. Your assessment cannot take more time than you have. If you need to test knowledge that exceeds the constraint, you have to use out-of-class assessment.
The major disadvantage of out-of-class work, of course, is that it is uncontrolled. Exams are a check on this: If the student got an A on the out-of-class work, but failed the section on the exam that tested the same knowledge, then somebody else did his out-of-class assignment for him. But you can assess more complex knowledge on out-of-class work than in-class work, which may or may not be a major consideration in your class. And you can minimize the inflation caused by out-of-class assignments.
I prefer to use a mix. Let’s take a stats class as an example. Two practical exams (done in a computer lab on computer files, not multiple-choice exams), ten in-class exercises, and ten out-of-class assignments. Students get two sources of feedback (exercises and assignments) more or less weekly, and the exercises act as an additional check on student knowledge (as opposed to parent or neighbor or friend down the street knowledge).
Fundamental Principle of assessment
Out-of-class assessment should never exceed 50% of the total. If it does, the assessment has become a joke. If your principal has some kind of idiotic rules that maximize the weight you may give exams, give in-class exercises or quizzes to make up the difference. And one more time, just in case you missed it.
Out-of-class assessment should never exceed 50% of the total.
This applies even to classes where subjective assessment is necessary or desirable. If you teach a history class and assign a paper, that’s great, but also assign in-class work. You can give essay or short-answer exams in class, but the paper should not exceed 50% of the total class assessment, or in many cases, that total assessment will not reflect student knowledge.
Assessment design
The midtem and final exams test the material covered in the first and second halves of the semester. The in-class exercises and out-of-class projects focus on a narrower field of knowledge, what was covered immediately previous to the assessment. I prefer that non-exam assessments be as tightly focused as possible, say an exercise and a project for each chapter. These assessments are for the students, so they can always have a clear idea of how they’re doing. The more tightly focused the work is, the easier it is for students to track themselves.
I always write previous knowledge into these assessments, however, just as I refer in class back to something we’ve done previously as frequently as I can. I don’t teach students a series of discrete, unrelated skills, but a body of knowledge that unfolds over time. Each new topic is based upon older topics. I reinforce the relationships from topic to topic because the relationships are part of the knowledge students need.
For a stats class, practical exams are a more accurate knowledge assessment than written exams. Practical exams test whether students understand the material and can appropriately analyze data. Written exams remove this hands-on assessment to an abstract level. All of the work for this class is practical, and not abstract. The exercises and projects are problem sets. If you’re teaching a history class, you may very well want to assign short-answer and essay exams and papers. The principles, however, are the same. Keep your assessment accurate and honest.
Watch your wording. You’re assessing the class material, and unless you’re teaching English, avoid language that could complicate assessment by throwing up roadbloacks (class-related jargon, of course, is exempt from this): Don’t use "dearth," but "lack." You can avoid this directive when designing out-of-class work, because they have the time to work through the language, but on in-class work, keep your language direct, simple, and clear.
This isn’t "dumbing down" the class. It’s assessing the class material, and only the class material. If you feel your students are nearly illiterate, and many of us do, take it up in faculty meetings, not on your exams.
Weighting assessments
Too often, weighting assessments is nothing more than making sure the numbers come out. But there is far more to it than making sure your points add up. Let’s look at a few possible weights for the assessments in the stat class (1000 points possible).
|
%-age
|
Points
|
|
| Midterm: |
40% |
400 |
|
| Final: |
40% |
400 |
|
| Exercises (10): |
10% |
100 |
(10 pts each) |
| Projects (10): |
10% |
100
|
(10 pts each) |
There are a couple of problems here. First, the exams are weighted too heavily, and the exercises and projects too lightly. Students assign worth to class work based on how it’s weighted. A student who comes to class and does the exercises and takes the exams but doesn’t bother to do the projects (or doesn’t come to class, but does the projects and takes the exams) can theoretically do well in this system, as any student can see. This system sends a clear message, that you don’t value their attendance in class, and you don’t place much value on their out-of-class work.
There’s another problem. The exercises and projects are both ten points each, and a hundred points total. But exercises are in-class assessments, and because of time constraints, are simpler than out-of-class projects. In this system, the weighting does not reflect the complexity of the work.
Here’s another one.
|
%-age
|
Points
|
|
| Midterm: |
25% |
250 |
|
| Final: |
25% |
250 |
|
| Exercises (10): |
10% |
100 |
(10 pts each) |
| Projects (10): |
40% |
400
|
(40 pts each) |
This system solves the two problems with the last: Exams are half the grade, and exercises and projects are the other half; and the more complex projects are weighted more heavily than the in-class exercises. Yes, this system is better than the last, but it is also problematic as an assessment of student knowledge.
The problem is that out-of-class projects are weighted too heavily. With no control over these assessments, a student can get somebody else to do his projects and come out of the class with an inflated assessment. The exams are not weighted heavily enough to counter the potential grade inflation. Also, the project and exercise weights do not accurately reflect the difference in complexity. Projects are more complex, yes, but not that much more so than exercise or exam problems.
Here’s one more.
|
%-age
|
Points
|
|
| Midterm: |
35% |
350 |
|
| Final: |
35% |
350 |
|
| Exercises (10): |
10% |
100 |
(10 pts each) |
| Projects (10): |
20% |
200
|
(20 pts each) |
This system is nearly ideal. Exams are weighted more heavily than weekly assessments, but not so much more so that students will be tempted to blow off exercises or projects. Projects are 20% of the final assessment, twice as much as exercises, which accurately reflects the comparative complexity, and they are not weighted so heavily that a student whose parents or friends do his work for him can counter the more accurate knowledge assessment of in-class exercises and exams.
Assessing student work
So your students have submitted the projects. Now what? Other than grading, that is.
Grading itself is a complex issue. Let’s start with the partial credit issue. Should you give partial credit, or give all-or-nothing credit?
Remember that the exericses and projects serve as much as feedback for students as they do assessment. If you give partial credit on such assignments, students will have a much clearer, more detailed picture of what they do and do not know. I’m as sympathetic as any to the "no partial credit in the real world" argument, but it isn’t the real world. It’s your class. If you must have all-or-nothing problems, put them on the exams (and make sure your students know well ahead of time that they are all-or-nothing problems).
I don’t give all-or-nothing problems. Let’s say on an exercise, problem, or exam, a student does a one-tailed t-test rather than a two-tailed t-test, or Pearson correlations rather than Spearman, or does the correct test, but interprets the results with the wrong statistic. The student has an imperfect knowledge of the material, certainly, but he is moving in the right direction. He has a partial grasp of the material, but some grasp, and there is a galaxy of difference between using the correct test but interpreting the results with the wrong statistic and running a linear regression when the student should be doing a z-test. Both will get the wrong final answer, yes, but in the first example, the student is partially correct and on the right path, but in the second, he’s completely wrong in every way. To give him a zero on the problem when he is partially correct negates what he has demonstrated that he does know, even if the final answer is incorrect.
I do, however, apply gestalt to assigning partial credit. That is, I do not just divide a question into parts and assign some credit for each, but also assign some additional credit for getting a problem completely correct. That appropriately assesses students who have mastered the material and students who have only partially mastered the material.
If you assign partial credit, write a specific rubric for exactly how you will assign credit, and stick to it. And have some sort of mechanism in place so students can exactly what they missed and got correct, so they are, indeed, getting feedback, and not just a number or grade. I strongly suggest that you make all rubrics and grading policies available to your students. Nothing should be a mystery to them, and let me point out here that fairness is an ethical issue. That specific, detailed rubric will ensure fairness if you stick to it; after all, that’s why it exists. If it turns out that your rubric really sucks, change it next time around and if you haven’t yet created rubrics for future assignments use it to help you design better ones, but stick to it for this assessment. Don’t change course midstream.
In some fields, particularly mathematically and scientifically oriented fields, error cascade is also a grading issue. Error cascade is when one problem is based upon another or the input required for one problem is in whole or in part the output of an earlier problem, so that getting the second problem correct depends upon whether the student got the first correct. The issues, however, are much the same as for partial credit.
Error cascade is most likely in integrated problem sets. The difference between error cascade and partial credit is that in the former, students who get that first problem wrong are going to be excessively penalized when they get succeeding problems wrong because of the error. Students will object, and you had better have very good arguments in place for them. And have very good arguments in place for the Dean, because you will likely end up in his office as well.
If you want to use integrated problem sets, but don’t want to penalize cascading errors, then plug in the correct values for problems and check the resulting answers. If this sounds like too much work for you, then don’t assign integrate problem sets, or integrate them, but don’t give students problems whose correct solutions depend on their having correctly solved previous problems.
Finally, and I cannot overemphasize how important this is, assess your students’ work as soon as they turn it in, and get it back to them in as little time as possible. If they turn in projects on Fridays, have them back to students by the following Tuesday at the latest. Sure, I know it’s a pain in the ass, but feedback is not useful unless students have time to use it. There was an article on Inside Higher Ed last week about a university that is fining faculty for not turning grades in on time. I found it astounding. The wrath of the registrar, in my experience, is enough to ensure that faculty turn in grades on time (we were given 48 hours from the day the final was given to turn in final grades for the semester).
Oh. And have fun!
