?

Log in

No account? Create an account

Previous Entry | Next Entry

Today, I'd like to tell you a couple of brief stories that I hope will illustrate my feelings about grading and my rationale for favoring exams over homeworks (especially in grading) more and more over the years.

First story: why I give homework a value greater than zero

During my first year as an undergrad, I took Intermediate Programming in C (600.118). To make a long story short, I was not yet a good programmer. I dropped one assignment (an excellent one that I will post later) and I scored a middling 'B' on the hour exams. In the end, I was 0.15% short of an 'A-' in the course and the instructor, a Ph.D. student, gave me a 'B+'. I remember going in to go over the assignments and the final exam with the instructor and asking whether he could have given me any points back on the final, for three reasons:

  • 1. Even a couple of points would have raised me to an 'A-'.

  • 2. The lowest 'B+' was 0.15% higher than my score, but the second-lowest was about 0.5% higher than that. I felt the cutoff was a little statistically arbitrary, as it wasn't on an absolute scale.

  • 3. I actually felt I deserved an 'A-' because I had produced work of comparable quality to the 'B+' students.


The instructor, a Ph.D. student, declined, saying that:

  • 1. You have to draw the line somewhere. True, and it was his prerogative to do so.

  • 2. He couldn't have told you what his undergrad GPA was, yet he got into grad school. Probably true, but to each his (or her) own: my undergrad GPA (which helped get me into UIUC) was a 3.75 and my graduate GPA (which in fact does not matter, because as they say in grad school, "A is average, B is bad, and C is catastrophic") was a 4.0.

  • 3. I shouldn't care so much about grades. "If the story of your life was a book, this wouldn't even be worth a footnote in the appendix." This was incorrect, and here's my footnote: I resolved never to get a B again in a CS course - ever - and I didn't. Three and a half years later, I got my B.S. and M.S.Eng. in CS; eight years later, I got my Ph.D. in CS; nine years later, I signed my contract as a faculty member here at K-State; and fifteen years later, I was promoted and tenured.


A side effect of the above anecdote, though, was that I came out of the course feeling that homework had been worth too little for the amount of study, planning, and effort that we put into it. As Travis Bradshaw wrote, I didn't like that we did all that work seemingly to be ignored and for minimal credit.


Second story: why I've been giving homework less value each year

During my first year of grad school, I worked for one year as a research assistant (RA) in a programming languages group. I learned a few things about research in general, though I had not yet come into my own quality as a researcher. More important, though, I learned one or two things about learning.

I was a much better programmer using imperative programming languages such as C++ by this time, though I was still mediocre with functional programming. In other words, I could produce code as quickly as I've ever been able to, but I still wasn't really a good programmer yet in the sense of being able to "code smarter, not harder". Consequently, I was accustomed to use CC or g++ to "see how many bugs there were", etc.

A few things happened in 1993-1995 that really opened my eyes to this mode of behavior. The first was that Uday Reddy, my supervisor at the time, said to me that:
It shows a lack of understanding, and is a poor programming habit, to let the compiler do your work for you. The compiler is not a debugger, nor is it a style checker in the way of lint. Type error messages are not "feedback"; they are how the compiler breaks.

If you think about it, this is also true of marked test papers and homeworks. When someone makes an error on an exam or problem set, all we (the instructors, teaching assistants, and graders) can do is deduct or assign points according to the interpretation we deem most appropriate. This may be the harshest, or the most lenient, or - if we are particularly conscientious - the most accurate one.

A second thing that happened in fall, 1994 was that I worked for Sam Kamin as TA for CS325 (Programming Language Principles, a course between CIS 505 and CIS 705 here), I spent an inordinate amount of time grading. On one homework, I made out 110 "grading sheets" with comments on each of 8 or 10 parts. This earned me the ire of the head TA and a bemused explanation from the instructor on what they felt an appropriate level of detail was.

Since then, I have mellowed and matured a bit, and I no longer think the maximum level of detail is necessarily the most conducive to learning. I also see my dad's point of view better. He was adamant that homework should be worth zero, or a minimal amount of credit, not only because it was easier to plagiarize, but because it was really optional: students who have enough background and natural talent might better spend their time preparing for exams, working on a term project, or studying for other courses. This, too, is true, though I resisted it for years because I felt so short-changed by perenially having a solid 'A' in courses and dropping to 'A-' whenever we had a tough final exam. I never felt safe, and in math courses I would feel nauseatingly nervous going in to finals. (I ended up getting a 'B' in my ODE course and taking Numerical Analysis pass/fail because of this.)

A third thing that happened in 1995 was that while we were preparing for qualifying exams, zurich31 showed me a tape of a lecture by Edsger Dijkstra. In it, he reminded the programmer that "when your program fails, you made an error" and that they shouldn't call such errors "bugs" because it anthropomorphizes their own mistakes and externalizes responsibility. I think grades have a similar numbing effect: that is, it's easy to think that grades are assigned rather than earned.

My point is that scores alone are not good feedback, and "knowing how you stand in a course" is just about the poorest indicator of your own understanding that there is. Students, in my opinion, should learn to develop some self-assessment skills. This is especially true of grad students. I know I had poor self-assessment even in terms of comparing myself to other students when I was a first-year undergrad, but it was much better by the time I was a third or fourth year undergrad. By the end of grad school if not at the beginning, students should also be able to gauge their own knowledge "relative to the discipline" - based upon how well they know textbooks, and how broad and deep the scope of one's knowledge is relative to the literature. This skill took me at least another four years to begin to develop, and for me it's still gradually improving.

It's not always easy to know or guess the extent of one's ignorance. But "reinforcement learning by grades", so to speak, is a very weak method with low signal-to-noise ratio.

--
Banazir

Comments

( 9 comments — Leave a comment )
myng_rabbyt
May. 4th, 2006 07:29 pm (UTC)
This is an interesting discourse on how you view the tasks you give students, as far as projects and homework goes, and I appreciate it. But let me ask you this: why do you feel that exams are more adequate for proving the level of comprehension, retention, and expression of one's knowledge of a subject?
chaosinaskirt
May. 4th, 2006 09:21 pm (UTC)
I'm not banazir, but I can say that as far as project-oriented courses (like engineering and computer science) go, I don't believe that exams are more adequate for proving comprehension and rentention simply because the time constraints don't allow you to really get deep with what's going on. You have to simplify things so that the exam can be finished in an hour, or two hours, whatever the case may be. You can't necessarily get into the nuances between x over y with that little of time.

For the student, I'm not sure if it's fair either - because you can know and understand what's going on and make a stupid math error (which isn't so much a reflection of what you know, but how well you can use a calculator). It also isn't very realistic to how things will go in the real world. In the real world, there may be deadlines, but you've got resources galore - and if you need to put in a few late days to get the project done, so be it. You don't necessarily have the luxury of double or triple checking your work. You don't have the almighty computer to help you do the mathematics behind more complicated problems.

Labs, projects and take home exams (or proctored exams, where you can sit and take the exam until you're done rather than when the next group of people needs the room) seem like they're a more accurate representation of what you know and can apply. If you don't understand the theory, you won't be able to explain why x, y, or z happened this way...etc, etc.
myng_rabbyt
May. 5th, 2006 01:17 pm (UTC)
See, I agree with you that exams aren't the end-all, be-all for testing one's knowledge and ability. My mother, who is a former high school teacher turned high school counselor. She has a degree in counseling and therapy, which includes training in psychometry (the science of tests and testing).

In the school system where she works, they have increased the number of (state-mandated) standardized tests that students must pass before being eligible for graduation. Additionally, my mother-in-law, who teaches at the elementary school level, also has noted an increase in the number of tests students at her level (first grade) have to take.

I know that we are primarily discussing tests in the context of specific classes, but my point is: more educational institutions are putting their faith into tests, and I don't think that's such a good idea. The students spend as much time going over the material they need to know for the test as they do trying to learn the information they're supposed to be learning at that level.

As brilliant as a student may be, once you put them under the gun to cough up information, you already have them at a disadvantage, which decreases their ability to accurately and thoughtfully respond to a problem or scenario.
banazir
May. 4th, 2006 11:23 pm (UTC)
Several reasons...
I'm glad baranoouji and chaosinaskirt put in their $0.04 while I was giving my final Database Systems lecture of the semester. They both raise good points.

For the record, I still agree that the time limit imposed by exams is very artificial, and it's really not a good measure of student's expected performance in a job setting. However, insofar as:

1. exams can be made open-book, open-notes, or take-home;
2. the time pressure of an exam does admittedly help us to assess students' ability to think clearly, work problems efficiently and without a lot of backtracking;
3. students do sometimes cheat on homework because it's worth less (making the cost-effectiveness of running plagiarism checks too low for some instructors to bother with), the threshold of visiblility is lower, and the ethical compunction is lessened;

it is probably advisable to have exams be worth about half the grade.

I usually have a score breakdown as follows:

In courses with one midterm and one final: 25% homework, 5% class participation, 5% miscellany (paper reviews or peer review), 15% midterm, 25% final, 25% project

In courses with two hour exams and one final: 30% homework, 5% class participation, 15% project or paper, 10% per hour exam, 30% final

In courses with one midterm and no final: 10% homework, 5% class participation, 25% midterm, 60% project

--
Banazir
myng_rabbyt
May. 5th, 2006 01:35 pm (UTC)
Re: Several reasons...
I agree with you on the pros of exams for testing students and for establishing an effective means of grading them. But I see that you also include papers and projects, and I have a lot more faith in papers and projects than I do in exams. Papers and projects (p&P henceforth) encourage the use of skills that are -not- area specific--skills like problem solving, source evaluation, and so forth--that aren't usually actively taught at the college level (unfortunately).

P&P also provide students with the luxury of self-guide work and synthesis (whether they view it that way or not). It also allows them the opportunity to bring unrelated skills into the fray. If you require a presentation in a class, then a student can showcase their artistic creativity, or their skill with media production, to produce something relevant and interesting. See what I mean?

The only problem with making a test a large portion of a grade for a class is that it can thwart the efforts of the best student. Say one of your best students comes in on test day, sick as a dog. They've studied, and you trust (and just know in your heart) that they've prepared. But despite all that, they don't do well on the test, due to illness. You know they know the material; they just weren't in the right shape to do their best. Would you let their grade stand, or would you allow them to retake the test?
baranoouji
May. 4th, 2006 07:38 pm (UTC)
My two cents (with some sense I hope)
Beloved and Enlightened Banazir --

One of the issues that I have with my current CS class is the fact that homework comes strictly in the form of practical projects (Make program X that results in Y...) but the exams come in the form of theory-oriented questions (usually dealing with a few bits of code correction at the end. Our quizzes are "fill in the vocabulary" and don't really help us to prepare for the theory aspect of the exams.

I've personally remedied this (or at least tried to) by wading through the texts prior to the exam. However, I still feel during exams that I am forgetting unconscionable amounts of theory every time just because my mind is not accustomed to expressing said information.

In conclusion, homework that has nothing to do with tests are annoying, because the tests win in the end.

Perhaps this will cheer you during the dreaded finals.

-E.

PS. I'm going to California for the Summit! Hooray!
dankamongmen
May. 4th, 2006 11:23 pm (UTC)
It shows a lack of understanding, and is a poor programming habit, to let the compiler do your work for you. The compiler is not a debugger, nor is it a style checker in the way of lint. Type error messages are not "feedback"; they are how the compiler breaks.

!!!!!!!!! !!!!!!!!!!! !!!!!!!

it becomes a mantra now. so let it be written, so let it be done.
banazir
May. 5th, 2006 12:45 am (UTC)
Quotation by Uday Reddy
You can attribute that one to Uday S. Reddy, presently head of the department at Birmingham.

--
Banazir
bojojoti
May. 5th, 2006 04:45 am (UTC)
*raises hand* Is the 5% you award class participation based on attendance, involvement in discussions, or both?

I'm the mother of two very different styles of student. My son hates busy work and was able to get admirable grades by high test performance during his high school years. (As a college student, he has found it necessary to do homework if he wants to receive an 'A.') My daughter, on the other hand, does her homework scrupulously, but she dreads taking tests. She feels she does not test well, but she has been able to maintain a 3.81 even with very demanding classes this year.

Individuals learn differently and test differently; therefore, it would be nearly impossible to accurately discern the specific amount of knowledge each student has acquired. Even if one could, it doesn't necessarily mean the knowledge could be applied. I suppose life tests that adequately.
( 9 comments — Leave a comment )

Latest Month

December 2008
S M T W T F S
 123456
78910111213
14151617181920
21222324252627
28293031   

KSU Genetic and Evolutionary Computation (GEC) Lab

Teunciness

Breakfast

Science, Technology, Engineering, Math (STEM) Communities

Fresh Pages

Tags

Powered by LiveJournal.com
Designed by Naoto Kishi