Is this really the best system, though? I've heard recently that many top universities have moved completely away from comps and quals, to a "progress review" system. We had heard of this practice, first used in CS graduate education by Carnegie Mellon University's School of Computer Science, where it was called "Black Friday". I set out to examine this alternative system and consider its pros and cons.
Pros of the Black Friday system
Actually talking with students to see how far they've come certainly puts accountability squarely back on their shoulders - no more coasting after "finishing" a set of exams that, let's face it, vary greatly in difficulty for students with disparate technical backgrounds. However, it also puts accountability on the shoulders of faculty members, because with a semesterly review, the faculty have to recalibrate and resynchronize their concept of what acceptable progress is.
The hope, of course, is that they will also recalibrate their definitions of "good" and "excellent", the better to make good calls in admissions. Moreover, they'll have to actually look at progress: not just a static, good day/bad day dipstick of someone's level of knowledge, but real performance, as measured in a more real-world environment. (Even Ph.D.s sometimes get sent out of the ivory tower in to the cold, harsh outside!)
I'm generally all for paying genuine attention in evaluations, whether they are evaluations of applications for scholarships and admission, annual evaluations by a supervisor, or milestone evaluations for postgraduate study. In my opinion, if we expect graduates to go into the workforce, or into tenure-track positions for that matter, the more we should strive to create an environment that is true to the expectations and demands of those future careers. The more something adheres to those expectations, the truer it is, and the more credible the feedback given to the grad student.
Peter Lee on Black Friday at CMU
CMU professor Peter Lee writes in his blog:
It’s Black Friday.
No, this isn’t the shopping frenzy after Thanksgiving day. This is the end-of-semester evaluation of the Computer Science Ph.D. students. All of the faculty involved in advising Ph.D. students in our program gather together for a full day meeting, to discuss in detail the progress of each student — about 160 of them, more or less. As a group, we write a letter to each student, giving our evaluation of their progress through the program. In parallel, the students have a big all-day party. Perhaps out of guilt or a need to keep the faculty happy, the students provide the faculty with a huge bucket of beer. (This is not a joke! ;-)
Black Friday is truly one of the great traditions of the Computer Science Department. It has been copied by all of the other academic units in the School of Computer Science and by departments at other universities. The reason for the name “Black Friday”, besides the fact that it is held on the Friday of finals week, is that some students, um, get kicked out of the program. While an outsider might think that this would be a somber day, in fact it turns out to be a day of celebration. For me, it is the one day each semester where I get to see each student’s face (a photo is projected on the screen) and hear the relevant faculty brag about each student’s accomplishments.
Jeanette Wing on Black Friday at CMU
Peter adds: "Perhaps the best explanation of why Black Friday is great was given by Jeannette Wing. Printed with her permission, here is her essay..."
The Importance of Black Friday
by Jeannette M. Wing
- I. Black Friday is for the students.
It is our only means of evaluating them. It is their only official feedback from the department on their progress.
Students do not have oral exams, written exams, or qualifying exams. They get only Black Friday letters. Their standing in the program is determined solely by our Black Friday meetings.
Writing Black Friday letters is also a way for us to personalize the feedback we give each student. This individual attention our students receive is what makes Carnegie Mellon Computer Science special. Current and past students surveyed said that Black Friday was the #1 feature of our Ph.D. program that should not change.
- II. Black Friday is for the faculty too.
It is a way for us to calibrate our standards across areas.
It is a way for us to ensure consistency in evaluating our students.
It is a way for us to share our values.
Black Friday is a good time to share advice with each other on how to advise students, especially those who are free-spirited or those who are in trouble.
Black Friday is a good occasion to learn about our students, not just what research they are doing but also their outside interests and personality traits. It is a good occasion to learn about each other’s research, through what our students do. It is a good occasion to meet each other, some of whom we would never see otherwise.
It is a fantastic opportunity to learn about departmental culture. The inside scoop. How your fellow colleagues think. Who are the nice guys. Who are the curmudgeons.
The time that would go into creating, conducting, and administering qualifying exams goes instead to attending Black Friday meetings.
It is the only official meeting that faculty are required to attend.
– Jeannette Wing
Cons of the Black Friday system
You could argue that in a department that doesn't already have a highly open culture, it's a little risky to entrust the fate of grad students to the tough colleagues who would come in with meat axes. I've heard this same argument regarding the hiring of deans, and frankly, I think there's a double standard here. We entrust our fate to administrators all the time, yet there's generally an atmosphere of anxious uncertainty, if not outright fear, when inevitable change comes.
What are we afraid of? Rising expectations? My opinion is that if we are willing to raise expectations for students, we shouldn't be afraid of them ourselves, and we should be willing. I'll risk advocating meritocracy for a moment by saying that if one is worried that the meanie next door will flunk out your protegee, one should train her as well as possible so that the meanie's fist thuds against nothing but solid steel. (Don't people watch kung fu movies any more?)
A more valid concern, I think, is that a department's culture can come to reflect impatience with certain kinds of science, some of which take longer and are riskier. Once again, we see this problem reflected in the "tenure clock": publish or perish, but do it within six years. Yes, well, I wrote six proposals a year for my first six years, and I got one funded per year (plus I came in with one). Now I write two proposals a year, still get one funded, and have time to do higher-impact work with the four proposals' worth of time and effort! Plus, my students are happier because I concentrate my effort on the multi-year proposals and renewals instead of taking a scattershot approach with a couple of short-term small grants, a couple of mid-range medium grants, and a couple of long-shot large grants. Has my research gotten less risky, less innovative? Not really. You could say that I can afford to engage in research activity with longer payoff times now. I think that's a mistake, though, or rather, it's a mistake to think it's right to treat the time to be slow and methodical as an earned privilege rather than intrinsically the right thing to do. In topic areas such as theoretical computer science, and perhaps across all of CS, it just is, and we'd do a disservice to grad students to turn their world into a pure "up or out" scenario.
Conclusion: My view is that Black Friday can be a good thing if we have basically turned exams into formalities. It can also ease the strain to something more equitable if we've made things too competitive (which was the rationale for moving away from pure comps in the first place). I don't think standards are intrinsically "too challenging" or "not challenging enough"; rather, I feel that we need to let students "rise to the level of expectations", in the words of Jaime Escalante. We also need to feel some personal accountability for showing them how to achieve this objective. As long as they think they are just jumping through hoops to satisfy us "as a faculty", there will continue to be people who just fall through the cracks. That includes passing without really getting to the next level! In the end, the tasks and milestones we set should challenge students to get better, rather than strike them immobile with the fear of failure. It should be not "up or out", but excelsior.