Telescope Allocation Committee Assignment
Here is a
(bookmarked!) PDF file containing all the proposals.
You will be assigned as a lead reviewer on two proposals, and a
secondary reviewer on another four (these assignments will be
emailed out individually). Your written assignment is as follows:
each proposal you are lead reviewer on, write a detailed, one page
review of the proposal. 2/3 of this review should focus on the
scientific and technical aspects, 1/3 should focus on writing and
- For each proposal you are secondary reviewer
on, write a short one paragraph summary of your evaluation. This should
be largely a scientific and technical review; only comment on
presentation issues if they are egregious.
- For all proposals you are primary or secondary reviewer
for, give a numerical rating that ranges from 5 (excellent) to
1 (poor). INTEGER VALUES ONLY, PLEASE.
- These written reviews are due at the beginning of class
on Dec 3rd, and cannot be turned in late!
Do not reveal to others which proposals you are primary
You cannot talk about the proposals with the author while
you are reviewing (this is redundant with the rule above, but is
worth spelling out explicitly!)
Your review will be shared with the author after the TAC
process is complete, but your name will be redacted from your
On Thu Dec 3rd, we will have the TAC discussion in class. There
will be two separate TACs, assigned as follows (the asterisk
denotes the TAC Chairperson):
On this day, bring TWO COPIES of your reviews: one to
turn in to me, one to have for your own use in the discussion.
Each TAC will discuss proposals written by the PIs in the
other TACs -- you will not discuss proposals written by
your fellow TAC members. For each proposal, there will be a ten
minute discussion led by the primary reviewer, but involving all
members of the TAC. After this discussion, each TAC member will
privately grade the proposal on a scale from 5 (excellent) to 1
(poor). You are allowed to change your score from your initial
written review if you feel that is appropriate based on the
discussion. Note that it is very important to keep to the timed
schedule -- it's the TAC Chair's task to make sure that their TAC
stays on schedule.
At the end of the TAC meetings, each person will turn in their
final ratings to me, the Chair of the Super TAC. At that point, I
will average all the ratings for each proposal, and that average
will be the proposal's final score. If there are any ties at that
point, I will break the tie with my own evaluation. The top three
proposals will be awarded prizes.
General Guidelines for your reviews (particularly for the
- Make sure you explain what worked and what didn't. Your
written review should match your numerical rating: Don't rate
a proposal a 4 and then point out nothing but problems, and
don't rate a proposal a 1 and then feel bad for the proposer
and simply say nice things.
- Give suggestions for what might improve it -- these
suggestions can range from strategic/conceptual ("I think your
sample of galaxies could be better defined by...") to
technical ("I would have found the proposal more compelling if
you had included an estimate of exposure time") to
presentation ("Figure 4 was really confusing; it would have
been easier to understand if you had presented <something
about typos etc is fine if there are too many of them, but don't give a
laundry-list of typo corrections. Just say "Proposal needs
proofreading" and move on. If your review consists of nothing but lists
of typos, you haven't done your job and it's not particularly useful to
- Think about the review criteria we talked about: technical
feasibility, compelling science, proper academic presentation,
- Make the review substantive and carefully written -- a
shallow or poorly written review is as bad as a poor proposal.
Remember, I will be evaluating both the quality of your
proposal and the quality of your reviews.
- Make sure your ratings have discrimination power -- i.e.,
the numerical rankings should show a spread in quality. This
doesn't mean that you have to give *someone* a 1 and somebody
else a 5, but it does mean that you shouldn't simply rate all
the reviews a 4, for example.
For course grade purposes, your written reviews will be graded
based on the quality of your review and how well you follow those
guidelines. However, your evaluation of the other proposals will
not impact what grade I give to those proposals themselves.