Telescope Allocation Committee Assignment
There will be two separate TACs; each TAC will discuss
proposals written by the Primary Investigators (PIs) in the other
TAC -- you will not discuss proposals written by your fellow TAC
members.
The TAC assignments are as follows (the asterisk denotes the TAC
Chairperson):
TAC 1
|
TAC 2
|
Gemma*
|
Keri*
|
Tom
|
Yunchuan
|
Robby
|
Alex |
Jason
|
Zion
|
Caleb
|
Sebastien
|
|
Carter
|
Each person will be assigned as a primary reviewer on two proposals,
and a secondary reviewer on a third. Your written assignment is as
follows:
- For each proposal you are primary reviewer on, write a
detailed, one page (single-spaced) review of the proposal. 2/3
of this review should focus on the scientific and technical
aspects, 1/3 should focus on writing and presentation.
- For each proposal you are secondary reviewer on, write
a one paragraph (substantial and single-spaced) summary of your
evaluation. This should be be a scientific and technical review;
do not comment on presentation/writing issues if they are
particularly problematic.
- For all proposals, give a numerical rating that ranges
from 5 (excellent) to 1 (poor). INTEGER VALUES ONLY, PLEASE.
- These written reviews and numerical ratings are due (via
Canvas) on Apr 6th
CONFIDENTIALITY RULES:
Do not reveal to others which
proposals you are primary or secondary reviewer on.
Do not talk with others about the
proposals you are reviewing.
Your reviews will be shared with the
proposal authors after the TAC process is complete, but your name
will be redacted from your review.
TAC Meeting
On Thu Apr 14th, we will have the TAC discussion in
class. Be sure to bring your reviews to class for use in the TAC
(hardcopy or electronic on your laptop is fine).
The TAC process works like this (for each TAC seperately):
- Triage stage: Based on the numerical scores, the
proposal with the lowest average numerical scores will be
immediately eliminated from discussion, unless a committee
member wants to rescue it.
- Discussion stage: Each of the four remaining proposals
gets a ten minute discussion led by the primary reviewers, but
involving all members of the TAC. After this discussion, each
TAC member will privately grade the proposal on a scale from 5
(excellent) to 1 (poor). You are allowed to change your score
from your initial written review if you feel that is appropriate
based on the discussion. Note that it is very important to keep
to the timed schedule, and it's the TAC Chair's task to make
sure that their TAC stays on schedule.
- Final scoring: At the end of the Discussion stage, each
person will turn in to me their final ratings for all proposals
that their TAC reviewed. At that point, I will average all the
ratings for each proposal, and that average will be the
proposal's final score. If there are any ties at that point, I
will break the tie with my own evaluation. The top three
proposals will be awarded prizes.
General Guidelines for your reviews (particularly for the
primary reviews):
- Primary reviewers should double check observing times (or
review previous studies, if cited to support chosen exposure
times). You want to make sure the observations are possible!
- Check a few of the cited sources if they are used to make
crucial arguments; make sure the cited source backs up the
claims in the proposal.
- Make sure you explain what parts of the proposal you found
convincing and well-argued, and which parts were unclear or
needed more work. Your written review should match your
numerical rating: Don't rate a proposal a 4 and then point out
nothing but problems, and don't rate a proposal a 1 and then
feel bad for the proposer and simply say nice things.
- Give suggestions for what might improve it -- these
suggestions can range from strategic/conceptual ("I think your
sample of galaxies could be better defined by...") to technical
("I would have found the proposal more compelling if you had
included an estimate of exposure time") to presentation ("Figure
4 was really confusing; it would have been easier to understand
if you had presented <something else> instead...").
- Commenting about typos etc is fine if there are too many of
them, but don't give a laundry-list of typo corrections. Just
say "Proposal needs proofreading" and move on. If your review
consists of nothing but lists of typos, you haven't done your
job and it's not particularly useful to the proposer.
- Think about the review criteria
we talked about: technical feasibility, compelling science,
proper academic presentation, etc.
- Make the review substantive and carefully written -- a shallow
or poorly written review is as bad as a poor proposal. Remember,
the grade I give you for the TAC assignment will be based on the
quality of your reviews.
- Make sure your ratings have discrimination power -- i.e., the
numerical rankings should show a spread in quality. This doesn't
mean that you have to balance every good score with a bad score,
but if (for example) you give a 4 to all the proposals you
review, that is not particularly helpful to the process.
- Your reviews must be professional in tone.
Submission Format
Please submit your reviews via Canvas in the form of a single
pdf file with the following pages:
- Page 1: your numerical scores for each of the five proposals
your TAC is reviewing
- Page 2: primary review #1
- Page 3: primary review #2
- Page 4: secondary review
TAC Assignment Grading:
For course grade purposes, your written reviews will be
graded based on the quality of your review and how well you follow
these guidelines. However, your evaluation of the other proposals
will not affect the grade I gave to those proposals themselves.