One of the first National Institutes of Health (NIH) Roadmap initiatives to be launched was the Director's Pioneer Award. This award was established to "identify and fund investigators of exceptionally creative abilities and diligence, for a sufficient term (five years) to allow them to develop and test far-ranging ideas." Nine excellent scientists were chosen as NIH Pioneers, but the selection of all men is at odds with the percentage of women receiving doctoral degrees for the past three decades, serving as principal investigators on NIH research grants, and achieving recognition as scientific innovators in non-NIH award competitions. The absence of women Pioneers provokes the following question: In the context of extant research on the impact of gender-based assumptions on evaluation of men and women in traditionally male fields, such as science, were there aspects about the process of nomination, evaluation, and selection that inadvertently favored men? We present evidence to suggest that women scientists would be disadvantaged by the following components of the NIH Director's Pioneer Award initiative: (1) time pressure placed on evaluators, (2) absence of face-to-face discussion about applicants, (3) ambiguity of performance criteria, given the novelty of the award, combined with an emphasis on subjective assessment of leadership, potential achievements rather than actual accomplishments, and risk taking, (4) emphasis on self-promotion, (5) weight given to letters of recommendation, and (6) the need for finalists to make a formal, in-person presentation in which the individual and not his or her science was the focus of evaluation. We offer an analysis of this process to encourage the NIH to embark on self-study and to educate all reviewers regarding an evidence-based approach to gender and evaluation.