Theory choice, the problem of accepting/rejecting scientific theories, is philosophically interesting in part because it involves appeal to nonempirical factors that can only be justified by philosophical considerations. The emphasis in this paper is on the historical as opposed to the experimental sciences—including astronomy, evolutionary biology, and especially historical geology—with examples taken from seventeenth through nineteenth centuries. The fact that evidential reasoning inherently requires a choice of philosophical/methodological principles is demonstrated through reference both to historical cases and to general philosophical considerations. This paper argues that methodological principles play a crucial role in turning empirical data into evidence for/against theories, and it outlines some of the particular evidential and methodological difficulties faced in the historical sciences. Choices of methodological principles depend on nonempirical factors, and because definitive arguments can rarely be found, they are largely a matter of judgment. “Scientific” debates are thus sometimes really disputes over philosophical taste and judgment. Moreover, it is often the case that clear judgments about the incorrectness/correctness of a methodological principle used in a specific context can only be made retrospectively. In part by looking at connections among Isaac Newton, David Hume, and Charles Lyell, and in part by examining Lyell’s own arguments, I argue that it was reasonable for Lyell to adopt uniformitarianism as a central methodological principle. Through arguments and historical examples, I also show that there are limits to the acceptability of the uniformitarian position.