This author reviews the major issues surrounding field-based (as opposed to strictly laboratory-based) research, focusing on issues specific or of greatest importance to soil science. Soil science’s history could perhaps be described as a fusion of physical geography and geology with agronomy, and many published studies in the soil science journals show these roots. Following the lead of previous authors, who have included ecologists, statisticians, and philosophers and historians of science, this author divides field research into 2 major categories, broadly manipulative studies and mensurative studies. Manipulative studies are, under some definitions including one tentatively employed in this paper, the only type of study that qualify for the name “experiment”, and involve complete control over experimental conditions by the researcher. Treatments in an experiment are directly related to replication, and can be applied with great precision. Mensurative studies are those that at least partly use features of the environment beyond the control of the researcher to test hypotheses or discover new information. The key feature of a mensurative study is that the features of interest are clearly defined but not controlled (i.e. not randomized) by the person conducting the study.
Replication, and avoiding pseudoreplication, is of great importance in all types of studies. However, the replication built into a manipulative experiment in the form of repeated application of treatments is distinct from the replication of a mensurative study using repeated features of the environment. That these are different types of replication is stated in this paper, but I found no more detail or explanation than that. Pseudoreplication in this paper is discussed little in the context of independence of samples; rather the discussed risk is of attempting to draw inferences beyond the inference space of the study. This is a problem in both major types of study, and can be avoided by carefully determining and describing the inference space, and expanding that space by greater replication; too-small sample sizes are quite simply labeled as unpublishable in this paper, a sentiment I can agree with.
Determining the required sample size is a major issue for all types of studies. In this author’s presentation, this is an early step in the design of the study, after the biological and statistical questions have been established but before data collection begins. There is some discussion here as well of statistical power (the chance of avoiding a Type II error, that is of failing to reject a false null hypothesis) and recommendations of flexibility regarding especially alpha values (the chance of making a Type I error, that is of rejecting a null hypothesis that is not false). For a number of reasons, some of which are practical and logistical, alpha values larger than the ubiquitous 0.05 are encouraged, because in many cases the consequences of the 2 types of error are not even, and one may wish to concentrate on reducing the probability of a Type II error.
This paper describes 10 commonly-encountered study designs in soil science and related disciplines, and then discusses study-design concerns common to all such as replication and the need to clearly define study units, samples, populations, and other important aspects. Finally, this author presents the conclusions from all of these examples and considerations in the form of a short list of key recommendations. Quoting directly:
1. A clear definition of the research question is the initial (and most critical) step. This definition dictates the type of research design that is appropriate and the specific design issues associated with different research types.
2. The appropriateness of a given research design can be judged only after a thorough review of what is known about the research question. Exploratory pattern studies can be very informative at an early stage of research, but yield little new information for well-established research topics. Equally, the imposition of a set of treatments if little is known of the processes controlling responses is unlikely to produce comprehensive interpretations.
3. There is never a good reason for haphazard sampling – the rationale for selecting sampling points in pedological, soil geomorphic, or inventory studies should be clearly stated.
4. A clear definition of the population and the elements that comprise the population under study is very important.
5. The definition of the population dictates the extent of the study and the physical or temporal space that the results pertain to, which is critical to avoid pseudoreplication.
6. The sample support, spacing, and extent of the study must be consistent with what is known of the processes controlling the phenomena being studied.
7. The construction of hypotheses for formal testing should be based on sound physical or biological reasoning, and sufficient samples should be taken to allow reliable testing of the alternative hypotheses.
8. The exclusion of phenomena because they cannot be replicated is inherently limiting to the expansion of our knowledge of soils. Innovative approaches must continue to be developed and applied so that we can expand the scale at which field studies can be undertaken.
No comments:
Post a Comment