The Insidious Fixed Count Sub-sample…

I recently presented a research paper on three common errors associated with using bioassessment methods for non-bioassessment purposes at the Annual Meeting of the Western Division American Fisheries Society. The presentation focused on errors relating to development and design of macroinvertebrate bioassessment models, as well as some issues on the myths of variance homogeneity and “taxonomic representativeness.”

Today, however, I want to address the unintentional effect of the fixed count sub-sampling protocols on sub-sampling effort. Back in the early days of bioassessment… back when everyone was sufficiently embarrassed by the assumptions and limitations of the bioassessment study design they knew was intended to be a screening tool… (see Plafkin et al. 1989 p.1-2) there was a short debate on the most appropriate method to subsample benthic samples. This debate, was best represented by 2 synthesis papers presented in the Journal of the North American Benthological Society in 1993 (or was it 1996?; write me for exact citations if you want). The Fixed count method “won” this debate, but not for the reasons one might think.

On the one hand, the argument was made that by using a fixed-count methods, that samples from different locations may not constitute a uniform field effort (Countermarch).  The counter point was made that by standardizing a fixed count, that the effort was standardized in a different (i.e. non-spatial) dimension (Hawkins)… That is, rather than standardizing the sample in space (as with a standard unit area) the samples were standardized in the dimension of the species richness accumulation curve. This might sound kind of strange to people that are not used to thinking of data in terms of a many-dimensional (multivariate) species-space… but honestly there are many tools to help us visualize data this way and it IS a legitimate point.  The problem is, that this point was brought up as to rationalize the methods commonly employed by the initial bioassessment studies, and was then its limitations were quickly forgotten and bioassessment protocols were recommended universally with the assumption that they are suitable for many situations that are inappropriate.

Currently bioassessment protocols–including EMAP, RBP, and RIVPACS designs–are recommended for hydropower relicensing, impact assessment and basic ecological assessments of change… and most often these methods provide zero power for detecting effects.

here is one example of a non-statistical problem. In many types of assessments we are interested in changes in the density of certain types or groups of aquatic macroinvertebrates. Though the methods are standardized, they are not spatially standardized. So what? Who cares? Right?

When this is forgotten people make poor conclusions. The graph above shows how the fixed count method affects the spatial uniformity of samples collected using a fixed count of organisms.  note that for some samples the 0.1 square meter of area was used to get the target number. However, when small Nais worms were abundant, some samples used less than 0.01 or o.oo1 square meters.  The two orders of magnitude range in variation of sample size does not affect all species uniformly. Larger organisms are less likely to be collected in tiny sample areas. Consider, the effect of an entire river, reach, or riffle being represented by a sample corresponding to 1 square inch of sample area.

Comments are closed.