EU-wide monitoring methods and systems of surveillance for species and habitats of Community interest
  A research project funded by the European Union 
  The EuMon integrated Biodiversity Monitoring & Assessment Tool
 BioMAT > Background info module 2 > Meta-analysis
Meta-analysis

Meta-analyses are quantitative methods that combine estimates instead of raw data. Hence, they can statistically draw more general and quantitative conclusions compared to single studies or reviews and they provide new insights and research directions: They are strongly recommended for the integration of monitoring schemes, as they can deal with heterogeneous data from different monitoring schemes to produce trends at supra-national scale, both considering the effects of time or of any causes of changes.

Principle

The main idea behind meta-analysis is that outcomes of different independent studies are treated as input units for the analysis of a general pattern. In other words, just as individual studies summarise data collected from many points to answer a specific research question (i.e. each point is a separate data-point in the analysis), a meta-analysis summarises data from individual studies that address a specific research question (i.e., each study is a separate data-point in the analysis). Such an approach combines information from contrasting monitoring schemes differing in their sampling designs, monitored objects, data characteristics, and to some extent even statistical methods.

Effect size (ES)

In meta-analyses, the outcomes of the different studies (i.e. estimates of effects across analyses) are expressed in terms of a common currency called "effect size". The effect size is a standardized estimate of the magnitude of the effect of the explanatory variable. Any standardised index can be an "effect size" (e.g., standardised mean difference, odds-ratio, correlation coefficient, risk ratio, proportion, standardised gain score) as long as it is comparable across studies (generally accomplished through standardisation), is independent of sample size and represents the magnitude and direction of the relationship of interest. Note that when only qualitative information is available for the tested effect or cause of change, non-parametric tests can be used.

Weights

The effect sizes derived from individual studies are weighted to yield a common estimate of the magnitude of the effect, bounded by confidence intervals. A meta-analysis puts more weight on studies with more precise estimates.

Analysis

A meta-analysis has a good probability of detecting the effect of the cause of change across all observations, which is not the case for separate tests on each single dataset. In a meta-analysis, the homogeneity of effect sizes among studies is examined to determine whether all studies share a common effect size. The statistical power of the resulting meta-analysis will depend on the magnitude and precision of the effects in the combined monitoring schemes, but power should be reasonable in the case of small to moderate effects in all monitoring schemes. With heterogeneity in effect sizes, moderating variables (e.g. region, species…) that are responsible for this variability can be identified by coding each variable in the database and analysing either mean differences (for categorical variables) or weighted regression (for continuous variables). Meta-analyses can thus estimate average trends across monitoring schemes, but also discriminate sets of regions with contrasting trends (with tests of homogeneity of effect size).

Several free software programs are available to conduct meta-analysis, see for example the list at: http://www.psychwiki.com/wiki/Meta-analysis. Bayesian frameworks are particularly suitable for conducting meta-analyses, because of the close connection between hierarchical modelling and meta-analysis where lower-level components (studies) are combined to make higher-level (regional) inferences. Bayesian meta-analyses present several advantages over conventional weighted linear regression of classical meta-analyses.

Biases and sensitivities

Although meta-analyses are promising tools to evaluate trends and status of biodiversity, they have potential biases and sensitivities to data availability:

Biases
Biases can arise due to (i) heterogeneity in methods and robustness of pooled studies, (ii) non-representativeness of studies, in particular because of a higher likelihood of publishing significant than non-significant results, (iii) by non-independence of data, both within and between studies, which is typically the case in monitoring due to repeated measurements at temporal and spatial scales. For example, non-independence caused by the inclusion of multiple sites from a single study can generate significant bias in overall effect size. This could occur, for example, if all or most sites within a study are geographically close or ecologically similar to each other, and hence are likely to yield effect sizes of similar magnitude. Alternatively, monitoring within a study likely uses similar methods at all sites, so that effect sizes are more likely to be similar within a study than among studies using different methods. Note that non-independence among studies can also arise if studies from the same laboratory, for example, tend to be based on the same methods, which may generate more similar effect sizes than those obtained from different research groups. Various methods and software packages are available to assess biases (see details in Deliverable 3.5.1). Note that Adams (2008) presented a new general statistical template for meta-analyses that can account for non-independence among studies by directly incorporating the correlation structure among studies.

Sensitivity to sample size and to geographical and temporal spread of data
The accuracy of meta-analyses in generating estimates of rates of change for any biodiversity element depends on the availability of data. Data availability may be constrained in a number of ways, including the existence of few surveys across the range of the focal element, or adequate sampling in only some, but not all, parts of this range. All meta-analyses are sensitive to sample size (larger sample sizes improve the accuracy of estimates) and to the uneven geographical distribution of sites.

Key references

  • Adams D.C. 2008. Phylogenetic meta-analysis. Evolution 62: 567-572.
  • Henry P., Lengyel S., Nowicki P., Julliard R., Clobert J., Celik T., Gruber B., Schmeller D.S., Babij V., Henle K. 2008. Integrating ongoing biodiversity monitoring: potential benefits and methods. Biodiversity and Conservation 17: 3357-3382.
  • Lengyel, S., A. Kobler, L. Kutnar, E. Framstad, P.-Y. Henry, V. Babij, B. Gruber, D. Schmeller & K. Henle (2008): A review and a framework for the integration of biodiversity monitoring at the habitat level. Biodivers. Conserv. 17: 3341-3356.
  • Liermann M., Hilborn R. 1997. Depensation in fish stocks: a hierarchic Bayesian metaanalysis. Candian Journal of Fisheries and Aquatic Sciences 54: 1976-1984.
  • Osenberg C.W., Sarnelle O., Cooper S.D., Holt R.D. 1999. Resolving ecological questions through metaanalysis: goals, metrics, and models. Ecology 80: 1105-1117

EuMon core team; August 2014


Module 2 background information /b>
Other BioMAT background information
Overview BioMAT background information
 Contract number: 006463