EVIDENCE FROM STUDIES IN OCCUPATIONAL SETTINGS Our knowledge of the effects of occupational hazards can be enhanced by studies conducted in occupational settings. Consider the example of formaldehyde, an agent with multiple adverse health endpoints that has undergone extensive study.

Through large cohort studies in occupational settings, including embalmers and users of formaldehyde resins or molding compounds, excesses of lymphohematopoietic malignancies associated with peak exposures (greater than 2 ppm) have been observed, as explained in a paper that appeared in the May 2009 issue of the Journal of the National Cancer Institute. From a toxicology perspective, it is not clear how formaldehyde exposure would be associated with excess lymphohematopoietic malignancies because the body naturally produces significant quantities of formaldehyde. Research findings point to potentially important differences between formaldehyde in occupational settings, where workers are typically exposed to solutions of 30–50 percent formaldehyde in water and stabilized with alcohol, compared with the formaldehyde that the body naturally produces. Could the alcohol, or the hemiacetal that forms through reaction of the formaldehyde and alcohol, contribute to the growth of tumors? Would studies on paraformaldehyde, a solid source of formaldehyde that does not contain the stabilizers, result in similar adverse health effects?

It is now well recognized that lymphohematopoietic malignancies have been observed in workers exposed to formaldehyde. While uncertainty remains regarding the mechanism that leads to these malignancies, this example illustrates the benefits of evaluating determinants of injury and health in the workplace environment.

IDENTIFYING SUPPORTING EVIDENCE: RELEVANCE Appraisal of research papers begins with identification of the hypothesis. Often found at the end of the introduction, the hypothesis may be presented as a series of questions posed by the investigators.

When searching for supporting evidence for your evaluation, one of the first steps is to consider whether the question addressed by the study matches the question you wish to answer. This seemingly straightforward assessment sometimes requires careful consideration. Questions addressed by published studies tend to have several components. One component is often a factor that might affect a health outcome, such as an exposure to a hazardous agent or an intervention. Other components typically include the population under study, the outcome being evaluated, and, when applicable, the reference or control group strategy.

There could be important differences between your target population and that which the researchers set out to study, such as differences in socioeconomic status or industry sector. A strategy for preventing workplace musculoskeletal disorders may be effective in one population but not in another.

Ensuring that the question addressed by the study matches yours prevents you from applying the information out of context. Inappropriate application of study findings can lead to decisions that are not supported by scientific research. Consider, for example, a study that evaluates the effects of a low-dose exposure to a hazardous agent. Concluding that the hazardous agent does not lead to adverse health outcomes may be erroneous when extrapolating from the low doses discussed in the study to the peak exposures observed in your workplace. Conversely, extrapolating findings from highly exposed populations may lead to undue concern when exposure levels encountered in your workplace are much lower.

A timeframe is often implicit in the study question, but it may be helpful to explicitly identify the timeframe when dissecting the question. Consider whether the timeframe after which the outcome measures (for example, the adverse health effects) were observed aligns with your question to determine whether the science appropriately supports your decision-making process.

IDENTIFYING SUPPORTING EVIDENCE: STUDY DESIGN Reviews can be thought of as secondary research on individual studies. Critical appraisal of secondary research allows you to differentiate between a well-done systematic review and a subjective summary paper. A key element of systematic reviews is an assessment of heterogeneity between study results. Statistical tests for heterogeneity, although often limited in power, provide a means of evaluating whether the results from different studies are likely to reflect a single underlying effect. Where differences between study results are observed, readers should consider whether the variations are cause for concern. This is particularly important when assessing reviews that combine results of observational studies, which may be more prone to bias and confounding than experimental studies.

A systematic review can be evaluated for potential publication bias, a distortion of the scientific record caused by the greater likelihood that studies with statistically significant findings will be published. Effects of publication bias may also be observed if multiple publications of a single study are included in the same review.

Primary research may be either experimental, where researchers are actively involved (as in experimental trials), or observational, where researchers are passively involved (as in cohort and case-control studies). For ethical reasons, studies that evaluate the effects of exposures to potentially harmful agents are generally observational studies. The basic study design can guide confidence in study conclusions. Some study designs are more prone to bias and confounding than others; see Figure 2 for a hierarchy of study designs adapted from the Oxford Centre for Evidence-Based Medicine. Where bias or confounding is suspected, our overall confidence in the reported results decreases, though the study may still be useful as part of our evolving understanding of the science.

An evaluation of the methodological details within a scientific study can increase or decrease confidence in the quality of the data, analysis, and conclusions. Considerations should include a critical evaluation of participant selection, the setup and management of study groups, and the method of ascertaining the outcome measure (for example, disease or case status).

Participant selection. Consider whether the subjects in the study are representative of the target population. Bias may result if the sample of participants included in the study yields a different result than what would have been obtained if the entire population were enrolled. Consider whether those selected to participate in the study have higher or lower exposures, or are more or less sick, than those not participating.

A classic example of biased participant selection is the vinyl chloride epidemiology initially commissioned by manufacturers. As explained in Deceit and Denial: The Deadly Politics of Industrial Pollution, the cohort under examination excluded some workers with the longest exposure to the monomer while including younger workers with marginal exposure. As a result, the manufacturers were able to tout an overall mortality rate 75 percent that of the general population—the “healthy worker effect”—and therefore “no excess” cancer deaths, despite increasing liver and brain cancer incidence with time of exposure.

Establishing study groups. In comparative evaluations, throughout the course of the study the main difference between the groups should be the factor that is being tested or evaluated. For example, for a study that evaluates the effects of noise on hearing loss, the magnitude of the noise exposure should be the only difference between the exposed group and the reference group. For observational studies, which do not benefit from randomizing participants to groups, the way in which groups are matched plays an important role in mitigating potential effects of confounding variables. Statistical adjustments are often used to control for confounding and approximate the comparable status of the groups.

A potential problem for studies that assess the effects of an occupational exposure is incorrect categorization of participants by exposure status. Where applicable, consider the source and the quality of participants’ work history, the process of identifying exposure groups or job exposure matrices, and the process of estimating exposures (for example, quantitative measurements, modeling techniques, professional judgment, or self-reported survey data). If quantitative measurements have been obtained, consider the sampling strategy, the number of measurements, the analytical methods used, and whether changes in methods over time may be important to the exposure assessment effort.

Maintaining the study groups. If study conditions are likely to change, the groups may not remain comparable. In settings where exposures to occupational hazards may change over time, consider how participants’ exposures may be affected throughout the study’s follow-up period. Could these changes introduce confounding issues that affect interpretation?

  INTERPRETING RESULTS The results section of a primary research manuscript is the core of the publication. All other sections are built around the collected and analyzed data. It can be helpful to scrutinize each table and figure: do they report raw data? If the data have been transformed, does the study explain how and why? What statistical tests or methods are applied?

Studies publicized in news outlets garner attention to the study team and sponsor, potentially motivating both presentation of the data and the study narrative. Information in the actual figures and tables, versus the text description, may be informative for gaining a better understanding of the scientific findings.

If the findings appear to show an effect, the effect may be real or due to chance. There are generally two ways that statistics are used to evaluate whether the results were due to chance: hypothesis testing (p-values) and estimation (confidence intervals). P-values and confidence intervals are used to express confidence that the test value is correct.

A typical p-value of < 0.05 conveys a less than 5 percent chance that the results are coincidental. The 95 percent confidence interval (CI) expresses the range of values that likely contains the real result. A tighter range is most often associated with a smaller standard deviation, a larger sample size, or both. The 99 percent CI would be wider than a 95 percent CI. If the interval contains the value that indicates no effect (that is, one for a ratio or zero for a difference), the association is not statistically significant. For example, an interval of 0.38 - 1.8 for an odds ratio or relative risk would indicate the association is not significant. Conversely, when the interval does not contain the value that indicates no effect, we can assume the results are statistically significant.

While we must carefully consider whether a study is relevant and valid before deciding to use the results, we must also balance the need to adopt protective measures based on an evolving body of scientific information. The threshold of evidence that you need to support a decision may vary. In the absence of a strong association between exposures and adverse health effects from a well-designed study, a preliminary indication of a possible association may be sufficient for you to err on the side of caution. The industrial hygienist responsible for activities such as hazard communication and working with the occupational physician to oversee the medical surveillance program must stay informed as the science evolves. As the example of formaldehyde demonstrates, illustrating the association between exposure and lymphohematopoietic malignancies involved comprehensive investigations that took time. Neither the IH nor the physician would have the option to go back in time and modify their actions should it later be discovered that they were not commensurate with the available risk information. TOOLS FOR APPRAISAL Having the tools to identify and critically appraise relevant research is key to effective decision-making. In general, a good quality study is one that is designed to prevent the findings from being affected by error, confounding factors, or bias. These potential influences, as ever-present challenges, do not necessarily negate results or conclusions but should inform interpretation of the collected data. Helpfully, having a keen eye for analytical processes and measurement techniques is the industrial hygienist’s area of expertise. Consideration of the issues commonly considered within study evaluations can be useful in gathering evidence for increased or decreased confidence in a study’s conclusions.

Send feedback to The Synergist.