Choosing a laboratory is an important decision for you, your team, and your organization. High-performing laboratories are vital to the protection of worker health and safety. For this reason, many laboratories routinely undergo proficiency testing to make sure they provide you with reliable data.

For many years, AIHA Proficiency Analytical Testing (PAT) Programs has offered services integral to the work of occupational and environmental health and safety professionals. In 1974, the administration of the Industrial Hygiene Proficiency Analytical Testing (IHPAT) Program established AIHA as a leader in proficiency testing. Since then, PAT Programs has expanded to eight different proficiency testing areas, including industrial hygiene-focused programs that evaluate the analytical capability of laboratories to test for asbestos, metals, organic solvents, and silica in air; to test for lead in air, dust, paint, and soil; and to identify microbiological cultures and spore images. Our long history of proficiency testing and deep understanding of the needs of OEHS professionals positions us to provide meaningful programs to the industry. Now, we are using that knowledge to introduce a new proficiency testing program for the gravimetric determination of airborne particulates.

THE IMPORTANCE OF PROFICIENCY TESTING Have you taken a moment to consider why it matters that PAT Programs continuously improve and expand our offerings? It’s because data is paramount to what you do. How can you protect those you are responsible for if you don’t have reliable information about contaminants of concern?

For those who aren’t familiar with proficiency testing, here’s how it works. PAT Programs ships samples for analysis to participating laboratories on a quarterly or tri-annual basis; the frequency varies depending on the program. The samples contain the analytes of interest. The laboratories analyze their samples and report their results so PAT Programs can determine their proficiency. The PAT Programs Board, which comprises qualified technical experts, reviews and approves all summary data for each round. Then the scores are finalized and the results distributed to participants.

Proficiency testing is an easy way for you to confirm that your laboratory partner is providing you with valid data. We know you are already using an accredited laboratory, so you trust the data, but proficiency testing gives you a way to routinely verify the analytical capability of your laboratory partner. Ask for a PAT report and use it as another tool in your toolkit for keeping people healthy and safe.

WHY AIRBORNE PARTICULATES? As a proficiency testing provider accredited to ISO/IEC 17043, Conformity Assessment—General Requirements for Proficiency Testing (A2LA Certificate 3300.01), PAT Programs routinely requests feedback from our participants. One of the standard questions we ask is what, if any, gaps our participants have identified in the proficiency testing services available to their laboratory. In 2019, feedback indicated a clear need for proficiency testing for laboratories that analyze samples for airborne particulates using NIOSH method 0500, Particulates Not Otherwise Regulated, Total; NIOSH 0600, Particulates Not Otherwise Regulated, Respirable; or NIOSH 5000, Carbon Black. The resulting market research showed a large enough population of laboratories to support the investigation of a new program, so I facilitated the development of a detailed research and development plan. From 2020 through 2021, PAT Programs staff, the PAT Board, and experts at RTI International, the IHPAT sample generator, worked together to develop a sample generation process that produced homogeneous and stable samples, as required by ISO/IEC 17043. But internal determination of those properties is not enough for us. We needed to ensure the samples performed well after shipping, handling, storage, and analysis at participating laboratory locations.

To get real-world usage data on the samples, PAT Programs administered two pilot rounds. The following discussion pulls back the curtain on the science of planning, developing, and validating a proficiency testing product.

PLAN, DO, CHECK, ACT The planning stages of the airborne particulate program began with a review of the common analytical methods to gather information on the sampling media and analytical ranges. One aspect of the NIOSH 0500, 0600, and 5000 methods that posed a challenge was the need for a tare weight of the filter before the sampling process takes place. Other gravimetric-based proficiency testing programs do not match the sampling method OEHS professionals are performing in the field because they use a different filter type and the concentration range is well above the range of concern. For laboratories using other proficiency testing programs, the administrative burden is high because it may require a back-and-forth mailing of the filter to obtain the tare weight prior to sample generation. We wanted to take an approach that was easy for laboratories to use and matched the sampling you are doing. But which was the better option: providing the tare weight directly, or providing matched-weight filters to use in lieu of a tare weight? This question would be the focus of Pilot Round 1.

Based on prior experience, the development team hypothesized two additional challenges to the technical feasibility of creating an airborne particulates program using 37 mm, 5 μm PVC filters:

1. NIOSH 0500 notes that “[static] electricity can cause erroneous weight readings.” Would the effects of static electricity interfere with the use of an automated weighing system? An automated weighing system is the most efficient way to determine tare weight and perform validation testing of the generated pilot samples. This technical feasibility question was a go/no-go area of investigation, as automated weighing provides both highly reproducible measurements, which are integral to a gravimetric-based program, as well as lower labor cost per filter than manual weighing.

2. Unlike 46.2 mm PTFE filters used for EPA PM 2.5 ambient air monitoring, the 37 mm, 5 μm PVC filters do not have a support ring. Again, the go/no-go question was whether the automated weighing system could manage the filters effectively without a significant amount of filter loss due to damage or incidents of filters becoming lodged in the automated weighing system machinery.

After sending ten blank PVC filters through the automated weighing system and utilizing five duplicate readings per filter, it was determined that neither static electricity nor the lack of a stability ring posed a significant risk to program development.

Ultimately, using two pilot study rounds, we were able to create a program that streamlines the proficiency testing process by providing all the information needed to determine the net mass loading and uses the same media (37 mm, 5 μm PVC filter) specified in NIOSH 0500, 0600, and 5000. The range of the program is 0.5–5 mg per filter, with a plan to evaluate performance and potentially lower the range to 2 mg, matching the upper range of the NIOSH methods, in the future.

In addition, the first pilot round evaluated the impact on the net mass loading (measured in milligrams) when presenting the laboratory with either a filter tare weight or a matched-weight (MW) filter set where one filter would serve as the tare weight value and the other would be the loaded sample. (As its name suggests, an MW filter is matched to the one in which the sample is generated, with a stated tolerance between the filters of up to 25 µg.) Each laboratory that participated in Pilot Round 1 was assigned to either the tare group or the MW group. To obtain the net mass loading, laboratories would subtract either the tare weight or the weight of the MW filter from the total weight of the filter that contains the sample, depending on group assignment. Either option would have provided a solid proficiency testing program, but the development team thought the MW option might reduce sample generation time because the samples would not require the labor hours to determine the tare weight using the automated weighing system. The logic was sound: fewer production steps and fewer labor hours lower the potential cost of generating samples, which lowers the costs of participating in the proficiency testing program. This potential labor cost savings of MW filters would be evaluated against the material cost of sending only a single loaded filter per sample with the tare weight option. The MW filters, sent as a set of two, are significantly more expensive than the single tare filter needed per sample.

To gather data on the aspects of concern and the technical success of the samples, PAT Programs sought 40 volunteer participants for Pilot Round 1. For the determination of net mass loading in mg, 20 pilot participants received a tare value and 20 received a matched-weight filter. A proprietary liquid deposition process was used to generate pilot samples. The concentration target for the samples was 0.2 mg and 1.5 mg for samples 1 and 2, respectively.

Once data submission for Pilot Round 1 closed, we evaluated data for each group—the tare group, the MW group, and all participants—using standard IHPAT statistics. We found a statistically significant difference between the mean values generated by the tare group and the MW group. This difference was consistent with homogeneity data generated by quality-control procedures performed prior to shipment of the samples to pilot participants.

To evaluate the pilot round, PAT Programs applied IHPAT scheme plan statistics, which required a minimum relative standard deviation (RSD) of 4 percent during data processing. Following this requirement, the RSDs for sample 1 for the tare group, and sample 2 for both the MW group and the tare group, were increased to 4 percent (see Table 1).

In a traditional proficiency testing round, the clear bimodality in the population data would have required PAT Programs to treat each group type as a separate population. As recommended in ISO/IEC 17043 as well as ISO 13528, Statistical Methods for Use in Proficiency Testing by Interlaboratory Comparisons, we removed visual outliers prior to statistical treatment.

Using original relative standard deviation (ORSD) as a performance indicator of the tightness of the datasets and considering the visual variability within the sample generation types, it was clear that the tare group performed better from a proficiency testing perspective (see Figure 1). Pilot Round 1 did not contain enough data to evaluate the influence of method selection on the summary statistics due to the high proportion of laboratories using NIOSH 0500. For this reason, the impact of method selection on the results was minimal and unlikely to be a concern during the evaluation of participants in future rounds.

To perform a stability study, pilot participants re-weighed and reported Pilot Round 1 sample values when submitting Pilot Round 2 data. Thirty-one pilot participants reported data from the first pilot round samples. The average difference of the filter total weight between weightings skewed slightly negative, averaging –0.0034 mg (1.62 percent of the 0.209 mg mean) for sample 1 and –0.0005 mg (0.03 percent of the 1.463 mg mean) for sample 2. Tare weight samples clustered more tightly around 0.00 mg, indicating less difference over time than the matched-weight filters, further supporting tare-provided sample generation as the mode of production. A one-way analysis of variance (ANOVA) determined that there was no statistically significant difference between the net mass loading means of the sample 1 and sample 2 results submitted during the first and second pilot rounds.

The development team also determined that the savings in labor time from use of the MW filters were not enough to offset their higher materials costs. So, the tare weight option not only led to better laboratory performance, it was also more efficient.

Only a few samples were shipped outside the U.S., but the data in Pilot Round 2 show that shipping long distances likely has little to no impact on sample effectiveness (see Figure 2).

As an OEHS professional, you make extremely important decisions almost every day. You understand and appreciate how continuous improvement of your processes, equipment, and skillset impact those decisions. Make sure your laboratory partner values continuous improvement as much as you do: encourage them to enroll in the new airborne particulates program, and be sure to ask for a PAT report in late 2022 to verify their performance.

Acknowledgements: PAT Programs thanks participants who volunteered their time and equipment for the pilot studies and the PAT Board for their feedback and guidance during the planning of the pilot studies. We would not have been able to complete this project without our sample generation partner, RTI International, which developed, generated, verified, packaged, and shipped all pilot samples, under contract with PAT Programs.

Send feedback to The Synergist.