Abstract:

International standard guidelines recognise that data from a precision experiment should be checked for outliers as a matter of routine before finalising the precision estimates. The reason is that results affected by uncontrolled variation in the analytical procedure occur at the rate of a few percent. In a precision experiment containing 20 or so results, even a single outlier can have deleterious effects on the precision estimates. Dedicated tests are recommended to identify which experimental unit has method replicates with unusually large deviation – the current preferred approach is Cochran's test described in ISO 5725-2:1994. Cochran's test can only detect a single experimental unit with unusually large replicates (that one with the maximum deviation between replicates), and so this test must be iterated after excluding the identified experimental unit to determine the presence of additional unusually large replicate deviations. An alternative approach is to use the model of the precision experiment directly to detect and identify outliers. This approach is traditionally called ‘model diagnostics’ or ‘residual analysis’. Model diagnostics is commonly prescribed as “a step” in data analysis in texts on linear regression and analysis of variance (ANOVA). This approach can simultaneously detect and identify multiple experimental units which exhibit unusually large deviation between method replicates. We illustrate this approach using data obtained from

a homogeneity study under a two-stage fully nested random effects ANOVA design conducted at the International Atomic Energy Agency (IAEA) in 2014 on a uranium bearing material. The measurands are elemental impurity concentrations relative to uranium determined by ICP-MS.

 

Highlights:

• The IAEA's standard procedure for purity analysis can only produce 8 determinations in a single analyst working day. To achieve the minimum required m = 10 samples, the samples in the study were analysed over the course of multiple days. Results within each day are subject to a unique instrument calibration and possible significant calibration uncertainty. This violates the required method repeatability conditions.
• The study adopted a two-stage fully nested random effects ANOVA in order to control for variability attributable to calibration.
• The model diagnostic approach addresses several shortcomings of Cochran's test:

1. The diagnostics approach appeals directly to the assumption that method replicates are Gaussian distributed (an assumption which is required by Cochran's test) and uses this assumption to identify anomalies.
2. Cochran's test can only test for ‘one anomaly at a time’, specifically that bottle with the largest difference, and must be iterated. In contrast the diagnostics approach can show simultaneously all bottles with unusually large within-bottle differences. Therefore, in a single application, the analyst can immediately learn how many experimental units and which ones are affected by unusually large differences between method replicates.
3. As Cochran's test necessarily assumes that the design is balanced (i.e. each bottle has the same number of replicates) both replicates from a bottle must be excluded to iterate Cochran's test. The diagnostic approach does not require this assumption and can be applied to unbalanced ANOVA models.

• Model diagnostics was applied to a homogeneity study for scandium and praseodymium in a high purity uranium material following four steps in the data analysis:

1. Initial data analysis by graphical inspection and initial ANOVA
2. Diagnostic analysis identifying anomalies and outliers.  Inspect for assignable cause and decide on removal.
3. Update ANOVA calculation and graphic after outlier removal.
4. Diagnostic analysis after outlier removal to justify final model fit.

 

Citation:

Walsh, S. J., Macsik, Z., Wegrzynek, D., Krieger, T., & Boulyga, S., “Model diagnostics for detecting and identifying method repeatability outliers in precision studies: application to a homogeneity study under a two-stage nested ANOVA”, Journal of Analytical Atomic Spectrometry, (2016), 31(3), 686-699.