Agreement Between Ordinal Scale

Agreement Between Ordinal Scale

An early study was conducted by Holmquist et al [11] to study variability in cervical cancer classifications from histological slides and to assess the overall level of compliance among pathologists. Seven pathologists evaluated and ranked 118 slides on an ordinal scale of five categories: (1) negative; (2) hyperplasia of the atypical disc; (3) on-site carcinoma; (4) plaque carcinoma with early electrical invasion; and (5) invasive cancer. A table of each classification, made by each pathologist, is presented in Landis and Koch [40]. The initial analysis, presented in Holmquist et al [11], focused on the study of the number of slides between each pair of pathologists considered higher by a pathologist in the couple. Table 5 presents the results of the adaptation of the GLMM model and the proposed synthesis measures, as well as existing synthesis measures. Based on the seven classifications of pathologists, the probability of being evaluated in categories 1 (negative) to 5 (most severe cancers) was 27%, 17%, 52%, 3% and 1.7%, suggesting that only a small portion of the slides indicate a more serious disease. The variability in classifications (`u2-4.13`) is significant in relation to the variability observed in pathologists (`v2-0.627`) and gives a moderately high value of `0.717. The Kappa -M-0.266 model (se – 0.032) indicates only a small, timed match of the seven pathologists. Each of the synthesis measurements based on Cohen`s cappa, fleiss` kappa, Conger`s and Light`s kappa and Cohens glmm-kappa yielded a slightly higher, chance-adjusted match (F-0.354, «LC-0.361» and «GLMM-0.296»).

This can be attributed, in the same way as the Gleason Grading example in Section 6.1, to prevalence effects in which serious pathologies (categories 4 and 5) are very rarely observed in this slideshow. Mielke et al`s kappa [42.47] is very small at 0.127, which shows the unlikely scenario that the seven pathologists would assign an identical classification to a given slide. The proposed basic approach uses the framework of generalized ordinal linear mixed linear linear models and generates a measurement based on a summary model for concordances for ordinal classifications based on variance components for non-observable variables. Unlike most other available synthesis measures, our proposed agreement corrects random agreement in a different formulation than Cohen Kappa and is therefore not affected by the prevalence of the disease. Previous work has demonstrated the use of variance components in match studies for binary classifications (e.g. B sick to non-sick classifications) [26.27] and ordinal classifications [23,28,29], in which the orderly nature of classifications presents a unique set of challenges for the estimation and modeling process beyond binary classifications. Our methods can be expanded to incorporate the characteristics of experts and subjects that may influence the agreement. Unbalanced observations are permitted, as not all experts rank all subjects in the sample. The characteristics of the various experts in the study can be analyzed and the nature of the population-based approach ensures that it is possible to draw conclusions about the agreement between the typical experts and the themes of their underlying populations, not just on experts and themes that were included in the study.

Comments are closed