Different measures of Inter-rater reliability and when to use them: Cohen’s kappa, Fleiss’ kappa, Scott’s Pi, Krippendorff’s alpha. See also the R code.
My research interests include data science for public good, quantum physics and psychology.