Support reliability and agreement measures

[mllg]

Some can be found in the irr and irrCAC packages.

[FBerding]

Here is an example for Krippendorff's Alpha with irr
`
library("irr")
true_labels<-cbind(0,1,0,1,2,3)
assigned_labels<-cbind(0,1,2,2,1,3)

#for nominal data (especially useful for text classification)
k_alpha<-kripp.alpha(rbind(true_labels,assigned_labels),method="nominal")$value
#for ordinal data (especially useful for text classification)
k_alpha<-kripp.alpha(rbind(true_labels,assigned_labels),method="nominal")$value
#for metric data
k_alpha<-kripp.alpha(rbind(true_labels,assigned_labels),method="nominal")$value

for kripp.alpha the raters are in the rows and the labels are in the columns