This paper has values of A' computed using the AUC (area under the ROC curve) version of A' rather than the Wilcoxon version. All known implementations of AUC as of this writing are inaccurate for some special cases. A particularly common situation where errors occur is when multiple data points have the exact same confidence, which is common for many data mining algorithms, particularly decision trees and decision rules. This specific error is confirmed as of this writing for RapidMiner and R. SPSS has other errors as well. These errors inflate values of A'. This paper therefore has inaccurate, inflated values of A'.

More accurate A' estimation is available, using the O(N^2) Wilcoxon implementation, is available at

This code is confirmed to be acceptably fast for data sets up to hundreds of millions of data points.