This came up as a rather entertaining political science question the other day. "It is better for N many criminals to go free than for 1 innocent person to be punished." This concept goes way back to the 18th century, originated by English judge William Blackstone. It is now known as Blackstone's Ratio in criminal law.
A few years ago, the National Center for State Courts ran an experiment where they compared cases when both the judge and the jury could submit guilty/not-guilty verdicts. Through signal analysis, they could predict not only what percentage of the time they disagreed, but predict who was wrong. The results pointed to approximately 17% of the jury verdicts being incorrect and "N" equaling roughly 1.43 guilty parties let go per innocent punished. On the other hand, about 12% of the judge's verdicts were incorrect leading to an N of 0.1 (1 guilty person let go for every 10 punished innocent people).1
Blackstone's pick for N was 10. My assumption for the reason behind the change in this ratio is that in the last 200 years, with tools such as modern forensic evidence, DNA sampling, fiber testing and omnipresent video cameras, we have made significant strides in being able to exonerate innocent people before the fact, and only bring guilty parties before the court.
In data security, we're continually bombarded with "false positives." We get false positives when our tools are set to be too sensitive – but most admins prefer this to the alternative of having them not be sensitive enough and miss an event entirely! This is not a new problem – what is new is that our tools are evolving in a way to reduce the amount of alerts we receiving, letting us take more time to analyze the ones that really need our attention.
As technology advances, we'll continue to lower the number of false positives we get, improving our organization's Blackstone Ratio – and this ratio is something that you can measure and prove to others that your security is improving over time. In the last year at GSI, we've dropped our false positives by 73.6% through reconfiguring and tuning our current monitoring systems. Additionally, we recently installed more security appliances that are even more accurate, so I expect this trend to continue. All of this adds up to data center security that's more accurate, more effective – and more measurable.
Footnotes
1. Spencer, Bruce, On Measuring the Balance between Wrongful Convictions and Wrongful Acquittals in Criminal Trials (November 7, 2007). 2nd Annual Conference on Empirical Legal Studies Paper. Available at SSRN: http://ssrn.com/abstract=997188