A critical feature of any analytical and statistical software is accuracy. You are making decisions based on the statistics obtained and you need to know you can rely on them.
We have documented our software development and validation process previously, but another good benchmark to test statistical software against is the NIST StRD. The Statistical Engineering and Mathematical and Computational Sciences Divisions of NIST’s Information Technology Laboratory developed datasets with certified values for a variety of statistical methods against which statistical software packages can be benchmarked. The certified values are computed using ultra-high precision floating point arithmetic and are accurate to 15 significant digits.
For more information about the NIST StRD see:
https://www.itl.nist.gov/div898/strd/
We tested version 4.00 of Analyse-it against the NIST StRD on an Intel Xeon dual processor PC.
No statistical package achieves perfect accuracy for all the tests and no one package performs best for every test. Most statistical packages use IEEE754 double precision (64bit) floating point arithmetic and due to finite precision, round-off, and truncation errors in numerical operations, are unable to obtain the exact certified value.
In the tests:
- Analyse-it performed consistently amongst the best on all tests.
- Analyse-it performed better than some of the most popular well-known statistical packages.
For more information on the performance of Analyse-it against the NIST StRD, and to download the workbooks containing the analyses, see:
Comments
I'm a PhD in computer science and an entrepreneur, I work in the field of numerical accuracy of computer calculations for critical applications. Your approach is very interesting, alors I didn't know about this database so thanks a lot for that. But I was wondering, is there a more recent one? It looks like this one is rather old. Is there no demand at all for a new benchmark on this topic ?
Sincerely,
Arnault
Comments are now closed.