Abstract

The rounding rule in computation is based on the principle that no result of computation can have more significant figures than the original data. Use of this principle in statistics is ill-advised and leads to loss of information by rounding. Rules for determining the maximum number of figures to be retained in simple statistical calculations are developed. Let m be the number of figures in the maximum plus the minimum values of the sample of n observations, where n = f x 10 k (f is a fractional number between 1 and 10 and, is an integer). Then: A) for samples of at least 100 the maximum number of figures will be: a) m + k or m + k + 1 for the sum and the mean, b) 2m + k, 2m + k + 1 or 2m + k + 2 for the variance, c) one less than the variance for the standard deviation; B) for smaller samples the maximum number of figures will be: a) m or m + 1 for the sum and the mean, b) 2m, 2m + 1 or 2m + 2 for the variance, c) one less than the variance for the standard deviation; C) the first figures in the variance and standard deviation may be zeros; D) when a calculated statistic is to be used in further calculations it is preferable not to round it first; E) the sum and sum of squares should be always included with summary statistics, to prevent ambiguity. Analogous methods can be readily applied to calculations of other statistics.

First Page Preview

First page PDF preview
You do not currently have access to this article.